2023-07-18T03:09:35.860 INFO:root:teuthology version: 0.0.1.dev147+gff9991b 2023-07-18T03:09:35.861 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747 --verbose --name rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747/orig.config.yaml 2023-07-18T03:09:35.907 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T03:09:35.992 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747 branch: wip-rishabh-2023Jul13-m-quota description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} email: null first_in_suite: false job_id: '7342747' kernel: client: branch: testing flavor: default kdb: true sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul13-m-quota ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13-m-quota sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13-m-quota suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi049.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots - tasks.cephfs.test_volumes.TestSubvolumeSnapshots teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b timestamp: 2023-07-18_02:11:50 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-18T03:09:35.993 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa; will attempt to use it 2023-07-18T03:09:35.994 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks 2023-07-18T03:09:35.994 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-18T03:09:35.996 INFO:teuthology.task.internal:Checking packages... 2023-07-18T03:09:36.026 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '81cf785aa7781c4e08381c540f890cb9bc987ef9' 2023-07-18T03:09:36.026 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-18T03:09:36.027 INFO:teuthology.packaging:ref: None 2023-07-18T03:09:36.027 INFO:teuthology.packaging:tag: None 2023-07-18T03:09:36.028 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13-m-quota 2023-07-18T03:09:36.028 INFO:teuthology.packaging:sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:09:36.028 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13-m-quota 2023-07-18T03:09:36.333 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4746.g81cf785a 2023-07-18T03:09:36.333 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-18T03:09:36.341 INFO:teuthology.task.internal:no buildpackages task found 2023-07-18T03:09:36.341 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-18T03:09:36.351 INFO:teuthology.task.internal:Saving configuration 2023-07-18T03:09:36.369 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-18T03:09:36.377 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T03:09:36.408 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 03:03:08.163847', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-18T03:09:36.436 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 03:03:08.162157', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-18T03:09:36.436 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-18T03:09:36.472 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-07-18T03:09:36.472 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-07-18T03:09:36.472 INFO:teuthology.run_tasks:Running task console_log... 2023-07-18T03:09:36.557 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f69889f54c0>, signals=[15]) 2023-07-18T03:09:36.559 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-18T03:09:36.566 INFO:teuthology.task.internal:Opening connections... 2023-07-18T03:09:36.566 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:09:36.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:09:36.638 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:09:36.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:09:36.705 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-18T03:09:36.712 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2023-07-18T03:09:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2023-07-18T03:09:36.733 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2023-07-18T03:09:36.793 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="Red Hat Enterprise Linux" 2023-07-18T03:09:36.793 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="8.6 (Ootpa)" 2023-07-18T03:09:36.793 INFO:teuthology.orchestra.run.smithi049.stdout:ID="rhel" 2023-07-18T03:09:36.793 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="fedora" 2023-07-18T03:09:36.793 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="8.6" 2023-07-18T03:09:36.793 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el8" 2023-07-18T03:09:36.794 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-18T03:09:36.794 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2023-07-18T03:09:36.794 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-18T03:09:36.794 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://www.redhat.com/" 2023-07-18T03:09:36.794 INFO:teuthology.orchestra.run.smithi049.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-18T03:09:36.794 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-18T03:09:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:09:36.795 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-18T03:09:36.795 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-18T03:09:36.795 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-18T03:09:36.795 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-18T03:09:36.796 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2023-07-18T03:09:36.821 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2023-07-18T03:09:36.842 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2023-07-18T03:09:36.842 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2023-07-18T03:09:36.901 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="Red Hat Enterprise Linux" 2023-07-18T03:09:36.902 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="8.6 (Ootpa)" 2023-07-18T03:09:36.902 INFO:teuthology.orchestra.run.smithi107.stdout:ID="rhel" 2023-07-18T03:09:36.903 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="fedora" 2023-07-18T03:09:36.903 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="8.6" 2023-07-18T03:09:36.903 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el8" 2023-07-18T03:09:36.903 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-18T03:09:36.904 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2023-07-18T03:09:36.904 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-18T03:09:36.904 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://www.redhat.com/" 2023-07-18T03:09:36.905 INFO:teuthology.orchestra.run.smithi107.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-18T03:09:36.905 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-18T03:09:36.905 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:09:36.905 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-18T03:09:36.906 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-18T03:09:36.906 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-18T03:09:36.906 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-18T03:09:36.907 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2023-07-18T03:09:36.930 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-18T03:09:36.939 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-18T03:09:36.945 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-18T03:09:36.945 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2023-07-18T03:09:36.948 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2023-07-18T03:09:36.966 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-18T03:09:37.005 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-18T03:09:37.005 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2023-07-18T03:09:37.007 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2023-07-18T03:09:37.032 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-18T03:09:37.096 INFO:teuthology.run_tasks:Running task kernel... 2023-07-18T03:09:37.111 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'} 2023-07-18T03:09:37.112 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-07-18T03:09:37.112 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-07-18T03:09:37.113 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-18T03:09:37.114 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-18T03:09:37.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:09:37.133 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-07-18T03:09:37.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:09:37.136 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-07-18T03:09:37.518 DEBUG:teuthology.task.kernel:sha1 for client.1 is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-18T03:09:37.519 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-18T03:09:37.519 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2023-07-18T03:09:37.523 DEBUG:teuthology.task.kernel:sha1 for client.0 is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-18T03:09:37.523 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-18T03:09:37.524 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2023-07-18T03:09:37.539 INFO:teuthology.orchestra.run.smithi107.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-18T03:09:37.540 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-18T03:09:37.541 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-18T03:09:37.541 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum upgrade -y linux-firmware 2023-07-18T03:09:37.543 INFO:teuthology.orchestra.run.smithi049.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-18T03:09:37.544 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-18T03:09:37.544 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-18T03:09:37.544 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum upgrade -y linux-firmware 2023-07-18T03:09:38.741 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2023-07-18T03:09:38.742 INFO:teuthology.orchestra.run.smithi049.stdout:Unable to read consumer identity 2023-07-18T03:09:38.772 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-18T03:09:38.772 INFO:teuthology.orchestra.run.smithi107.stdout:Unable to read consumer identity 2023-07-18T03:09:40.437 INFO:teuthology.orchestra.run.smithi049.stdout:Copr repo for python3-asyncssh owned by ceph 7.1 kB/s | 3.5 kB 00:00 2023-07-18T03:09:40.534 INFO:teuthology.orchestra.run.smithi107.stdout:Copr repo for python3-asyncssh owned by ceph 8.5 kB/s | 3.5 kB 00:00 2023-07-18T03:09:45.018 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 16 MB 00:04 2023-07-18T03:09:48.418 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 436 kB/s | 24 kB 00:00 2023-07-18T03:09:48.446 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 2.0 MB/s | 16 MB 00:07 2023-07-18T03:09:49.129 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2023-07-18T03:09:49.130 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2023-07-18T03:09:49.130 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2023-07-18T03:09:49.235 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.1... 2023-07-18T03:09:49.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-18T03:09:49.474 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ 2023-07-18T03:09:49.475 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-18T03:09:49.477 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi107.front.sepia.ceph.com to complete... 2023-07-18T03:09:52.095 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 658 kB/s | 24 kB 00:00 2023-07-18T03:09:52.829 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2023-07-18T03:09:52.830 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2023-07-18T03:09:52.830 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2023-07-18T03:09:52.967 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.0... 2023-07-18T03:09:52.968 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-18T03:09:53.329 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ 2023-07-18T03:09:53.330 DEBUG:teuthology.orchestra.run.smithi049:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-18T03:09:53.332 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi049.front.sepia.ceph.com to complete... 2023-07-18T03:14:56.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18 03:14:56 URL:https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm [380684764/380684764] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-18T03:14:56.918 INFO:teuthology.orchestra.run.smithi049.stderr:FINISHED --2023-07-18 03:14:56-- 2023-07-18T03:14:56.919 INFO:teuthology.orchestra.run.smithi049.stderr:Total wall clock time: 5m 4s 2023-07-18T03:14:56.919 INFO:teuthology.orchestra.run.smithi049.stderr:Downloaded: 1 files, 363M in 5m 3s (1.20 MB/s) 2023-07-18T03:14:56.920 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.0... 2023-07-18T03:14:56.921 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-18T03:14:58.707 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying... ######################################## 2023-07-18T03:14:58.764 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing... ######################################## 2023-07-18T03:14:58.852 INFO:teuthology.orchestra.run.smithi049.stdout:Updating / installing... 2023-07-18T03:15:13.479 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18 03:15:13 URL:https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm [380684764/380684764] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-18T03:15:13.479 INFO:teuthology.orchestra.run.smithi107.stderr:FINISHED --2023-07-18 03:15:13-- 2023-07-18T03:15:13.479 INFO:teuthology.orchestra.run.smithi107.stderr:Total wall clock time: 5m 24s 2023-07-18T03:15:13.479 INFO:teuthology.orchestra.run.smithi107.stderr:Downloaded: 1 files, 363M in 5m 24s (1.12 MB/s) 2023-07-18T03:15:13.480 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.1... 2023-07-18T03:15:13.481 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-18T03:15:15.099 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying... ######################################## 2023-07-18T03:15:15.149 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing... ######################################## 2023-07-18T03:15:15.247 INFO:teuthology.orchestra.run.smithi107.stdout:Updating / installing... 2023-07-18T03:15:39.388 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-6.5.0_rc1_ga440c79eee02-1 ######################################## 2023-07-18T03:15:53.130 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-6.5.0_rc1_ga440c79eee02-1 ######################################## 2023-07-18T03:16:04.068 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi049.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-18T03:16:04.069 DEBUG:teuthology.orchestra.run.smithi049:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-18T03:16:04.120 INFO:teuthology.orchestra.run.smithi049.stdout:/boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:04.121 INFO:teuthology.orchestra.run.smithi049.stdout:/boot/config-6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:04.121 INFO:teuthology.orchestra.run.smithi049.stdout:/boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:04.121 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:04.121 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel 2023-07-18T03:16:04.121 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch 2023-07-18T03:16:04.122 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86 2023-07-18T03:16:04.122 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto 2023-07-18T03:16:04.122 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-18T03:16:04.122 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-18T03:16:04.122 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-18T03:16:04.122 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-18T03:16:04.122 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-18T03:16:04.122 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-18T03:16:04.123 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-18T03:16:04.123 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-18T03:16:04.123 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-18T03:16:04.123 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-18T03:16:04.123 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-18T03:16:04.123 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-18T03:16:04.123 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-18T03:16:04.124 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-18T03:16:04.124 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-18T03:16:04.124 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-18T03:16:04.124 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-18T03:16:04.124 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-18T03:16:04.124 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-18T03:16:04.124 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-18T03:16:04.124 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel 2023-07-18T03:16:04.125 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu 2023-07-18T03:16:04.125 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce 2023-07-18T03:16:04.125 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-18T03:16:04.125 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm 2023-07-18T03:16:04.125 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-18T03:16:04.125 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-18T03:16:04.125 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-18T03:16:04.125 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto 2023-07-18T03:16:04.126 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-18T03:16:04.126 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-18T03:16:04.126 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-18T03:16:04.126 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx 2023-07-18T03:16:04.126 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-18T03:16:04.126 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-18T03:16:04.126 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-18T03:16:04.127 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-18T03:16:04.127 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-18T03:16:04.127 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-18T03:16:04.127 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-18T03:16:04.127 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-18T03:16:04.127 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-18T03:16:04.127 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-18T03:16:04.127 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-18T03:16:04.128 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-18T03:16:04.128 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-18T03:16:04.128 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-18T03:16:04.128 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-18T03:16:04.128 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-18T03:16:04.128 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-18T03:16:04.128 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-18T03:16:04.128 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-18T03:16:04.129 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-18T03:16:04.129 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-18T03:16:04.129 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-18T03:16:04.129 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-18T03:16:04.129 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-18T03:16:04.129 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-18T03:16:04.129 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-18T03:16:04.130 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-18T03:16:04.130 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-18T03:16:04.130 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-18T03:16:04.130 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-18T03:16:04.130 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-18T03:16:04.130 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-18T03:16:04.130 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-18T03:16:04.130 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-18T03:16:04.131 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-18T03:16:04.131 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-18T03:16:04.131 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-18T03:16:04.131 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-18T03:16:04.131 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-18T03:16:04.131 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-18T03:16:04.131 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-18T03:16:04.131 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-18T03:16:04.132 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-18T03:16:04.132 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-18T03:16:04.132 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-18T03:16:04.132 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers 2023-07-18T03:16:04.132 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi 2023-07-18T03:16:04.132 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-18T03:16:04.132 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-18T03:16:04.133 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/custom_method.ko 2023-07-18T03:16:04.133 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-18T03:16:04.133 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-18T03:16:04.133 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-18T03:16:04.133 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-18T03:16:04.133 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-18T03:16:04.133 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata 2023-07-18T03:16:04.133 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-18T03:16:04.134 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-18T03:16:04.134 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ata_generic.ko 2023-07-18T03:16:04.134 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-18T03:16:04.134 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-18T03:16:04.134 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-18T03:16:04.134 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-18T03:16:04.134 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-18T03:16:04.134 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-18T03:16:04.135 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-18T03:16:04.135 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-18T03:16:04.135 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-18T03:16:04.135 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-18T03:16:04.135 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-18T03:16:04.135 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-18T03:16:04.135 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-18T03:16:04.135 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-18T03:16:04.136 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-18T03:16:04.136 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-18T03:16:04.136 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-18T03:16:04.136 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-18T03:16:04.136 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-18T03:16:04.136 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-18T03:16:04.136 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-18T03:16:04.137 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-18T03:16:04.137 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-18T03:16:04.137 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-18T03:16:04.137 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-18T03:16:04.137 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-18T03:16:04.137 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-18T03:16:04.137 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-18T03:16:04.137 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-18T03:16:04.138 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-18T03:16:04.138 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-18T03:16:04.138 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-18T03:16:04.138 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-18T03:16:04.138 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-18T03:16:04.138 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-18T03:16:04.138 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sis.ko 2023-07-18T03:16:04.138 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-18T03:16:04.139 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-18T03:16:04.139 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-18T03:16:04.139 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-18T03:16:04.139 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-18T03:16:04.139 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-18T03:16:04.139 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-18T03:16:04.139 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-18T03:16:04.139 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-18T03:16:04.140 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-18T03:16:04.140 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-18T03:16:04.140 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-18T03:16:04.140 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-18T03:16:04.140 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-18T03:16:04.140 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-18T03:16:04.140 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-18T03:16:04.141 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-18T03:16:04.141 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay 2023-07-18T03:16:04.141 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-18T03:16:04.141 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base 2023-07-18T03:16:04.141 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap 2023-07-18T03:16:04.141 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-18T03:16:04.141 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-18T03:16:04.141 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma 2023-07-18T03:16:04.142 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-18T03:16:04.142 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block 2023-07-18T03:16:04.142 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe 2023-07-18T03:16:04.142 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-18T03:16:04.142 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/brd.ko 2023-07-18T03:16:04.142 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd 2023-07-18T03:16:04.142 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-18T03:16:04.142 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-18T03:16:04.143 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/loop.ko 2023-07-18T03:16:04.143 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx 2023-07-18T03:16:04.143 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-18T03:16:04.143 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-18T03:16:04.143 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk 2023-07-18T03:16:04.143 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-18T03:16:04.143 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-18T03:16:04.143 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-18T03:16:04.144 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/virtio_blk.ko 2023-07-18T03:16:04.144 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback 2023-07-18T03:16:04.144 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-18T03:16:04.144 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkfront.ko 2023-07-18T03:16:04.144 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram 2023-07-18T03:16:04.144 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-18T03:16:04.144 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char 2023-07-18T03:16:04.144 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-18T03:16:04.145 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random 2023-07-18T03:16:04.145 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-18T03:16:04.145 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-18T03:16:04.145 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-18T03:16:04.145 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-18T03:16:04.145 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-18T03:16:04.145 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi 2023-07-18T03:16:04.145 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-18T03:16:04.146 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-18T03:16:04.146 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-18T03:16:04.146 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-18T03:16:04.146 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-18T03:16:04.146 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-18T03:16:04.146 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-18T03:16:04.146 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave 2023-07-18T03:16:04.147 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-18T03:16:04.147 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-18T03:16:04.147 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-18T03:16:04.147 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm 2023-07-18T03:16:04.147 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm.ko 2023-07-18T03:16:04.147 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-18T03:16:04.147 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-18T03:16:04.147 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-18T03:16:04.148 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-18T03:16:04.148 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-18T03:16:04.148 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/uv_mmtimer.ko 2023-07-18T03:16:04.148 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/virtio_console.ko 2023-07-18T03:16:04.148 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq 2023-07-18T03:16:04.148 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-18T03:16:04.148 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-18T03:16:04.148 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-18T03:16:04.149 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-18T03:16:04.149 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-18T03:16:04.149 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-18T03:16:04.149 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto 2023-07-18T03:16:04.149 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp 2023-07-18T03:16:04.149 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-18T03:16:04.149 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-18T03:16:04.149 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel 2023-07-18T03:16:04.150 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat 2023-07-18T03:16:04.150 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common 2023-07-18T03:16:04.150 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-18T03:16:04.150 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-18T03:16:04.150 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-18T03:16:04.150 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-18T03:16:04.150 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-18T03:16:04.151 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio 2023-07-18T03:16:04.151 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-18T03:16:04.151 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca 2023-07-18T03:16:04.151 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-18T03:16:04.151 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq 2023-07-18T03:16:04.151 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-18T03:16:04.151 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma 2023-07-18T03:16:04.151 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw 2023-07-18T03:16:04.152 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-18T03:16:04.152 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat 2023-07-18T03:16:04.152 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-18T03:16:04.152 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac 2023-07-18T03:16:04.152 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-18T03:16:04.153 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-18T03:16:04.153 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-18T03:16:04.153 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-18T03:16:04.153 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-18T03:16:04.153 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-18T03:16:04.153 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-18T03:16:04.153 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-18T03:16:04.153 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-18T03:16:04.154 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-18T03:16:04.154 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-18T03:16:04.154 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-18T03:16:04.154 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-18T03:16:04.154 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware 2023-07-18T03:16:04.154 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/edd.ko 2023-07-18T03:16:04.154 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-18T03:16:04.154 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio 2023-07-18T03:16:04.155 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-18T03:16:04.155 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-18T03:16:04.155 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid 2023-07-18T03:16:04.155 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-18T03:16:04.155 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-18T03:16:04.155 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-18T03:16:04.155 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-18T03:16:04.156 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-18T03:16:04.156 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-18T03:16:04.156 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-18T03:16:04.156 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-18T03:16:04.156 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-18T03:16:04.156 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-18T03:16:04.156 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-18T03:16:04.156 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-18T03:16:04.157 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-18T03:16:04.157 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-18T03:16:04.157 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-18T03:16:04.157 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-18T03:16:04.157 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-18T03:16:04.157 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-18T03:16:04.157 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-18T03:16:04.157 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-18T03:16:04.158 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-18T03:16:04.158 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-18T03:16:04.158 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-18T03:16:04.158 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-18T03:16:04.158 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-18T03:16:04.158 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-18T03:16:04.158 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-18T03:16:04.158 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-18T03:16:04.159 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-18T03:16:04.159 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-18T03:16:04.159 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-18T03:16:04.159 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-18T03:16:04.159 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-18T03:16:04.159 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-18T03:16:04.159 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-18T03:16:04.159 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-18T03:16:04.160 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-18T03:16:04.160 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-18T03:16:04.160 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-18T03:16:04.160 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-18T03:16:04.160 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-18T03:16:04.160 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-18T03:16:04.160 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-18T03:16:04.161 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-18T03:16:04.161 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-18T03:16:04.161 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-18T03:16:04.161 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-18T03:16:04.161 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-18T03:16:04.161 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-18T03:16:04.161 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-18T03:16:04.161 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-18T03:16:04.162 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-18T03:16:04.162 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-18T03:16:04.162 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-18T03:16:04.162 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-18T03:16:04.162 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-18T03:16:04.162 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-18T03:16:04.162 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-18T03:16:04.162 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-18T03:16:04.163 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-18T03:16:04.163 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-18T03:16:04.163 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-18T03:16:04.163 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-18T03:16:04.163 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-18T03:16:04.163 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv 2023-07-18T03:16:04.163 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-18T03:16:04.163 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-18T03:16:04.164 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-18T03:16:04.164 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon 2023-07-18T03:16:04.164 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-18T03:16:04.164 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-18T03:16:04.164 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-18T03:16:04.164 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-18T03:16:04.164 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-18T03:16:04.165 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-18T03:16:04.165 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-18T03:16:04.165 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-18T03:16:04.165 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-18T03:16:04.165 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-18T03:16:04.165 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-18T03:16:04.165 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-18T03:16:04.165 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-18T03:16:04.166 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-18T03:16:04.166 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-18T03:16:04.166 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-18T03:16:04.166 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-18T03:16:04.166 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-18T03:16:04.166 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-18T03:16:04.166 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-18T03:16:04.166 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-18T03:16:04.167 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-18T03:16:04.167 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-18T03:16:04.167 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-18T03:16:04.167 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-18T03:16:04.167 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-18T03:16:04.167 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-18T03:16:04.167 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-18T03:16:04.167 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-18T03:16:04.168 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-18T03:16:04.168 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-18T03:16:04.168 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-18T03:16:04.168 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-18T03:16:04.168 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-18T03:16:04.168 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-18T03:16:04.168 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-18T03:16:04.169 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-18T03:16:04.169 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-18T03:16:04.169 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-18T03:16:04.169 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-18T03:16:04.169 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-18T03:16:04.169 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-18T03:16:04.169 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-18T03:16:04.169 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-18T03:16:04.170 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-18T03:16:04.170 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-18T03:16:04.170 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-18T03:16:04.170 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-18T03:16:04.170 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-18T03:16:04.170 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-18T03:16:04.170 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-18T03:16:04.170 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-18T03:16:04.171 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-18T03:16:04.171 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-18T03:16:04.171 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-18T03:16:04.171 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-18T03:16:04.171 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-18T03:16:04.171 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-18T03:16:04.171 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-18T03:16:04.171 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-18T03:16:04.172 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-18T03:16:04.172 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-18T03:16:04.172 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-18T03:16:04.172 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-18T03:16:04.172 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-18T03:16:04.172 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-18T03:16:04.172 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-18T03:16:04.173 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-18T03:16:04.173 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-18T03:16:04.173 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-18T03:16:04.173 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-18T03:16:04.173 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-18T03:16:04.173 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-18T03:16:04.173 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-18T03:16:04.173 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-18T03:16:04.174 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-18T03:16:04.174 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-18T03:16:04.174 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-18T03:16:04.174 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-18T03:16:04.174 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-18T03:16:04.174 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-18T03:16:04.174 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-18T03:16:04.174 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-18T03:16:04.175 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-18T03:16:04.175 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-18T03:16:04.175 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-18T03:16:04.175 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-18T03:16:04.175 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-18T03:16:04.175 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus 2023-07-18T03:16:04.175 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-18T03:16:04.175 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-18T03:16:04.176 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-18T03:16:04.176 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-18T03:16:04.176 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-18T03:16:04.176 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-18T03:16:04.176 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-18T03:16:04.176 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-18T03:16:04.176 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-18T03:16:04.176 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-18T03:16:04.177 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-18T03:16:04.177 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-18T03:16:04.177 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-18T03:16:04.177 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-18T03:16:04.177 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-18T03:16:04.177 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-18T03:16:04.177 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-18T03:16:04.177 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-18T03:16:04.178 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-18T03:16:04.178 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-18T03:16:04.178 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-18T03:16:04.178 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-18T03:16:04.178 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-18T03:16:04.178 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-18T03:16:04.178 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-18T03:16:04.179 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-18T03:16:04.179 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-18T03:16:04.179 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-18T03:16:04.179 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-18T03:16:04.179 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-18T03:16:04.179 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-18T03:16:04.179 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-18T03:16:04.179 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-18T03:16:04.180 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-18T03:16:04.180 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-18T03:16:04.180 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-18T03:16:04.180 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-18T03:16:04.180 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-18T03:16:04.180 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-18T03:16:04.180 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-18T03:16:04.180 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-18T03:16:04.181 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c 2023-07-18T03:16:04.181 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos 2023-07-18T03:16:04.181 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-18T03:16:04.181 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-18T03:16:04.181 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses 2023-07-18T03:16:04.182 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-18T03:16:04.182 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-18T03:16:04.182 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-18T03:16:04.182 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-18T03:16:04.182 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-18T03:16:04.182 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-18T03:16:04.182 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-18T03:16:04.183 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-18T03:16:04.183 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-18T03:16:04.183 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-18T03:16:04.183 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-18T03:16:04.183 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-18T03:16:04.183 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-18T03:16:04.183 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-18T03:16:04.183 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-18T03:16:04.184 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-18T03:16:04.184 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-18T03:16:04.184 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-18T03:16:04.184 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-18T03:16:04.184 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-18T03:16:04.184 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-18T03:16:04.184 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-18T03:16:04.184 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-18T03:16:04.185 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-dev.ko 2023-07-18T03:16:04.185 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-18T03:16:04.185 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-18T03:16:04.185 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-18T03:16:04.185 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-18T03:16:04.185 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband 2023-07-18T03:16:04.185 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core 2023-07-18T03:16:04.186 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-18T03:16:04.186 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-18T03:16:04.186 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-18T03:16:04.186 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-18T03:16:04.186 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-18T03:16:04.186 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-18T03:16:04.186 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-18T03:16:04.186 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw 2023-07-18T03:16:04.187 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4 2023-07-18T03:16:04.187 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-18T03:16:04.187 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4 2023-07-18T03:16:04.187 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-18T03:16:04.187 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5 2023-07-18T03:16:04.187 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-18T03:16:04.187 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca 2023-07-18T03:16:04.187 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-18T03:16:04.188 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp 2023-07-18T03:16:04.188 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib 2023-07-18T03:16:04.188 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-18T03:16:04.188 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser 2023-07-18T03:16:04.188 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-18T03:16:04.188 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert 2023-07-18T03:16:04.188 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-18T03:16:04.188 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp 2023-07-18T03:16:04.189 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-18T03:16:04.189 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt 2023-07-18T03:16:04.189 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-18T03:16:04.189 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input 2023-07-18T03:16:04.189 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport 2023-07-18T03:16:04.189 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-18T03:16:04.189 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-18T03:16:04.189 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-18T03:16:04.190 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-18T03:16:04.190 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-18T03:16:04.190 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-18T03:16:04.190 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard 2023-07-18T03:16:04.190 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-18T03:16:04.190 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse 2023-07-18T03:16:04.190 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-18T03:16:04.191 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-18T03:16:04.191 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-18T03:16:04.191 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-18T03:16:04.191 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-18T03:16:04.191 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-18T03:16:04.191 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-18T03:16:04.191 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-18T03:16:04.191 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4 2023-07-18T03:16:04.192 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-18T03:16:04.192 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio 2023-07-18T03:16:04.192 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-18T03:16:04.192 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-18T03:16:04.192 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-18T03:16:04.192 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-18T03:16:04.192 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-18T03:16:04.192 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu 2023-07-18T03:16:04.193 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd 2023-07-18T03:16:04.193 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-18T03:16:04.193 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds 2023-07-18T03:16:04.193 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-18T03:16:04.193 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-18T03:16:04.193 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-18T03:16:04.193 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-18T03:16:04.193 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger 2023-07-18T03:16:04.194 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-18T03:16:04.194 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-18T03:16:04.194 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-18T03:16:04.194 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-18T03:16:04.194 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-18T03:16:04.194 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-18T03:16:04.194 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-18T03:16:04.195 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-18T03:16:04.195 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md 2023-07-18T03:16:04.195 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache 2023-07-18T03:16:04.195 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-18T03:16:04.195 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-18T03:16:04.195 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-18T03:16:04.195 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-18T03:16:04.195 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-18T03:16:04.196 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-18T03:16:04.196 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-18T03:16:04.196 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-18T03:16:04.196 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-18T03:16:04.196 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-18T03:16:04.196 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-18T03:16:04.196 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-18T03:16:04.196 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-18T03:16:04.197 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-18T03:16:04.197 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-18T03:16:04.197 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-18T03:16:04.197 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-18T03:16:04.197 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-18T03:16:04.197 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-18T03:16:04.197 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data 2023-07-18T03:16:04.197 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-18T03:16:04.198 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-18T03:16:04.198 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-18T03:16:04.198 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-18T03:16:04.198 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-18T03:16:04.198 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message 2023-07-18T03:16:04.198 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion 2023-07-18T03:16:04.198 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-18T03:16:04.199 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-18T03:16:04.199 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-18T03:16:04.199 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-18T03:16:04.199 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-18T03:16:04.199 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-18T03:16:04.199 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-18T03:16:04.199 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd 2023-07-18T03:16:04.199 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-18T03:16:04.200 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-18T03:16:04.200 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mfd-core.ko 2023-07-18T03:16:04.200 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-18T03:16:04.200 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-18T03:16:04.200 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-18T03:16:04.200 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-18T03:16:04.200 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc 2023-07-18T03:16:04.200 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl 2023-07-18T03:16:04.201 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-18T03:16:04.201 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-18T03:16:04.201 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-18T03:16:04.201 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-18T03:16:04.201 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710 2023-07-18T03:16:04.201 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-18T03:16:04.201 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom 2023-07-18T03:16:04.201 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-18T03:16:04.202 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-18T03:16:04.202 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-18T03:16:04.202 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-18T03:16:04.202 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-18T03:16:04.202 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-18T03:16:04.202 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-18T03:16:04.202 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-18T03:16:04.202 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d 2023-07-18T03:16:04.203 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-18T03:16:04.203 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-18T03:16:04.203 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei 2023-07-18T03:16:04.203 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-18T03:16:04.203 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-18T03:16:04.203 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-18T03:16:04.203 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru 2023-07-18T03:16:04.204 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-18T03:16:04.204 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp 2023-07-18T03:16:04.204 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-18T03:16:04.204 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-18T03:16:04.204 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-18T03:16:04.204 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-18T03:16:04.204 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-18T03:16:04.204 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-18T03:16:04.205 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-18T03:16:04.205 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci 2023-07-18T03:16:04.205 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-18T03:16:04.205 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net 2023-07-18T03:16:04.205 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding 2023-07-18T03:16:04.205 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-18T03:16:04.205 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa 2023-07-18T03:16:04.205 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53 2023-07-18T03:16:04.206 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-18T03:16:04.206 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-18T03:16:04.206 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-18T03:16:04.206 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx 2023-07-18T03:16:04.206 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-18T03:16:04.206 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-18T03:16:04.206 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-18T03:16:04.206 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet 2023-07-18T03:16:04.207 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com 2023-07-18T03:16:04.207 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-18T03:16:04.207 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-18T03:16:04.207 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-18T03:16:04.207 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-18T03:16:04.207 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390 2023-07-18T03:16:04.207 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-18T03:16:04.207 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-18T03:16:04.208 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-18T03:16:04.208 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-18T03:16:04.208 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec 2023-07-18T03:16:04.208 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-18T03:16:04.208 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere 2023-07-18T03:16:04.208 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-18T03:16:04.208 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon 2023-07-18T03:16:04.209 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-18T03:16:04.209 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera 2023-07-18T03:16:04.209 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-18T03:16:04.209 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd 2023-07-18T03:16:04.209 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-18T03:16:04.209 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-18T03:16:04.209 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-18T03:16:04.209 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros 2023-07-18T03:16:04.210 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx 2023-07-18T03:16:04.210 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-18T03:16:04.210 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-18T03:16:04.210 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-18T03:16:04.210 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-18T03:16:04.210 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-18T03:16:04.210 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx 2023-07-18T03:16:04.210 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-18T03:16:04.211 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-18T03:16:04.211 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom 2023-07-18T03:16:04.211 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-18T03:16:04.211 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-18T03:16:04.211 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-18T03:16:04.211 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-18T03:16:04.211 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-18T03:16:04.211 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet 2023-07-18T03:16:04.212 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-18T03:16:04.212 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-18T03:16:04.212 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade 2023-07-18T03:16:04.212 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna 2023-07-18T03:16:04.212 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-18T03:16:04.212 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio 2023-07-18T03:16:04.212 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-18T03:16:04.213 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-18T03:16:04.213 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-18T03:16:04.213 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-18T03:16:04.213 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-18T03:16:04.213 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-18T03:16:04.213 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-18T03:16:04.213 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-18T03:16:04.213 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-18T03:16:04.214 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-18T03:16:04.214 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco 2023-07-18T03:16:04.214 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic 2023-07-18T03:16:04.214 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-18T03:16:04.214 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec 2023-07-18T03:16:04.214 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip 2023-07-18T03:16:04.214 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-18T03:16:04.214 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-18T03:16:04.215 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-18T03:16:04.215 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-18T03:16:04.215 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-18T03:16:04.215 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-18T03:16:04.215 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink 2023-07-18T03:16:04.215 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-18T03:16:04.215 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-18T03:16:04.215 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-18T03:16:04.216 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex 2023-07-18T03:16:04.216 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet 2023-07-18T03:16:04.216 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-18T03:16:04.216 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-18T03:16:04.216 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-18T03:16:04.216 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel 2023-07-18T03:16:04.216 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-18T03:16:04.216 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000 2023-07-18T03:16:04.217 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-18T03:16:04.217 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e 2023-07-18T03:16:04.217 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-18T03:16:04.217 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k 2023-07-18T03:16:04.217 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-18T03:16:04.217 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e 2023-07-18T03:16:04.217 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-18T03:16:04.217 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf 2023-07-18T03:16:04.218 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-18T03:16:04.218 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb 2023-07-18T03:16:04.218 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-18T03:16:04.218 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf 2023-07-18T03:16:04.218 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-18T03:16:04.218 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-18T03:16:04.218 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-18T03:16:04.218 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-18T03:16:04.219 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-18T03:16:04.219 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-18T03:16:04.219 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell 2023-07-18T03:16:04.219 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-18T03:16:04.219 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-18T03:16:04.219 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-18T03:16:04.219 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox 2023-07-18T03:16:04.220 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-18T03:16:04.220 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-18T03:16:04.220 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-18T03:16:04.220 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-18T03:16:04.220 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-18T03:16:04.220 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-18T03:16:04.220 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel 2023-07-18T03:16:04.220 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-18T03:16:04.221 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom 2023-07-18T03:16:04.221 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-18T03:16:04.221 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-18T03:16:04.221 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi 2023-07-18T03:16:04.221 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-18T03:16:04.221 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-18T03:16:04.221 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion 2023-07-18T03:16:04.221 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-18T03:16:04.222 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia 2023-07-18T03:16:04.222 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-18T03:16:04.222 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines 2023-07-18T03:16:04.222 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-18T03:16:04.222 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-18T03:16:04.222 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic 2023-07-18T03:16:04.222 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-18T03:16:04.222 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-18T03:16:04.223 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-18T03:16:04.223 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-18T03:16:04.223 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-18T03:16:04.223 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc 2023-07-18T03:16:04.223 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-18T03:16:04.223 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek 2023-07-18T03:16:04.223 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-18T03:16:04.223 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-18T03:16:04.224 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-18T03:16:04.224 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-18T03:16:04.224 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker 2023-07-18T03:16:04.224 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-18T03:16:04.224 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc 2023-07-18T03:16:04.224 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-18T03:16:04.224 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan 2023-07-18T03:16:04.225 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-18T03:16:04.225 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis 2023-07-18T03:16:04.225 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-18T03:16:04.225 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-18T03:16:04.225 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc 2023-07-18T03:16:04.225 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-18T03:16:04.225 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-18T03:16:04.225 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-18T03:16:04.226 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-18T03:16:04.226 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro 2023-07-18T03:16:04.226 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-18T03:16:04.226 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-18T03:16:04.226 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-18T03:16:04.226 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun 2023-07-18T03:16:04.226 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-18T03:16:04.226 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-18T03:16:04.227 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-18T03:16:04.227 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-18T03:16:04.227 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti 2023-07-18T03:16:04.227 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-18T03:16:04.227 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti 2023-07-18T03:16:04.227 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-18T03:16:04.228 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via 2023-07-18T03:16:04.228 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-18T03:16:04.228 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-18T03:16:04.228 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet 2023-07-18T03:16:04.228 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-18T03:16:04.228 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-18T03:16:04.228 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom 2023-07-18T03:16:04.229 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-18T03:16:04.229 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-18T03:16:04.229 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv 2023-07-18T03:16:04.229 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-18T03:16:04.229 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154 2023-07-18T03:16:04.229 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-18T03:16:04.229 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-18T03:16:04.229 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan 2023-07-18T03:16:04.230 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-18T03:16:04.230 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-18T03:16:04.230 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-18T03:16:04.230 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio 2023-07-18T03:16:04.230 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-18T03:16:04.230 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-18T03:16:04.230 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-18T03:16:04.230 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-18T03:16:04.231 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-18T03:16:04.231 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/net_failover.ko 2023-07-18T03:16:04.231 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-18T03:16:04.231 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-18T03:16:04.231 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs 2023-07-18T03:16:04.231 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-18T03:16:04.231 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-18T03:16:04.232 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy 2023-07-18T03:16:04.232 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-18T03:16:04.232 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-18T03:16:04.232 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-18T03:16:04.232 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-18T03:16:04.232 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-18T03:16:04.232 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-18T03:16:04.232 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-18T03:16:04.233 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-18T03:16:04.233 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/dp83640.ko 2023-07-18T03:16:04.233 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-18T03:16:04.233 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-18T03:16:04.233 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-18T03:16:04.233 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-18T03:16:04.233 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-18T03:16:04.234 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-18T03:16:04.234 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-18T03:16:04.234 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-18T03:16:04.234 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-18T03:16:04.234 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-18T03:16:04.234 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-18T03:16:04.234 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-18T03:16:04.234 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp 2023-07-18T03:16:04.235 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-18T03:16:04.235 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-18T03:16:04.235 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-18T03:16:04.235 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-18T03:16:04.235 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-18T03:16:04.235 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-18T03:16:04.235 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-18T03:16:04.236 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-18T03:16:04.236 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-18T03:16:04.236 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip 2023-07-18T03:16:04.236 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slhc.ko 2023-07-18T03:16:04.236 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-18T03:16:04.236 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-18T03:16:04.236 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-18T03:16:04.236 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team 2023-07-18T03:16:04.237 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-18T03:16:04.237 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-18T03:16:04.237 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-18T03:16:04.237 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-18T03:16:04.237 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-18T03:16:04.237 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-18T03:16:04.237 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tun.ko 2023-07-18T03:16:04.237 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-18T03:16:04.238 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/virtio_net.ko 2023-07-18T03:16:04.238 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3 2023-07-18T03:16:04.238 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-18T03:16:04.238 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan 2023-07-18T03:16:04.238 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-18T03:16:04.238 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback 2023-07-18T03:16:04.238 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-18T03:16:04.239 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netfront.ko 2023-07-18T03:16:04.239 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb 2023-07-18T03:16:04.239 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-18T03:16:04.239 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme 2023-07-18T03:16:04.239 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host 2023-07-18T03:16:04.239 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-18T03:16:04.239 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-18T03:16:04.239 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport 2023-07-18T03:16:04.240 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-18T03:16:04.240 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-18T03:16:04.240 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-18T03:16:04.240 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-18T03:16:04.240 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci 2023-07-18T03:16:04.240 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug 2023-07-18T03:16:04.240 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-18T03:16:04.240 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie 2023-07-18T03:16:04.241 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-18T03:16:04.241 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-18T03:16:04.241 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia 2023-07-18T03:16:04.241 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-18T03:16:04.241 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-18T03:16:04.241 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-18T03:16:04.241 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl 2023-07-18T03:16:04.241 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel 2023-07-18T03:16:04.242 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-18T03:16:04.242 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform 2023-07-18T03:16:04.242 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86 2023-07-18T03:16:04.242 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-18T03:16:04.242 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-18T03:16:04.242 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-18T03:16:04.242 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-18T03:16:04.242 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-18T03:16:04.243 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-18T03:16:04.243 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-18T03:16:04.243 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-18T03:16:04.243 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-18T03:16:04.243 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-18T03:16:04.243 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-18T03:16:04.243 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-18T03:16:04.243 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel 2023-07-18T03:16:04.244 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-18T03:16:04.244 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-18T03:16:04.244 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-18T03:16:04.244 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-18T03:16:04.244 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-18T03:16:04.244 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-18T03:16:04.244 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-18T03:16:04.245 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-18T03:16:04.245 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-18T03:16:04.245 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-18T03:16:04.245 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-18T03:16:04.245 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-18T03:16:04.245 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap 2023-07-18T03:16:04.245 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-18T03:16:04.245 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-18T03:16:04.246 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps 2023-07-18T03:16:04.246 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients 2023-07-18T03:16:04.246 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-18T03:16:04.246 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-18T03:16:04.246 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-18T03:16:04.246 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-18T03:16:04.246 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp 2023-07-18T03:16:04.246 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-18T03:16:04.247 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-18T03:16:04.247 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc 2023-07-18T03:16:04.247 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-18T03:16:04.247 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-18T03:16:04.247 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-18T03:16:04.247 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-18T03:16:04.247 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-18T03:16:04.248 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-18T03:16:04.248 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-18T03:16:04.248 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-18T03:16:04.248 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-18T03:16:04.248 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-18T03:16:04.248 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-18T03:16:04.248 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-18T03:16:04.248 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-18T03:16:04.249 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-18T03:16:04.249 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-18T03:16:04.249 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-18T03:16:04.249 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-18T03:16:04.249 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-18T03:16:04.249 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-18T03:16:04.249 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-18T03:16:04.249 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-18T03:16:04.250 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-18T03:16:04.250 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-18T03:16:04.250 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-18T03:16:04.250 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-18T03:16:04.250 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-18T03:16:04.250 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-18T03:16:04.250 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-18T03:16:04.250 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-18T03:16:04.251 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-18T03:16:04.251 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-18T03:16:04.251 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-18T03:16:04.251 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi 2023-07-18T03:16:04.251 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-18T03:16:04.251 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-18T03:16:04.251 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-18T03:16:04.251 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-18T03:16:04.252 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-18T03:16:04.252 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid 2023-07-18T03:16:04.252 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-18T03:16:04.252 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-18T03:16:04.252 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx 2023-07-18T03:16:04.252 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-18T03:16:04.252 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-18T03:16:04.252 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-18T03:16:04.253 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr 2023-07-18T03:16:04.253 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-18T03:16:04.253 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-18T03:16:04.253 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi 2023-07-18T03:16:04.253 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-18T03:16:04.253 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa 2023-07-18T03:16:04.253 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-18T03:16:04.253 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc 2023-07-18T03:16:04.254 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-18T03:16:04.254 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i 2023-07-18T03:16:04.254 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-18T03:16:04.254 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-18T03:16:04.254 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor 2023-07-18T03:16:04.254 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-18T03:16:04.254 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi 2023-07-18T03:16:04.254 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-18T03:16:04.255 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-18T03:16:04.255 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-18T03:16:04.255 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-18T03:16:04.255 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-18T03:16:04.255 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-18T03:16:04.255 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler 2023-07-18T03:16:04.255 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-18T03:16:04.255 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-18T03:16:04.256 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-18T03:16:04.256 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-18T03:16:04.256 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-18T03:16:04.256 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r 2023-07-18T03:16:04.256 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-18T03:16:04.256 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-18T03:16:04.256 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe 2023-07-18T03:16:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-18T03:16:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-18T03:16:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic 2023-07-18T03:16:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-18T03:16:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-18T03:16:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-18T03:16:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-18T03:16:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-18T03:16:04.258 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-18T03:16:04.258 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-18T03:16:04.258 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci 2023-07-18T03:16:04.258 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-18T03:16:04.258 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-18T03:16:04.258 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-18T03:16:04.258 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc 2023-07-18T03:16:04.258 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-18T03:16:04.259 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-18T03:16:04.259 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-18T03:16:04.259 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas 2023-07-18T03:16:04.259 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-18T03:16:04.259 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc 2023-07-18T03:16:04.259 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-18T03:16:04.259 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid 2023-07-18T03:16:04.259 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-18T03:16:04.260 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-18T03:16:04.260 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-18T03:16:04.260 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-18T03:16:04.260 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas 2023-07-18T03:16:04.260 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-18T03:16:04.260 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas 2023-07-18T03:16:04.260 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-18T03:16:04.260 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-18T03:16:04.261 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001 2023-07-18T03:16:04.261 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-18T03:16:04.261 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-18T03:16:04.261 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-18T03:16:04.261 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx 2023-07-18T03:16:04.261 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-18T03:16:04.261 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-18T03:16:04.261 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx 2023-07-18T03:16:04.262 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-18T03:16:04.262 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-18T03:16:04.262 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-18T03:16:04.262 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-18T03:16:04.262 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-18T03:16:04.262 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-18T03:16:04.262 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-18T03:16:04.262 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-18T03:16:04.263 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-18T03:16:04.263 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-18T03:16:04.263 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-18T03:16:04.263 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2 2023-07-18T03:16:04.263 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-18T03:16:04.263 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-18T03:16:04.263 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-18T03:16:04.263 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-18T03:16:04.264 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb 2023-07-18T03:16:04.264 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-18T03:16:04.264 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target 2023-07-18T03:16:04.264 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi 2023-07-18T03:16:04.264 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-18T03:16:04.264 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback 2023-07-18T03:16:04.264 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-18T03:16:04.265 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-18T03:16:04.265 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-18T03:16:04.265 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-18T03:16:04.265 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-18T03:16:04.265 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-18T03:16:04.265 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc 2023-07-18T03:16:04.265 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-18T03:16:04.265 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal 2023-07-18T03:16:04.266 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel 2023-07-18T03:16:04.266 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal 2023-07-18T03:16:04.266 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-18T03:16:04.266 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-18T03:16:04.266 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-18T03:16:04.266 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-18T03:16:04.266 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-18T03:16:04.266 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-18T03:16:04.267 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-18T03:16:04.267 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-18T03:16:04.267 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-18T03:16:04.267 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-18T03:16:04.267 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-18T03:16:04.267 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-18T03:16:04.267 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-18T03:16:04.267 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-18T03:16:04.268 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-18T03:16:04.268 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty 2023-07-18T03:16:04.268 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless 2023-07-18T03:16:04.268 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-18T03:16:04.268 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-18T03:16:04.268 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-18T03:16:04.268 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-18T03:16:04.269 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial 2023-07-18T03:16:04.269 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250 2023-07-18T03:16:04.269 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-18T03:16:04.269 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-18T03:16:04.269 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm 2023-07-18T03:16:04.269 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-18T03:16:04.269 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-18T03:16:04.269 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs 2023-07-18T03:16:04.270 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core 2023-07-18T03:16:04.270 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-18T03:16:04.270 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host 2023-07-18T03:16:04.270 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-18T03:16:04.270 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio 2023-07-18T03:16:04.270 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-18T03:16:04.270 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-18T03:16:04.270 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-18T03:16:04.271 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-18T03:16:04.271 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-18T03:16:04.271 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb 2023-07-18T03:16:04.271 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class 2023-07-18T03:16:04.271 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-18T03:16:04.271 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-18T03:16:04.271 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-18T03:16:04.271 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-18T03:16:04.272 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host 2023-07-18T03:16:04.272 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-18T03:16:04.272 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image 2023-07-18T03:16:04.272 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-18T03:16:04.272 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-18T03:16:04.272 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc 2023-07-18T03:16:04.272 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-18T03:16:04.272 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-18T03:16:04.273 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-18T03:16:04.273 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-18T03:16:04.273 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-18T03:16:04.273 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-18T03:16:04.273 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-18T03:16:04.273 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-18T03:16:04.273 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-18T03:16:04.273 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-18T03:16:04.274 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga 2023-07-18T03:16:04.274 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-18T03:16:04.274 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-18T03:16:04.274 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-18T03:16:04.274 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-18T03:16:04.274 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-18T03:16:04.274 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-18T03:16:04.274 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-18T03:16:04.275 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy 2023-07-18T03:16:04.275 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-18T03:16:04.275 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial 2023-07-18T03:16:04.275 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-18T03:16:04.275 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-18T03:16:04.275 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-18T03:16:04.275 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-18T03:16:04.275 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-18T03:16:04.276 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-18T03:16:04.276 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-18T03:16:04.276 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-18T03:16:04.276 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-18T03:16:04.276 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-18T03:16:04.276 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-18T03:16:04.276 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-18T03:16:04.276 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-18T03:16:04.277 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-18T03:16:04.277 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-18T03:16:04.277 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-18T03:16:04.277 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-18T03:16:04.277 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-18T03:16:04.277 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-18T03:16:04.277 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-18T03:16:04.277 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-18T03:16:04.278 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-18T03:16:04.278 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-18T03:16:04.278 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-18T03:16:04.278 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-18T03:16:04.278 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-18T03:16:04.278 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-18T03:16:04.278 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-18T03:16:04.278 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-18T03:16:04.279 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-18T03:16:04.279 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-18T03:16:04.279 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-18T03:16:04.279 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-18T03:16:04.279 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-18T03:16:04.279 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-18T03:16:04.279 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-18T03:16:04.279 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-18T03:16:04.280 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-18T03:16:04.280 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-18T03:16:04.280 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-18T03:16:04.280 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-18T03:16:04.280 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-18T03:16:04.280 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-18T03:16:04.280 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-18T03:16:04.281 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-18T03:16:04.281 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage 2023-07-18T03:16:04.281 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-18T03:16:04.281 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-18T03:16:04.281 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-18T03:16:04.281 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-18T03:16:04.281 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-18T03:16:04.281 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-18T03:16:04.282 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-18T03:16:04.282 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-18T03:16:04.282 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-18T03:16:04.282 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-18T03:16:04.282 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-18T03:16:04.282 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-18T03:16:04.282 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-18T03:16:04.282 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-18T03:16:04.283 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-18T03:16:04.283 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip 2023-07-18T03:16:04.283 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-18T03:16:04.283 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-18T03:16:04.283 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-18T03:16:04.283 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio 2023-07-18T03:16:04.283 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci 2023-07-18T03:16:04.283 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-18T03:16:04.284 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-18T03:16:04.284 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-18T03:16:04.284 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-18T03:16:04.284 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost 2023-07-18T03:16:04.284 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-18T03:16:04.284 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-18T03:16:04.284 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-18T03:16:04.285 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-18T03:16:04.285 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video 2023-07-18T03:16:04.285 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight 2023-07-18T03:16:04.285 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-18T03:16:04.285 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-18T03:16:04.285 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-18T03:16:04.285 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio 2023-07-18T03:16:04.285 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio.ko 2023-07-18T03:16:04.286 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_balloon.ko 2023-07-18T03:16:04.286 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_mmio.ko 2023-07-18T03:16:04.286 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci.ko 2023-07-18T03:16:04.286 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-18T03:16:04.286 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-18T03:16:04.286 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_ring.ko 2023-07-18T03:16:04.286 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1 2023-07-18T03:16:04.286 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters 2023-07-18T03:16:04.287 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-18T03:16:04.287 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-18T03:16:04.287 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves 2023-07-18T03:16:04.287 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-18T03:16:04.287 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-18T03:16:04.287 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-18T03:16:04.287 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-18T03:16:04.287 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-18T03:16:04.288 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-18T03:16:04.288 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-18T03:16:04.288 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-18T03:16:04.288 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-18T03:16:04.288 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-18T03:16:04.288 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-18T03:16:04.288 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-18T03:16:04.288 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog 2023-07-18T03:16:04.289 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-18T03:16:04.289 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-18T03:16:04.289 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-18T03:16:04.289 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-18T03:16:04.289 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-18T03:16:04.289 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-18T03:16:04.289 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-18T03:16:04.289 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-18T03:16:04.290 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-18T03:16:04.290 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-18T03:16:04.290 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-18T03:16:04.290 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-18T03:16:04.290 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-18T03:16:04.290 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-18T03:16:04.290 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-18T03:16:04.290 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-18T03:16:04.291 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-18T03:16:04.291 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-18T03:16:04.291 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-18T03:16:04.291 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-18T03:16:04.291 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-18T03:16:04.291 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-18T03:16:04.291 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-18T03:16:04.291 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-18T03:16:04.292 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-18T03:16:04.292 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-18T03:16:04.292 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen 2023-07-18T03:16:04.292 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-18T03:16:04.292 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-18T03:16:04.292 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-18T03:16:04.292 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-18T03:16:04.293 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback 2023-07-18T03:16:04.293 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-18T03:16:04.293 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-18T03:16:04.293 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs 2023-07-18T03:16:04.293 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-18T03:16:04.293 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs 2023-07-18T03:16:04.293 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p 2023-07-18T03:16:04.293 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-18T03:16:04.294 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs 2023-07-18T03:16:04.294 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-18T03:16:04.294 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs 2023-07-18T03:16:04.294 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-18T03:16:04.294 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-18T03:16:04.294 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs 2023-07-18T03:16:04.294 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-18T03:16:04.294 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles 2023-07-18T03:16:04.295 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-18T03:16:04.295 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph 2023-07-18T03:16:04.295 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-18T03:16:04.295 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda 2023-07-18T03:16:04.295 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-18T03:16:04.295 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs 2023-07-18T03:16:04.295 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-18T03:16:04.295 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm 2023-07-18T03:16:04.296 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-18T03:16:04.296 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs 2023-07-18T03:16:04.296 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-18T03:16:04.296 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs 2023-07-18T03:16:04.296 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-18T03:16:04.296 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat 2023-07-18T03:16:04.296 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/fat.ko 2023-07-18T03:16:04.296 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-18T03:16:04.297 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/vfat.ko 2023-07-18T03:16:04.297 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache 2023-07-18T03:16:04.297 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-18T03:16:04.297 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse 2023-07-18T03:16:04.297 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-18T03:16:04.297 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/fuse.ko 2023-07-18T03:16:04.297 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2 2023-07-18T03:16:04.298 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-18T03:16:04.298 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs 2023-07-18T03:16:04.298 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-18T03:16:04.298 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus 2023-07-18T03:16:04.298 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-18T03:16:04.298 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs 2023-07-18T03:16:04.298 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-18T03:16:04.298 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs 2023-07-18T03:16:04.299 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-18T03:16:04.299 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd 2023-07-18T03:16:04.299 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-18T03:16:04.299 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix 2023-07-18T03:16:04.299 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-18T03:16:04.299 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs 2023-07-18T03:16:04.299 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-18T03:16:04.299 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs 2023-07-18T03:16:04.300 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout 2023-07-18T03:16:04.300 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-18T03:16:04.300 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout 2023-07-18T03:16:04.300 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-18T03:16:04.300 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout 2023-07-18T03:16:04.300 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-18T03:16:04.300 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-18T03:16:04.300 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-18T03:16:04.301 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-18T03:16:04.301 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common 2023-07-18T03:16:04.301 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-18T03:16:04.301 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-18T03:16:04.301 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd 2023-07-18T03:16:04.301 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-18T03:16:04.301 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2 2023-07-18T03:16:04.301 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-18T03:16:04.302 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls 2023-07-18T03:16:04.302 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-18T03:16:04.302 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-18T03:16:04.302 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-18T03:16:04.302 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-18T03:16:04.302 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-18T03:16:04.302 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-18T03:16:04.302 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-18T03:16:04.303 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-18T03:16:04.303 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-18T03:16:04.303 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-18T03:16:04.303 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-18T03:16:04.303 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-18T03:16:04.303 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-18T03:16:04.303 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-18T03:16:04.303 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-18T03:16:04.304 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-18T03:16:04.304 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-18T03:16:04.304 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-18T03:16:04.304 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-18T03:16:04.304 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-18T03:16:04.304 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-18T03:16:04.304 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-18T03:16:04.304 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-18T03:16:04.305 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-18T03:16:04.305 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-18T03:16:04.305 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-18T03:16:04.305 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-18T03:16:04.305 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-18T03:16:04.305 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-18T03:16:04.305 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-18T03:16:04.305 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-18T03:16:04.306 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-18T03:16:04.306 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-18T03:16:04.306 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-18T03:16:04.306 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-18T03:16:04.306 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-18T03:16:04.306 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-18T03:16:04.306 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-18T03:16:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-18T03:16:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-18T03:16:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-18T03:16:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-18T03:16:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-18T03:16:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-18T03:16:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-18T03:16:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-18T03:16:04.308 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-18T03:16:04.308 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-18T03:16:04.308 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-18T03:16:04.308 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2 2023-07-18T03:16:04.308 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster 2023-07-18T03:16:04.308 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-18T03:16:04.308 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm 2023-07-18T03:16:04.308 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-18T03:16:04.309 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs 2023-07-18T03:16:04.309 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-18T03:16:04.309 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-18T03:16:04.309 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-18T03:16:04.309 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-18T03:16:04.309 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-18T03:16:04.309 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs 2023-07-18T03:16:04.309 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-18T03:16:04.310 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore 2023-07-18T03:16:04.310 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-18T03:16:04.310 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs 2023-07-18T03:16:04.310 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-18T03:16:04.310 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs 2023-07-18T03:16:04.310 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-18T03:16:04.310 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb 2023-07-18T03:16:04.310 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client 2023-07-18T03:16:04.311 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-18T03:16:04.311 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common 2023-07-18T03:16:04.311 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-18T03:16:04.311 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-18T03:16:04.311 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs 2023-07-18T03:16:04.311 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-18T03:16:04.311 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv 2023-07-18T03:16:04.311 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-18T03:16:04.312 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf 2023-07-18T03:16:04.312 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-18T03:16:04.312 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs 2023-07-18T03:16:04.312 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-18T03:16:04.312 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs 2023-07-18T03:16:04.312 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-18T03:16:04.312 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel 2023-07-18T03:16:04.312 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu 2023-07-18T03:16:04.313 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu/rcutorture.ko 2023-07-18T03:16:04.313 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-18T03:16:04.313 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace 2023-07-18T03:16:04.313 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-18T03:16:04.313 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib 2023-07-18T03:16:04.313 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/asn1_encoder.ko 2023-07-18T03:16:04.313 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-18T03:16:04.314 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-18T03:16:04.314 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto 2023-07-18T03:16:04.314 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-18T03:16:04.314 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-18T03:16:04.314 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-18T03:16:04.314 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-18T03:16:04.314 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4 2023-07-18T03:16:04.314 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-18T03:16:04.315 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-18T03:16:04.315 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math 2023-07-18T03:16:04.315 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-18T03:16:04.315 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6 2023-07-18T03:16:04.315 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-18T03:16:04.315 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon 2023-07-18T03:16:04.315 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-18T03:16:04.315 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-18T03:16:04.316 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-18T03:16:04.316 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-18T03:16:04.316 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd 2023-07-18T03:16:04.316 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd/zstd_compress.ko 2023-07-18T03:16:04.316 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm 2023-07-18T03:16:04.316 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-18T03:16:04.316 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net 2023-07-18T03:16:04.316 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan 2023-07-18T03:16:04.317 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-18T03:16:04.317 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-18T03:16:04.317 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-18T03:16:04.317 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-18T03:16:04.317 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-18T03:16:04.317 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-18T03:16:04.317 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-18T03:16:04.317 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-18T03:16:04.318 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802 2023-07-18T03:16:04.318 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-18T03:16:04.318 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-18T03:16:04.318 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-18T03:16:04.318 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-18T03:16:04.318 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-18T03:16:04.318 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q 2023-07-18T03:16:04.318 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-18T03:16:04.319 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p 2023-07-18T03:16:04.319 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-18T03:16:04.319 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-18T03:16:04.319 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-18T03:16:04.319 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-18T03:16:04.319 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv 2023-07-18T03:16:04.319 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-18T03:16:04.319 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge 2023-07-18T03:16:04.320 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-18T03:16:04.320 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-18T03:16:04.320 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter 2023-07-18T03:16:04.320 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-18T03:16:04.320 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-18T03:16:04.320 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-18T03:16:04.320 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-18T03:16:04.320 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-18T03:16:04.321 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-18T03:16:04.321 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-18T03:16:04.321 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-18T03:16:04.321 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-18T03:16:04.321 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-18T03:16:04.321 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-18T03:16:04.321 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-18T03:16:04.321 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-18T03:16:04.322 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-18T03:16:04.322 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-18T03:16:04.322 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-18T03:16:04.322 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-18T03:16:04.322 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-18T03:16:04.322 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-18T03:16:04.322 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-18T03:16:04.322 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-18T03:16:04.323 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-18T03:16:04.323 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-18T03:16:04.323 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-18T03:16:04.323 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph 2023-07-18T03:16:04.323 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-18T03:16:04.323 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core 2023-07-18T03:16:04.323 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/failover.ko 2023-07-18T03:16:04.324 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-18T03:16:04.324 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp 2023-07-18T03:16:04.324 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-18T03:16:04.324 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-18T03:16:04.324 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-18T03:16:04.324 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-18T03:16:04.324 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver 2023-07-18T03:16:04.324 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver/dns_resolver.ko 2023-07-18T03:16:04.325 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa 2023-07-18T03:16:04.325 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-18T03:16:04.325 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-18T03:16:04.325 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-18T03:16:04.325 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-18T03:16:04.325 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-18T03:16:04.325 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154 2023-07-18T03:16:04.325 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan 2023-07-18T03:16:04.326 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-18T03:16:04.326 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-18T03:16:04.326 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-18T03:16:04.326 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4 2023-07-18T03:16:04.326 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-18T03:16:04.326 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-18T03:16:04.326 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-18T03:16:04.326 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-18T03:16:04.327 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-18T03:16:04.327 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-18T03:16:04.327 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-18T03:16:04.327 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-18T03:16:04.327 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-18T03:16:04.327 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-18T03:16:04.327 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter 2023-07-18T03:16:04.327 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-18T03:16:04.328 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-18T03:16:04.328 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-18T03:16:04.328 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-18T03:16:04.328 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-18T03:16:04.328 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-18T03:16:04.328 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-18T03:16:04.328 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-18T03:16:04.328 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-18T03:16:04.329 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-18T03:16:04.329 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-18T03:16:04.329 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-18T03:16:04.329 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-18T03:16:04.329 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-18T03:16:04.329 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-18T03:16:04.329 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-18T03:16:04.330 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-18T03:16:04.330 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-18T03:16:04.330 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-18T03:16:04.330 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-18T03:16:04.330 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-18T03:16:04.330 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-18T03:16:04.330 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-18T03:16:04.330 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-18T03:16:04.331 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-18T03:16:04.331 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-18T03:16:04.331 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-18T03:16:04.331 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-18T03:16:04.331 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-18T03:16:04.331 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-18T03:16:04.331 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-18T03:16:04.331 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-18T03:16:04.332 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-18T03:16:04.332 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-18T03:16:04.332 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-18T03:16:04.332 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-18T03:16:04.332 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-18T03:16:04.332 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-18T03:16:04.332 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6 2023-07-18T03:16:04.332 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-18T03:16:04.333 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-18T03:16:04.333 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-18T03:16:04.333 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-18T03:16:04.333 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-18T03:16:04.333 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-18T03:16:04.333 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-18T03:16:04.333 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter 2023-07-18T03:16:04.333 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-18T03:16:04.334 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-18T03:16:04.334 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-18T03:16:04.334 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-18T03:16:04.334 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-18T03:16:04.334 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-18T03:16:04.334 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-18T03:16:04.334 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-18T03:16:04.334 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-18T03:16:04.335 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-18T03:16:04.335 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-18T03:16:04.335 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-18T03:16:04.335 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-18T03:16:04.335 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-18T03:16:04.335 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-18T03:16:04.335 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-18T03:16:04.335 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-18T03:16:04.336 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-18T03:16:04.336 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-18T03:16:04.336 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-18T03:16:04.336 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-18T03:16:04.336 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-18T03:16:04.336 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-18T03:16:04.336 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-18T03:16:04.336 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-18T03:16:04.337 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-18T03:16:04.337 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-18T03:16:04.337 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key 2023-07-18T03:16:04.337 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-18T03:16:04.337 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp 2023-07-18T03:16:04.337 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-18T03:16:04.337 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-18T03:16:04.337 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-18T03:16:04.338 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-18T03:16:04.338 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-18T03:16:04.338 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-18T03:16:04.338 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-18T03:16:04.338 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc 2023-07-18T03:16:04.338 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-18T03:16:04.338 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154 2023-07-18T03:16:04.338 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-18T03:16:04.339 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls 2023-07-18T03:16:04.339 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-18T03:16:04.339 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter 2023-07-18T03:16:04.339 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset 2023-07-18T03:16:04.339 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-18T03:16:04.339 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-18T03:16:04.339 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-18T03:16:04.339 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-18T03:16:04.340 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-18T03:16:04.340 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-18T03:16:04.340 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-18T03:16:04.340 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-18T03:16:04.340 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-18T03:16:04.340 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-18T03:16:04.340 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-18T03:16:04.341 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-18T03:16:04.341 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-18T03:16:04.341 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-18T03:16:04.341 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-18T03:16:04.341 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-18T03:16:04.341 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs 2023-07-18T03:16:04.341 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-18T03:16:04.341 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-18T03:16:04.342 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-18T03:16:04.342 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-18T03:16:04.342 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-18T03:16:04.342 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-18T03:16:04.342 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-18T03:16:04.342 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-18T03:16:04.342 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-18T03:16:04.342 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-18T03:16:04.343 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-18T03:16:04.343 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-18T03:16:04.343 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-18T03:16:04.343 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-18T03:16:04.343 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-18T03:16:04.343 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-18T03:16:04.343 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-18T03:16:04.343 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-18T03:16:04.344 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-18T03:16:04.344 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-18T03:16:04.344 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-18T03:16:04.344 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-18T03:16:04.344 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-18T03:16:04.344 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-18T03:16:04.344 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-18T03:16:04.344 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-18T03:16:04.345 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-18T03:16:04.345 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-18T03:16:04.345 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-18T03:16:04.345 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-18T03:16:04.345 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-18T03:16:04.345 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-18T03:16:04.345 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-18T03:16:04.345 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-18T03:16:04.346 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-18T03:16:04.346 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-18T03:16:04.346 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-18T03:16:04.346 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-18T03:16:04.346 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-18T03:16:04.346 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-18T03:16:04.346 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-18T03:16:04.346 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-18T03:16:04.347 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-18T03:16:04.347 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-18T03:16:04.347 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-18T03:16:04.347 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-18T03:16:04.347 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-18T03:16:04.347 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-18T03:16:04.347 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-18T03:16:04.347 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-18T03:16:04.348 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-18T03:16:04.348 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-18T03:16:04.348 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-18T03:16:04.348 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-18T03:16:04.348 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-18T03:16:04.348 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-18T03:16:04.348 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-18T03:16:04.349 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-18T03:16:04.349 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_quota.ko 2023-07-18T03:16:04.349 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-18T03:16:04.349 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-18T03:16:04.349 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-18T03:16:04.349 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-18T03:16:04.349 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tunnel.ko 2023-07-18T03:16:04.349 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-18T03:16:04.350 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-18T03:16:04.350 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-18T03:16:04.350 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-18T03:16:04.350 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-18T03:16:04.350 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-18T03:16:04.350 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-18T03:16:04.350 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-18T03:16:04.350 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-18T03:16:04.351 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-18T03:16:04.351 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-18T03:16:04.351 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-18T03:16:04.351 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-18T03:16:04.351 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-18T03:16:04.351 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-18T03:16:04.351 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-18T03:16:04.351 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-18T03:16:04.352 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-18T03:16:04.352 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-18T03:16:04.352 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-18T03:16:04.352 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-18T03:16:04.352 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-18T03:16:04.352 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-18T03:16:04.352 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-18T03:16:04.352 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-18T03:16:04.353 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-18T03:16:04.353 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-18T03:16:04.353 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-18T03:16:04.353 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-18T03:16:04.353 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-18T03:16:04.353 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-18T03:16:04.353 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-18T03:16:04.353 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-18T03:16:04.354 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-18T03:16:04.354 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-18T03:16:04.354 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-18T03:16:04.354 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-18T03:16:04.354 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-18T03:16:04.354 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-18T03:16:04.354 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-18T03:16:04.354 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-18T03:16:04.355 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-18T03:16:04.355 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-18T03:16:04.355 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-18T03:16:04.355 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-18T03:16:04.355 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-18T03:16:04.355 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-18T03:16:04.355 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-18T03:16:04.355 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-18T03:16:04.356 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-18T03:16:04.356 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-18T03:16:04.356 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-18T03:16:04.356 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-18T03:16:04.356 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-18T03:16:04.356 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-18T03:16:04.356 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-18T03:16:04.356 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-18T03:16:04.357 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-18T03:16:04.357 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-18T03:16:04.357 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-18T03:16:04.357 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-18T03:16:04.357 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-18T03:16:04.357 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-18T03:16:04.357 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-18T03:16:04.357 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-18T03:16:04.358 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-18T03:16:04.358 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-18T03:16:04.358 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-18T03:16:04.358 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-18T03:16:04.358 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-18T03:16:04.358 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-18T03:16:04.358 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink 2023-07-18T03:16:04.359 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-18T03:16:04.359 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh 2023-07-18T03:16:04.359 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-18T03:16:04.359 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch 2023-07-18T03:16:04.359 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-18T03:16:04.359 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-18T03:16:04.359 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-18T03:16:04.359 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-18T03:16:04.360 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet 2023-07-18T03:16:04.360 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-18T03:16:04.360 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds 2023-07-18T03:16:04.360 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-18T03:16:04.360 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-18T03:16:04.360 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-18T03:16:04.360 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched 2023-07-18T03:16:04.360 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-18T03:16:04.361 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-18T03:16:04.361 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-18T03:16:04.361 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-18T03:16:04.361 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-18T03:16:04.361 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-18T03:16:04.361 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-18T03:16:04.361 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-18T03:16:04.361 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-18T03:16:04.362 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-18T03:16:04.362 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-18T03:16:04.362 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-18T03:16:04.362 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-18T03:16:04.362 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-18T03:16:04.362 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-18T03:16:04.362 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-18T03:16:04.362 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-18T03:16:04.363 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-18T03:16:04.363 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-18T03:16:04.363 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-18T03:16:04.363 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-18T03:16:04.363 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-18T03:16:04.363 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-18T03:16:04.363 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-18T03:16:04.363 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-18T03:16:04.364 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-18T03:16:04.364 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-18T03:16:04.364 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-18T03:16:04.364 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-18T03:16:04.364 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-18T03:16:04.364 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-18T03:16:04.364 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-18T03:16:04.364 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-18T03:16:04.365 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-18T03:16:04.365 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-18T03:16:04.365 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-18T03:16:04.365 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-18T03:16:04.365 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-18T03:16:04.365 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-18T03:16:04.365 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-18T03:16:04.365 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-18T03:16:04.366 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-18T03:16:04.366 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-18T03:16:04.366 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-18T03:16:04.366 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp 2023-07-18T03:16:04.366 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-18T03:16:04.366 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-18T03:16:04.366 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc 2023-07-18T03:16:04.366 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss 2023-07-18T03:16:04.367 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-18T03:16:04.367 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-18T03:16:04.367 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-18T03:16:04.367 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma 2023-07-18T03:16:04.367 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-18T03:16:04.367 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc 2023-07-18T03:16:04.367 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-18T03:16:04.367 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-18T03:16:04.368 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix 2023-07-18T03:16:04.368 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-18T03:16:04.368 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock 2023-07-18T03:16:04.368 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-18T03:16:04.368 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-18T03:16:04.368 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-18T03:16:04.368 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-18T03:16:04.368 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-18T03:16:04.369 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm 2023-07-18T03:16:04.369 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-18T03:16:04.369 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security 2023-07-18T03:16:04.369 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys 2023-07-18T03:16:04.369 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys 2023-07-18T03:16:04.369 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-18T03:16:04.369 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys 2023-07-18T03:16:04.369 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys/trusted.ko 2023-07-18T03:16:04.370 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt 2023-07-18T03:16:04.370 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib 2023-07-18T03:16:04.370 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-18T03:16:04.370 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-18T03:16:04.370 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-18T03:16:04.370 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-18T03:16:04.370 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-18T03:16:04.370 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-18T03:16:04.371 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-18T03:16:04.371 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-18T03:16:04.371 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-18T03:16:04.371 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-18T03:16:04.371 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-18T03:16:04.371 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-18T03:16:04.371 INFO:teuthology.orchestra.run.smithi049.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-18T03:16:04.372 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:04.372 DEBUG:teuthology.orchestra.run.smithi049:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-18T03:16:04.400 INFO:teuthology.orchestra.run.smithi049.stdout:postinstall scriptlet (using /bin/sh): 2023-07-18T03:16:04.400 INFO:teuthology.orchestra.run.smithi049.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 -a -r /boot/System.map-6.5.0-rc1-ga440c79eee02 ]; then 2023-07-18T03:16:04.401 INFO:teuthology.orchestra.run.smithi049.stdout:cp /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm 2023-07-18T03:16:04.401 INFO:teuthology.orchestra.run.smithi049.stdout:cp /boot/System.map-6.5.0-rc1-ga440c79eee02 /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-18T03:16:04.401 INFO:teuthology.orchestra.run.smithi049.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:04.401 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/installkernel 6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-18T03:16:04.401 INFO:teuthology.orchestra.run.smithi049.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-18T03:16:04.401 INFO:teuthology.orchestra.run.smithi049.stdout:fi 2023-07-18T03:16:04.401 INFO:teuthology.orchestra.run.smithi049.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-18T03:16:04.402 INFO:teuthology.orchestra.run.smithi049.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-18T03:16:04.402 INFO:teuthology.orchestra.run.smithi049.stdout:new-kernel-pkg --remove 6.5.0-rc1-ga440c79eee02 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-ga440c79eee02.img 2023-07-18T03:16:04.402 INFO:teuthology.orchestra.run.smithi049.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-18T03:16:04.402 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-install remove 6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:04.402 INFO:teuthology.orchestra.run.smithi049.stdout:fi 2023-07-18T03:16:04.402 INFO:teuthology.orchestra.run.smithi049.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-18T03:16:04.402 INFO:teuthology.orchestra.run.smithi049.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-18T03:16:04.402 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:04.403 INFO:teuthology.orchestra.run.smithi049.stdout:fi 2023-07-18T03:16:04.403 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -qi grub2-tools 2023-07-18T03:16:04.483 INFO:teuthology.orchestra.run.smithi049.stdout:Name : grub2-tools 2023-07-18T03:16:04.483 INFO:teuthology.orchestra.run.smithi049.stdout:Epoch : 1 2023-07-18T03:16:04.483 INFO:teuthology.orchestra.run.smithi049.stdout:Version : 2.02 2023-07-18T03:16:04.483 INFO:teuthology.orchestra.run.smithi049.stdout:Release : 123.el8 2023-07-18T03:16:04.484 INFO:teuthology.orchestra.run.smithi049.stdout:Architecture: x86_64 2023-07-18T03:16:04.484 INFO:teuthology.orchestra.run.smithi049.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-18T03:16:04.484 INFO:teuthology.orchestra.run.smithi049.stdout:Group : System Environment/Base 2023-07-18T03:16:04.484 INFO:teuthology.orchestra.run.smithi049.stdout:Size : 9265450 2023-07-18T03:16:04.484 INFO:teuthology.orchestra.run.smithi049.stdout:License : GPLv3+ 2023-07-18T03:16:04.484 INFO:teuthology.orchestra.run.smithi049.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-18T03:16:04.484 INFO:teuthology.orchestra.run.smithi049.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-18T03:16:04.485 INFO:teuthology.orchestra.run.smithi049.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-18T03:16:04.485 INFO:teuthology.orchestra.run.smithi049.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-18T03:16:04.485 INFO:teuthology.orchestra.run.smithi049.stdout:Relocations : (not relocatable) 2023-07-18T03:16:04.485 INFO:teuthology.orchestra.run.smithi049.stdout:Packager : Red Hat, Inc. 2023-07-18T03:16:04.485 INFO:teuthology.orchestra.run.smithi049.stdout:Vendor : Red Hat, Inc. 2023-07-18T03:16:04.485 INFO:teuthology.orchestra.run.smithi049.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-18T03:16:04.485 INFO:teuthology.orchestra.run.smithi049.stdout:Summary : Support tools for GRUB. 2023-07-18T03:16:04.485 INFO:teuthology.orchestra.run.smithi049.stdout:Description : 2023-07-18T03:16:04.486 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:16:04.486 INFO:teuthology.orchestra.run.smithi049.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-18T03:16:04.486 INFO:teuthology.orchestra.run.smithi049.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-18T03:16:04.486 INFO:teuthology.orchestra.run.smithi049.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-18T03:16:04.486 INFO:teuthology.orchestra.run.smithi049.stdout:hardware devices. 2023-07-18T03:16:04.486 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:16:04.486 INFO:teuthology.orchestra.run.smithi049.stdout:This subpackage provides tools for support of all platforms. 2023-07-18T03:16:04.487 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-18T03:16:04.487 INFO:teuthology.task.kernel:Updating grub on smithi049 to boot 6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:04.488 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-18T03:16:07.263 INFO:teuthology.orchestra.run.smithi049.stderr:Generating grub configuration file ... 2023-07-18T03:16:09.587 INFO:teuthology.orchestra.run.smithi049.stderr:done 2023-07-18T03:16:09.589 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2023-07-18T03:16:09.647 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.7efGJSqQac 2023-07-18T03:16:09.647 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7efGJSqQac 2023-07-18T03:16:09.719 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.7efGJSqQac 2023-07-18T03:16:09.861 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.7efGJSqQac is 6KB 2023-07-18T03:16:09.881 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.7efGJSqQac 2023-07-18T03:16:09.897 DEBUG:teuthology.orchestra.run.smithi049:> sudo /bin/ls /boot/loader/entries || true 2023-07-18T03:16:09.964 INFO:teuthology.orchestra.run.smithi049.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-18T03:16:09.965 INFO:teuthology.orchestra.run.smithi049.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-18T03:16:09.965 INFO:teuthology.orchestra.run.smithi049.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02.conf 2023-07-18T03:16:09.966 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:10.122 DEBUG:teuthology.orchestra.run.smithi049:> sudo shutdown -r now 2023-07-18T03:16:20.292 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-18T03:16:20.292 DEBUG:teuthology.orchestra.run.smithi107:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-18T03:16:20.344 INFO:teuthology.orchestra.run.smithi107.stdout:/boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:20.344 INFO:teuthology.orchestra.run.smithi107.stdout:/boot/config-6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:20.344 INFO:teuthology.orchestra.run.smithi107.stdout:/boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:20.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:20.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel 2023-07-18T03:16:20.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch 2023-07-18T03:16:20.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86 2023-07-18T03:16:20.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto 2023-07-18T03:16:20.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-18T03:16:20.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-18T03:16:20.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-18T03:16:20.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-18T03:16:20.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-18T03:16:20.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-18T03:16:20.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-18T03:16:20.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-18T03:16:20.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-18T03:16:20.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-18T03:16:20.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-18T03:16:20.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-18T03:16:20.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-18T03:16:20.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-18T03:16:20.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-18T03:16:20.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-18T03:16:20.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-18T03:16:20.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-18T03:16:20.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-18T03:16:20.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-18T03:16:20.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel 2023-07-18T03:16:20.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu 2023-07-18T03:16:20.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce 2023-07-18T03:16:20.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-18T03:16:20.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm 2023-07-18T03:16:20.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-18T03:16:20.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-18T03:16:20.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-18T03:16:20.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto 2023-07-18T03:16:20.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-18T03:16:20.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-18T03:16:20.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-18T03:16:20.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx 2023-07-18T03:16:20.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-18T03:16:20.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-18T03:16:20.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-18T03:16:20.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-18T03:16:20.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-18T03:16:20.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-18T03:16:20.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-18T03:16:20.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-18T03:16:20.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-18T03:16:20.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-18T03:16:20.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-18T03:16:20.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-18T03:16:20.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-18T03:16:20.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-18T03:16:20.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-18T03:16:20.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-18T03:16:20.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-18T03:16:20.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-18T03:16:20.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-18T03:16:20.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-18T03:16:20.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-18T03:16:20.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-18T03:16:20.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-18T03:16:20.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-18T03:16:20.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-18T03:16:20.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-18T03:16:20.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-18T03:16:20.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-18T03:16:20.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-18T03:16:20.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-18T03:16:20.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-18T03:16:20.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-18T03:16:20.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-18T03:16:20.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-18T03:16:20.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-18T03:16:20.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-18T03:16:20.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-18T03:16:20.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-18T03:16:20.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-18T03:16:20.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-18T03:16:20.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-18T03:16:20.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-18T03:16:20.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-18T03:16:20.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-18T03:16:20.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-18T03:16:20.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers 2023-07-18T03:16:20.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi 2023-07-18T03:16:20.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-18T03:16:20.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-18T03:16:20.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/custom_method.ko 2023-07-18T03:16:20.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-18T03:16:20.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-18T03:16:20.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-18T03:16:20.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-18T03:16:20.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-18T03:16:20.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata 2023-07-18T03:16:20.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-18T03:16:20.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-18T03:16:20.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ata_generic.ko 2023-07-18T03:16:20.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-18T03:16:20.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-18T03:16:20.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-18T03:16:20.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-18T03:16:20.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-18T03:16:20.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-18T03:16:20.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-18T03:16:20.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-18T03:16:20.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-18T03:16:20.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-18T03:16:20.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-18T03:16:20.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-18T03:16:20.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-18T03:16:20.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-18T03:16:20.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-18T03:16:20.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-18T03:16:20.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-18T03:16:20.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-18T03:16:20.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-18T03:16:20.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-18T03:16:20.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-18T03:16:20.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-18T03:16:20.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-18T03:16:20.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-18T03:16:20.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-18T03:16:20.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-18T03:16:20.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-18T03:16:20.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-18T03:16:20.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-18T03:16:20.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-18T03:16:20.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-18T03:16:20.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-18T03:16:20.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-18T03:16:20.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-18T03:16:20.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-18T03:16:20.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sis.ko 2023-07-18T03:16:20.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-18T03:16:20.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-18T03:16:20.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-18T03:16:20.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-18T03:16:20.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-18T03:16:20.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-18T03:16:20.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-18T03:16:20.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-18T03:16:20.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-18T03:16:20.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-18T03:16:20.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-18T03:16:20.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-18T03:16:20.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-18T03:16:20.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-18T03:16:20.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-18T03:16:20.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-18T03:16:20.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-18T03:16:20.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay 2023-07-18T03:16:20.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-18T03:16:20.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base 2023-07-18T03:16:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap 2023-07-18T03:16:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-18T03:16:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-18T03:16:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma 2023-07-18T03:16:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-18T03:16:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block 2023-07-18T03:16:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe 2023-07-18T03:16:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-18T03:16:20.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/brd.ko 2023-07-18T03:16:20.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd 2023-07-18T03:16:20.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-18T03:16:20.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-18T03:16:20.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/loop.ko 2023-07-18T03:16:20.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx 2023-07-18T03:16:20.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-18T03:16:20.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-18T03:16:20.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk 2023-07-18T03:16:20.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-18T03:16:20.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-18T03:16:20.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-18T03:16:20.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/virtio_blk.ko 2023-07-18T03:16:20.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback 2023-07-18T03:16:20.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-18T03:16:20.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkfront.ko 2023-07-18T03:16:20.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram 2023-07-18T03:16:20.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-18T03:16:20.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char 2023-07-18T03:16:20.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-18T03:16:20.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random 2023-07-18T03:16:20.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-18T03:16:20.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-18T03:16:20.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-18T03:16:20.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-18T03:16:20.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-18T03:16:20.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi 2023-07-18T03:16:20.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-18T03:16:20.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-18T03:16:20.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-18T03:16:20.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-18T03:16:20.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-18T03:16:20.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-18T03:16:20.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-18T03:16:20.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave 2023-07-18T03:16:20.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-18T03:16:20.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-18T03:16:20.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-18T03:16:20.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm 2023-07-18T03:16:20.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm.ko 2023-07-18T03:16:20.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-18T03:16:20.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-18T03:16:20.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-18T03:16:20.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-18T03:16:20.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-18T03:16:20.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/uv_mmtimer.ko 2023-07-18T03:16:20.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/virtio_console.ko 2023-07-18T03:16:20.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq 2023-07-18T03:16:20.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-18T03:16:20.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-18T03:16:20.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-18T03:16:20.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-18T03:16:20.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-18T03:16:20.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-18T03:16:20.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto 2023-07-18T03:16:20.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp 2023-07-18T03:16:20.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-18T03:16:20.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-18T03:16:20.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel 2023-07-18T03:16:20.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat 2023-07-18T03:16:20.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common 2023-07-18T03:16:20.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-18T03:16:20.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-18T03:16:20.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-18T03:16:20.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-18T03:16:20.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-18T03:16:20.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio 2023-07-18T03:16:20.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-18T03:16:20.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca 2023-07-18T03:16:20.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-18T03:16:20.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq 2023-07-18T03:16:20.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-18T03:16:20.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma 2023-07-18T03:16:20.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw 2023-07-18T03:16:20.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-18T03:16:20.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat 2023-07-18T03:16:20.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-18T03:16:20.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac 2023-07-18T03:16:20.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-18T03:16:20.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-18T03:16:20.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-18T03:16:20.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-18T03:16:20.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-18T03:16:20.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-18T03:16:20.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-18T03:16:20.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-18T03:16:20.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-18T03:16:20.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-18T03:16:20.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-18T03:16:20.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-18T03:16:20.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-18T03:16:20.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware 2023-07-18T03:16:20.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/edd.ko 2023-07-18T03:16:20.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-18T03:16:20.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio 2023-07-18T03:16:20.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-18T03:16:20.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-18T03:16:20.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid 2023-07-18T03:16:20.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-18T03:16:20.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-18T03:16:20.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-18T03:16:20.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-18T03:16:20.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-18T03:16:20.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-18T03:16:20.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-18T03:16:20.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-18T03:16:20.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-18T03:16:20.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-18T03:16:20.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-18T03:16:20.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-18T03:16:20.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-18T03:16:20.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-18T03:16:20.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-18T03:16:20.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-18T03:16:20.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-18T03:16:20.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-18T03:16:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-18T03:16:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-18T03:16:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-18T03:16:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-18T03:16:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-18T03:16:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-18T03:16:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-18T03:16:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-18T03:16:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-18T03:16:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-18T03:16:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-18T03:16:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-18T03:16:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-18T03:16:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-18T03:16:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-18T03:16:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-18T03:16:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-18T03:16:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-18T03:16:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-18T03:16:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-18T03:16:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-18T03:16:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-18T03:16:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-18T03:16:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-18T03:16:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-18T03:16:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-18T03:16:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-18T03:16:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-18T03:16:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-18T03:16:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-18T03:16:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-18T03:16:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-18T03:16:20.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-18T03:16:20.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-18T03:16:20.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-18T03:16:20.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-18T03:16:20.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-18T03:16:20.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-18T03:16:20.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-18T03:16:20.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-18T03:16:20.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-18T03:16:20.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-18T03:16:20.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-18T03:16:20.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-18T03:16:20.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-18T03:16:20.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-18T03:16:20.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv 2023-07-18T03:16:20.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-18T03:16:20.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-18T03:16:20.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-18T03:16:20.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon 2023-07-18T03:16:20.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-18T03:16:20.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-18T03:16:20.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-18T03:16:20.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-18T03:16:20.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-18T03:16:20.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-18T03:16:20.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-18T03:16:20.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-18T03:16:20.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-18T03:16:20.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-18T03:16:20.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-18T03:16:20.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-18T03:16:20.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-18T03:16:20.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-18T03:16:20.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-18T03:16:20.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-18T03:16:20.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-18T03:16:20.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-18T03:16:20.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-18T03:16:20.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-18T03:16:20.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-18T03:16:20.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-18T03:16:20.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-18T03:16:20.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-18T03:16:20.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-18T03:16:20.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-18T03:16:20.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-18T03:16:20.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-18T03:16:20.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-18T03:16:20.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-18T03:16:20.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-18T03:16:20.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-18T03:16:20.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-18T03:16:20.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-18T03:16:20.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-18T03:16:20.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-18T03:16:20.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-18T03:16:20.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-18T03:16:20.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-18T03:16:20.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-18T03:16:20.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-18T03:16:20.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-18T03:16:20.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-18T03:16:20.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-18T03:16:20.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-18T03:16:20.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-18T03:16:20.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-18T03:16:20.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-18T03:16:20.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-18T03:16:20.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-18T03:16:20.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-18T03:16:20.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-18T03:16:20.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-18T03:16:20.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-18T03:16:20.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-18T03:16:20.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-18T03:16:20.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-18T03:16:20.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-18T03:16:20.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-18T03:16:20.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-18T03:16:20.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-18T03:16:20.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-18T03:16:20.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-18T03:16:20.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-18T03:16:20.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-18T03:16:20.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-18T03:16:20.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-18T03:16:20.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-18T03:16:20.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-18T03:16:20.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-18T03:16:20.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-18T03:16:20.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-18T03:16:20.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-18T03:16:20.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-18T03:16:20.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-18T03:16:20.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-18T03:16:20.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-18T03:16:20.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-18T03:16:20.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-18T03:16:20.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-18T03:16:20.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-18T03:16:20.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-18T03:16:20.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-18T03:16:20.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-18T03:16:20.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-18T03:16:20.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-18T03:16:20.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-18T03:16:20.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-18T03:16:20.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus 2023-07-18T03:16:20.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-18T03:16:20.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-18T03:16:20.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-18T03:16:20.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-18T03:16:20.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-18T03:16:20.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-18T03:16:20.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-18T03:16:20.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-18T03:16:20.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-18T03:16:20.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-18T03:16:20.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-18T03:16:20.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-18T03:16:20.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-18T03:16:20.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-18T03:16:20.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-18T03:16:20.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-18T03:16:20.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-18T03:16:20.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-18T03:16:20.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-18T03:16:20.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-18T03:16:20.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-18T03:16:20.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-18T03:16:20.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-18T03:16:20.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-18T03:16:20.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-18T03:16:20.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-18T03:16:20.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-18T03:16:20.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-18T03:16:20.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-18T03:16:20.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-18T03:16:20.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-18T03:16:20.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-18T03:16:20.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-18T03:16:20.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-18T03:16:20.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-18T03:16:20.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-18T03:16:20.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-18T03:16:20.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-18T03:16:20.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-18T03:16:20.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-18T03:16:20.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-18T03:16:20.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c 2023-07-18T03:16:20.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos 2023-07-18T03:16:20.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-18T03:16:20.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-18T03:16:20.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses 2023-07-18T03:16:20.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-18T03:16:20.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-18T03:16:20.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-18T03:16:20.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-18T03:16:20.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-18T03:16:20.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-18T03:16:20.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-18T03:16:20.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-18T03:16:20.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-18T03:16:20.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-18T03:16:20.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-18T03:16:20.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-18T03:16:20.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-18T03:16:20.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-18T03:16:20.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-18T03:16:20.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-18T03:16:20.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-18T03:16:20.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-18T03:16:20.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-18T03:16:20.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-18T03:16:20.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-18T03:16:20.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-18T03:16:20.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-18T03:16:20.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-dev.ko 2023-07-18T03:16:20.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-18T03:16:20.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-18T03:16:20.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-18T03:16:20.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-18T03:16:20.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband 2023-07-18T03:16:20.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core 2023-07-18T03:16:20.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-18T03:16:20.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-18T03:16:20.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-18T03:16:20.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-18T03:16:20.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-18T03:16:20.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-18T03:16:20.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-18T03:16:20.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw 2023-07-18T03:16:20.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4 2023-07-18T03:16:20.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-18T03:16:20.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4 2023-07-18T03:16:20.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-18T03:16:20.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5 2023-07-18T03:16:20.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-18T03:16:20.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca 2023-07-18T03:16:20.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-18T03:16:20.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp 2023-07-18T03:16:20.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib 2023-07-18T03:16:20.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-18T03:16:20.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser 2023-07-18T03:16:20.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-18T03:16:20.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert 2023-07-18T03:16:20.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-18T03:16:20.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp 2023-07-18T03:16:20.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-18T03:16:20.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt 2023-07-18T03:16:20.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-18T03:16:20.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input 2023-07-18T03:16:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport 2023-07-18T03:16:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-18T03:16:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-18T03:16:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-18T03:16:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-18T03:16:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-18T03:16:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-18T03:16:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard 2023-07-18T03:16:20.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-18T03:16:20.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse 2023-07-18T03:16:20.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-18T03:16:20.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-18T03:16:20.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-18T03:16:20.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-18T03:16:20.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-18T03:16:20.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-18T03:16:20.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-18T03:16:20.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-18T03:16:20.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4 2023-07-18T03:16:20.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-18T03:16:20.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio 2023-07-18T03:16:20.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-18T03:16:20.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-18T03:16:20.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-18T03:16:20.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-18T03:16:20.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-18T03:16:20.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu 2023-07-18T03:16:20.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd 2023-07-18T03:16:20.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-18T03:16:20.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds 2023-07-18T03:16:20.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-18T03:16:20.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-18T03:16:20.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-18T03:16:20.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-18T03:16:20.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger 2023-07-18T03:16:20.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-18T03:16:20.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-18T03:16:20.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-18T03:16:20.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-18T03:16:20.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-18T03:16:20.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-18T03:16:20.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-18T03:16:20.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-18T03:16:20.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md 2023-07-18T03:16:20.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache 2023-07-18T03:16:20.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-18T03:16:20.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-18T03:16:20.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-18T03:16:20.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-18T03:16:20.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-18T03:16:20.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-18T03:16:20.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-18T03:16:20.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-18T03:16:20.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-18T03:16:20.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-18T03:16:20.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-18T03:16:20.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-18T03:16:20.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-18T03:16:20.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-18T03:16:20.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-18T03:16:20.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-18T03:16:20.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-18T03:16:20.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-18T03:16:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-18T03:16:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data 2023-07-18T03:16:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-18T03:16:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-18T03:16:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-18T03:16:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-18T03:16:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-18T03:16:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message 2023-07-18T03:16:20.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion 2023-07-18T03:16:20.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-18T03:16:20.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-18T03:16:20.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-18T03:16:20.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-18T03:16:20.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-18T03:16:20.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-18T03:16:20.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-18T03:16:20.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd 2023-07-18T03:16:20.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-18T03:16:20.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-18T03:16:20.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mfd-core.ko 2023-07-18T03:16:20.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-18T03:16:20.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-18T03:16:20.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-18T03:16:20.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-18T03:16:20.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc 2023-07-18T03:16:20.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl 2023-07-18T03:16:20.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-18T03:16:20.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-18T03:16:20.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-18T03:16:20.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-18T03:16:20.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710 2023-07-18T03:16:20.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-18T03:16:20.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom 2023-07-18T03:16:20.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-18T03:16:20.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-18T03:16:20.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-18T03:16:20.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-18T03:16:20.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-18T03:16:20.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-18T03:16:20.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-18T03:16:20.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-18T03:16:20.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d 2023-07-18T03:16:20.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-18T03:16:20.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-18T03:16:20.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei 2023-07-18T03:16:20.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-18T03:16:20.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-18T03:16:20.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-18T03:16:20.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru 2023-07-18T03:16:20.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-18T03:16:20.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp 2023-07-18T03:16:20.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-18T03:16:20.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-18T03:16:20.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-18T03:16:20.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-18T03:16:20.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-18T03:16:20.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-18T03:16:20.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-18T03:16:20.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci 2023-07-18T03:16:20.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-18T03:16:20.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net 2023-07-18T03:16:20.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding 2023-07-18T03:16:20.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-18T03:16:20.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa 2023-07-18T03:16:20.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53 2023-07-18T03:16:20.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-18T03:16:20.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-18T03:16:20.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-18T03:16:20.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx 2023-07-18T03:16:20.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-18T03:16:20.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-18T03:16:20.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-18T03:16:20.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet 2023-07-18T03:16:20.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com 2023-07-18T03:16:20.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-18T03:16:20.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-18T03:16:20.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-18T03:16:20.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-18T03:16:20.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390 2023-07-18T03:16:20.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-18T03:16:20.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-18T03:16:20.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-18T03:16:20.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-18T03:16:20.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec 2023-07-18T03:16:20.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-18T03:16:20.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere 2023-07-18T03:16:20.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-18T03:16:20.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon 2023-07-18T03:16:20.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-18T03:16:20.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera 2023-07-18T03:16:20.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-18T03:16:20.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd 2023-07-18T03:16:20.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-18T03:16:20.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-18T03:16:20.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-18T03:16:20.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros 2023-07-18T03:16:20.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx 2023-07-18T03:16:20.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-18T03:16:20.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-18T03:16:20.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-18T03:16:20.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-18T03:16:20.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-18T03:16:20.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx 2023-07-18T03:16:20.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-18T03:16:20.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-18T03:16:20.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom 2023-07-18T03:16:20.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-18T03:16:20.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-18T03:16:20.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-18T03:16:20.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-18T03:16:20.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-18T03:16:20.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet 2023-07-18T03:16:20.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-18T03:16:20.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-18T03:16:20.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade 2023-07-18T03:16:20.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna 2023-07-18T03:16:20.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-18T03:16:20.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio 2023-07-18T03:16:20.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-18T03:16:20.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-18T03:16:20.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-18T03:16:20.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-18T03:16:20.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-18T03:16:20.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-18T03:16:20.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-18T03:16:20.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-18T03:16:20.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-18T03:16:20.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-18T03:16:20.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco 2023-07-18T03:16:20.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic 2023-07-18T03:16:20.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-18T03:16:20.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec 2023-07-18T03:16:20.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip 2023-07-18T03:16:20.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-18T03:16:20.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-18T03:16:20.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-18T03:16:20.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-18T03:16:20.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-18T03:16:20.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-18T03:16:20.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink 2023-07-18T03:16:20.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-18T03:16:20.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-18T03:16:20.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-18T03:16:20.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex 2023-07-18T03:16:20.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet 2023-07-18T03:16:20.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-18T03:16:20.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-18T03:16:20.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-18T03:16:20.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel 2023-07-18T03:16:20.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-18T03:16:20.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000 2023-07-18T03:16:20.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-18T03:16:20.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e 2023-07-18T03:16:20.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-18T03:16:20.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k 2023-07-18T03:16:20.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-18T03:16:20.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e 2023-07-18T03:16:20.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-18T03:16:20.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf 2023-07-18T03:16:20.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-18T03:16:20.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb 2023-07-18T03:16:20.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-18T03:16:20.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf 2023-07-18T03:16:20.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-18T03:16:20.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-18T03:16:20.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-18T03:16:20.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-18T03:16:20.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-18T03:16:20.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-18T03:16:20.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell 2023-07-18T03:16:20.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-18T03:16:20.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-18T03:16:20.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-18T03:16:20.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox 2023-07-18T03:16:20.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-18T03:16:20.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-18T03:16:20.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-18T03:16:20.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-18T03:16:20.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-18T03:16:20.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-18T03:16:20.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel 2023-07-18T03:16:20.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-18T03:16:20.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom 2023-07-18T03:16:20.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-18T03:16:20.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-18T03:16:20.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi 2023-07-18T03:16:20.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-18T03:16:20.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-18T03:16:20.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion 2023-07-18T03:16:20.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-18T03:16:20.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia 2023-07-18T03:16:20.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-18T03:16:20.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines 2023-07-18T03:16:20.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-18T03:16:20.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-18T03:16:20.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic 2023-07-18T03:16:20.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-18T03:16:20.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-18T03:16:20.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-18T03:16:20.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-18T03:16:20.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-18T03:16:20.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc 2023-07-18T03:16:20.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-18T03:16:20.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek 2023-07-18T03:16:20.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-18T03:16:20.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-18T03:16:20.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-18T03:16:20.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-18T03:16:20.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker 2023-07-18T03:16:20.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-18T03:16:20.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc 2023-07-18T03:16:20.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-18T03:16:20.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan 2023-07-18T03:16:20.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-18T03:16:20.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis 2023-07-18T03:16:20.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-18T03:16:20.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-18T03:16:20.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc 2023-07-18T03:16:20.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-18T03:16:20.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-18T03:16:20.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-18T03:16:20.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-18T03:16:20.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro 2023-07-18T03:16:20.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-18T03:16:20.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-18T03:16:20.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-18T03:16:20.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun 2023-07-18T03:16:20.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-18T03:16:20.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-18T03:16:20.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-18T03:16:20.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-18T03:16:20.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti 2023-07-18T03:16:20.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-18T03:16:20.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti 2023-07-18T03:16:20.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-18T03:16:20.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via 2023-07-18T03:16:20.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-18T03:16:20.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-18T03:16:20.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet 2023-07-18T03:16:20.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-18T03:16:20.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-18T03:16:20.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom 2023-07-18T03:16:20.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-18T03:16:20.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-18T03:16:20.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv 2023-07-18T03:16:20.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-18T03:16:20.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154 2023-07-18T03:16:20.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-18T03:16:20.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-18T03:16:20.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan 2023-07-18T03:16:20.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-18T03:16:20.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-18T03:16:20.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-18T03:16:20.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio 2023-07-18T03:16:20.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-18T03:16:20.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-18T03:16:20.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-18T03:16:20.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-18T03:16:20.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-18T03:16:20.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/net_failover.ko 2023-07-18T03:16:20.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-18T03:16:20.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-18T03:16:20.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs 2023-07-18T03:16:20.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-18T03:16:20.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-18T03:16:20.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy 2023-07-18T03:16:20.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-18T03:16:20.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-18T03:16:20.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-18T03:16:20.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-18T03:16:20.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-18T03:16:20.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-18T03:16:20.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-18T03:16:20.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-18T03:16:20.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/dp83640.ko 2023-07-18T03:16:20.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-18T03:16:20.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-18T03:16:20.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-18T03:16:20.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-18T03:16:20.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-18T03:16:20.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-18T03:16:20.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-18T03:16:20.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-18T03:16:20.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-18T03:16:20.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-18T03:16:20.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-18T03:16:20.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-18T03:16:20.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp 2023-07-18T03:16:20.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-18T03:16:20.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-18T03:16:20.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-18T03:16:20.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-18T03:16:20.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-18T03:16:20.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-18T03:16:20.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-18T03:16:20.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-18T03:16:20.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-18T03:16:20.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip 2023-07-18T03:16:20.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slhc.ko 2023-07-18T03:16:20.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-18T03:16:20.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-18T03:16:20.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-18T03:16:20.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team 2023-07-18T03:16:20.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-18T03:16:20.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-18T03:16:20.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-18T03:16:20.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-18T03:16:20.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-18T03:16:20.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-18T03:16:20.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tun.ko 2023-07-18T03:16:20.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-18T03:16:20.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/virtio_net.ko 2023-07-18T03:16:20.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3 2023-07-18T03:16:20.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-18T03:16:20.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan 2023-07-18T03:16:20.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-18T03:16:20.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback 2023-07-18T03:16:20.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-18T03:16:20.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netfront.ko 2023-07-18T03:16:20.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb 2023-07-18T03:16:20.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-18T03:16:20.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme 2023-07-18T03:16:20.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host 2023-07-18T03:16:20.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-18T03:16:20.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-18T03:16:20.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport 2023-07-18T03:16:20.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-18T03:16:20.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-18T03:16:20.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-18T03:16:20.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-18T03:16:20.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci 2023-07-18T03:16:20.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug 2023-07-18T03:16:20.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-18T03:16:20.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie 2023-07-18T03:16:20.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-18T03:16:20.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-18T03:16:20.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia 2023-07-18T03:16:20.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-18T03:16:20.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-18T03:16:20.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-18T03:16:20.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl 2023-07-18T03:16:20.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel 2023-07-18T03:16:20.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-18T03:16:20.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform 2023-07-18T03:16:20.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86 2023-07-18T03:16:20.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-18T03:16:20.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-18T03:16:20.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-18T03:16:20.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-18T03:16:20.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-18T03:16:20.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-18T03:16:20.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-18T03:16:20.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-18T03:16:20.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-18T03:16:20.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-18T03:16:20.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-18T03:16:20.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-18T03:16:20.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel 2023-07-18T03:16:20.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-18T03:16:20.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-18T03:16:20.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-18T03:16:20.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-18T03:16:20.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-18T03:16:20.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-18T03:16:20.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-18T03:16:20.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-18T03:16:20.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-18T03:16:20.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-18T03:16:20.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-18T03:16:20.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-18T03:16:20.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap 2023-07-18T03:16:20.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-18T03:16:20.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-18T03:16:20.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps 2023-07-18T03:16:20.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients 2023-07-18T03:16:20.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-18T03:16:20.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-18T03:16:20.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-18T03:16:20.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-18T03:16:20.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp 2023-07-18T03:16:20.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-18T03:16:20.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-18T03:16:20.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc 2023-07-18T03:16:20.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-18T03:16:20.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-18T03:16:20.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-18T03:16:20.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-18T03:16:20.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-18T03:16:20.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-18T03:16:20.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-18T03:16:20.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-18T03:16:20.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-18T03:16:20.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-18T03:16:20.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-18T03:16:20.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-18T03:16:20.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-18T03:16:20.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-18T03:16:20.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-18T03:16:20.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-18T03:16:20.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-18T03:16:20.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-18T03:16:20.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-18T03:16:20.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-18T03:16:20.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-18T03:16:20.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-18T03:16:20.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-18T03:16:20.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-18T03:16:20.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-18T03:16:20.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-18T03:16:20.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-18T03:16:20.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-18T03:16:20.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-18T03:16:20.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-18T03:16:20.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-18T03:16:20.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-18T03:16:20.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi 2023-07-18T03:16:20.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-18T03:16:20.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-18T03:16:20.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-18T03:16:20.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-18T03:16:20.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-18T03:16:20.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid 2023-07-18T03:16:20.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-18T03:16:20.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-18T03:16:20.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx 2023-07-18T03:16:20.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-18T03:16:20.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-18T03:16:20.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-18T03:16:20.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr 2023-07-18T03:16:20.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-18T03:16:20.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-18T03:16:20.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi 2023-07-18T03:16:20.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-18T03:16:20.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa 2023-07-18T03:16:20.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-18T03:16:20.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc 2023-07-18T03:16:20.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-18T03:16:20.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i 2023-07-18T03:16:20.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-18T03:16:20.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-18T03:16:20.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor 2023-07-18T03:16:20.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-18T03:16:20.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi 2023-07-18T03:16:20.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-18T03:16:20.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-18T03:16:20.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-18T03:16:20.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-18T03:16:20.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-18T03:16:20.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-18T03:16:20.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler 2023-07-18T03:16:20.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-18T03:16:20.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-18T03:16:20.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-18T03:16:20.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-18T03:16:20.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-18T03:16:20.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r 2023-07-18T03:16:20.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-18T03:16:20.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-18T03:16:20.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe 2023-07-18T03:16:20.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-18T03:16:20.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-18T03:16:20.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic 2023-07-18T03:16:20.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-18T03:16:20.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-18T03:16:20.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-18T03:16:20.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-18T03:16:20.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-18T03:16:20.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-18T03:16:20.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-18T03:16:20.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci 2023-07-18T03:16:20.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-18T03:16:20.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-18T03:16:20.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-18T03:16:20.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc 2023-07-18T03:16:20.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-18T03:16:20.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-18T03:16:20.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-18T03:16:20.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas 2023-07-18T03:16:20.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-18T03:16:20.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc 2023-07-18T03:16:20.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-18T03:16:20.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid 2023-07-18T03:16:20.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-18T03:16:20.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-18T03:16:20.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-18T03:16:20.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-18T03:16:20.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas 2023-07-18T03:16:20.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-18T03:16:20.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas 2023-07-18T03:16:20.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-18T03:16:20.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-18T03:16:20.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001 2023-07-18T03:16:20.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-18T03:16:20.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-18T03:16:20.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-18T03:16:20.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx 2023-07-18T03:16:20.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-18T03:16:20.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-18T03:16:20.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx 2023-07-18T03:16:20.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-18T03:16:20.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-18T03:16:20.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-18T03:16:20.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-18T03:16:20.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-18T03:16:20.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-18T03:16:20.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-18T03:16:20.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-18T03:16:20.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-18T03:16:20.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-18T03:16:20.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-18T03:16:20.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2 2023-07-18T03:16:20.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-18T03:16:20.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-18T03:16:20.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-18T03:16:20.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-18T03:16:20.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb 2023-07-18T03:16:20.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-18T03:16:20.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target 2023-07-18T03:16:20.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi 2023-07-18T03:16:20.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-18T03:16:20.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback 2023-07-18T03:16:20.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-18T03:16:20.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-18T03:16:20.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-18T03:16:20.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-18T03:16:20.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-18T03:16:20.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-18T03:16:20.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc 2023-07-18T03:16:20.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-18T03:16:20.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal 2023-07-18T03:16:20.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel 2023-07-18T03:16:20.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal 2023-07-18T03:16:20.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-18T03:16:20.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-18T03:16:20.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-18T03:16:20.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-18T03:16:20.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-18T03:16:20.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-18T03:16:20.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-18T03:16:20.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-18T03:16:20.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-18T03:16:20.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-18T03:16:20.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-18T03:16:20.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-18T03:16:20.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-18T03:16:20.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-18T03:16:20.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-18T03:16:20.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty 2023-07-18T03:16:20.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless 2023-07-18T03:16:20.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-18T03:16:20.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-18T03:16:20.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-18T03:16:20.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-18T03:16:20.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial 2023-07-18T03:16:20.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250 2023-07-18T03:16:20.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-18T03:16:20.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-18T03:16:20.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm 2023-07-18T03:16:20.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-18T03:16:20.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-18T03:16:20.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs 2023-07-18T03:16:20.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core 2023-07-18T03:16:20.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-18T03:16:20.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host 2023-07-18T03:16:20.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-18T03:16:20.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio 2023-07-18T03:16:20.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-18T03:16:20.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-18T03:16:20.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-18T03:16:20.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-18T03:16:20.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-18T03:16:20.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb 2023-07-18T03:16:20.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class 2023-07-18T03:16:20.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-18T03:16:20.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-18T03:16:20.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-18T03:16:20.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-18T03:16:20.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host 2023-07-18T03:16:20.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-18T03:16:20.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image 2023-07-18T03:16:20.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-18T03:16:20.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-18T03:16:20.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc 2023-07-18T03:16:20.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-18T03:16:20.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-18T03:16:20.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-18T03:16:20.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-18T03:16:20.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-18T03:16:20.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-18T03:16:20.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-18T03:16:20.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-18T03:16:20.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-18T03:16:20.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-18T03:16:20.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga 2023-07-18T03:16:20.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-18T03:16:20.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-18T03:16:20.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-18T03:16:20.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-18T03:16:20.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-18T03:16:20.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-18T03:16:20.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-18T03:16:20.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy 2023-07-18T03:16:20.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-18T03:16:20.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial 2023-07-18T03:16:20.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-18T03:16:20.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-18T03:16:20.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-18T03:16:20.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-18T03:16:20.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-18T03:16:20.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-18T03:16:20.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-18T03:16:20.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-18T03:16:20.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-18T03:16:20.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-18T03:16:20.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-18T03:16:20.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-18T03:16:20.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-18T03:16:20.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-18T03:16:20.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-18T03:16:20.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-18T03:16:20.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-18T03:16:20.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-18T03:16:20.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-18T03:16:20.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-18T03:16:20.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-18T03:16:20.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-18T03:16:20.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-18T03:16:20.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-18T03:16:20.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-18T03:16:20.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-18T03:16:20.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-18T03:16:20.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-18T03:16:20.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-18T03:16:20.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-18T03:16:20.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-18T03:16:20.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-18T03:16:20.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-18T03:16:20.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-18T03:16:20.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-18T03:16:20.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-18T03:16:20.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-18T03:16:20.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-18T03:16:20.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-18T03:16:20.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-18T03:16:20.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-18T03:16:20.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-18T03:16:20.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-18T03:16:20.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-18T03:16:20.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-18T03:16:20.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage 2023-07-18T03:16:20.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-18T03:16:20.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-18T03:16:20.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-18T03:16:20.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-18T03:16:20.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-18T03:16:20.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-18T03:16:20.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-18T03:16:20.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-18T03:16:20.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-18T03:16:20.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-18T03:16:20.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-18T03:16:20.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-18T03:16:20.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-18T03:16:20.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-18T03:16:20.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-18T03:16:20.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip 2023-07-18T03:16:20.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-18T03:16:20.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-18T03:16:20.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-18T03:16:20.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio 2023-07-18T03:16:20.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci 2023-07-18T03:16:20.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-18T03:16:20.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-18T03:16:20.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-18T03:16:20.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-18T03:16:20.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost 2023-07-18T03:16:20.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-18T03:16:20.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-18T03:16:20.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-18T03:16:20.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-18T03:16:20.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video 2023-07-18T03:16:20.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight 2023-07-18T03:16:20.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-18T03:16:20.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-18T03:16:20.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-18T03:16:20.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio 2023-07-18T03:16:20.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio.ko 2023-07-18T03:16:20.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_balloon.ko 2023-07-18T03:16:20.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_mmio.ko 2023-07-18T03:16:20.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci.ko 2023-07-18T03:16:20.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-18T03:16:20.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-18T03:16:20.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_ring.ko 2023-07-18T03:16:20.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1 2023-07-18T03:16:20.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters 2023-07-18T03:16:20.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-18T03:16:20.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-18T03:16:20.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves 2023-07-18T03:16:20.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-18T03:16:20.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-18T03:16:20.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-18T03:16:20.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-18T03:16:20.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-18T03:16:20.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-18T03:16:20.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-18T03:16:20.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-18T03:16:20.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-18T03:16:20.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-18T03:16:20.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-18T03:16:20.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-18T03:16:20.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog 2023-07-18T03:16:20.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-18T03:16:20.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-18T03:16:20.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-18T03:16:20.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-18T03:16:20.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-18T03:16:20.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-18T03:16:20.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-18T03:16:20.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-18T03:16:20.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-18T03:16:20.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-18T03:16:20.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-18T03:16:20.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-18T03:16:20.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-18T03:16:20.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-18T03:16:20.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-18T03:16:20.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-18T03:16:20.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-18T03:16:20.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-18T03:16:20.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-18T03:16:20.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-18T03:16:20.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-18T03:16:20.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-18T03:16:20.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-18T03:16:20.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-18T03:16:20.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-18T03:16:20.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-18T03:16:20.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen 2023-07-18T03:16:20.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-18T03:16:20.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-18T03:16:20.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-18T03:16:20.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-18T03:16:20.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback 2023-07-18T03:16:20.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-18T03:16:20.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-18T03:16:20.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs 2023-07-18T03:16:20.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-18T03:16:20.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs 2023-07-18T03:16:20.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p 2023-07-18T03:16:20.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-18T03:16:20.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs 2023-07-18T03:16:20.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-18T03:16:20.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs 2023-07-18T03:16:20.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-18T03:16:20.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-18T03:16:20.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs 2023-07-18T03:16:20.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-18T03:16:20.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles 2023-07-18T03:16:20.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-18T03:16:20.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph 2023-07-18T03:16:20.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-18T03:16:20.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda 2023-07-18T03:16:20.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-18T03:16:20.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs 2023-07-18T03:16:20.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-18T03:16:20.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm 2023-07-18T03:16:20.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-18T03:16:20.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs 2023-07-18T03:16:20.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-18T03:16:20.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs 2023-07-18T03:16:20.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-18T03:16:20.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat 2023-07-18T03:16:20.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/fat.ko 2023-07-18T03:16:20.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-18T03:16:20.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/vfat.ko 2023-07-18T03:16:20.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache 2023-07-18T03:16:20.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-18T03:16:20.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse 2023-07-18T03:16:20.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-18T03:16:20.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/fuse.ko 2023-07-18T03:16:20.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2 2023-07-18T03:16:20.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-18T03:16:20.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs 2023-07-18T03:16:20.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-18T03:16:20.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus 2023-07-18T03:16:20.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-18T03:16:20.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs 2023-07-18T03:16:20.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-18T03:16:20.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs 2023-07-18T03:16:20.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-18T03:16:20.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd 2023-07-18T03:16:20.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-18T03:16:20.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix 2023-07-18T03:16:20.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-18T03:16:20.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs 2023-07-18T03:16:20.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-18T03:16:20.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs 2023-07-18T03:16:20.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout 2023-07-18T03:16:20.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-18T03:16:20.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout 2023-07-18T03:16:20.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-18T03:16:20.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout 2023-07-18T03:16:20.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-18T03:16:20.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-18T03:16:20.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-18T03:16:20.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-18T03:16:20.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common 2023-07-18T03:16:20.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-18T03:16:20.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-18T03:16:20.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd 2023-07-18T03:16:20.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-18T03:16:20.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2 2023-07-18T03:16:20.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-18T03:16:20.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls 2023-07-18T03:16:20.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-18T03:16:20.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-18T03:16:20.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-18T03:16:20.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-18T03:16:20.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-18T03:16:20.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-18T03:16:20.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-18T03:16:20.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-18T03:16:20.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-18T03:16:20.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-18T03:16:20.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-18T03:16:20.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-18T03:16:20.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-18T03:16:20.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-18T03:16:20.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-18T03:16:20.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-18T03:16:20.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-18T03:16:20.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-18T03:16:20.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-18T03:16:20.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-18T03:16:20.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-18T03:16:20.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-18T03:16:20.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-18T03:16:20.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-18T03:16:20.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-18T03:16:20.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-18T03:16:20.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-18T03:16:20.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-18T03:16:20.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-18T03:16:20.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-18T03:16:20.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-18T03:16:20.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-18T03:16:20.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-18T03:16:20.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-18T03:16:20.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-18T03:16:20.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-18T03:16:20.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-18T03:16:20.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-18T03:16:20.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-18T03:16:20.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-18T03:16:20.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-18T03:16:20.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-18T03:16:20.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-18T03:16:20.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-18T03:16:20.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-18T03:16:20.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-18T03:16:20.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-18T03:16:20.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-18T03:16:20.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-18T03:16:20.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2 2023-07-18T03:16:20.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster 2023-07-18T03:16:20.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-18T03:16:20.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm 2023-07-18T03:16:20.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-18T03:16:20.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs 2023-07-18T03:16:20.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-18T03:16:20.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-18T03:16:20.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-18T03:16:20.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-18T03:16:20.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-18T03:16:20.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs 2023-07-18T03:16:20.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-18T03:16:20.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore 2023-07-18T03:16:20.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-18T03:16:20.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs 2023-07-18T03:16:20.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-18T03:16:20.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs 2023-07-18T03:16:20.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-18T03:16:20.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb 2023-07-18T03:16:20.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client 2023-07-18T03:16:20.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-18T03:16:20.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common 2023-07-18T03:16:20.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-18T03:16:20.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-18T03:16:20.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs 2023-07-18T03:16:20.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-18T03:16:20.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv 2023-07-18T03:16:20.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-18T03:16:20.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf 2023-07-18T03:16:20.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-18T03:16:20.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs 2023-07-18T03:16:20.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-18T03:16:20.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs 2023-07-18T03:16:20.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-18T03:16:20.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel 2023-07-18T03:16:20.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu 2023-07-18T03:16:20.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu/rcutorture.ko 2023-07-18T03:16:20.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-18T03:16:20.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace 2023-07-18T03:16:20.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-18T03:16:20.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib 2023-07-18T03:16:20.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/asn1_encoder.ko 2023-07-18T03:16:20.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-18T03:16:20.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-18T03:16:20.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto 2023-07-18T03:16:20.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-18T03:16:20.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-18T03:16:20.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-18T03:16:20.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-18T03:16:20.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4 2023-07-18T03:16:20.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-18T03:16:20.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-18T03:16:20.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math 2023-07-18T03:16:20.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-18T03:16:20.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6 2023-07-18T03:16:20.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-18T03:16:20.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon 2023-07-18T03:16:20.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-18T03:16:20.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-18T03:16:20.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-18T03:16:20.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-18T03:16:20.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd 2023-07-18T03:16:20.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd/zstd_compress.ko 2023-07-18T03:16:20.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm 2023-07-18T03:16:20.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-18T03:16:20.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net 2023-07-18T03:16:20.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan 2023-07-18T03:16:20.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-18T03:16:20.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-18T03:16:20.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-18T03:16:20.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-18T03:16:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-18T03:16:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-18T03:16:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-18T03:16:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-18T03:16:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802 2023-07-18T03:16:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-18T03:16:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-18T03:16:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-18T03:16:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-18T03:16:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-18T03:16:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q 2023-07-18T03:16:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-18T03:16:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p 2023-07-18T03:16:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-18T03:16:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-18T03:16:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-18T03:16:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-18T03:16:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv 2023-07-18T03:16:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-18T03:16:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge 2023-07-18T03:16:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-18T03:16:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-18T03:16:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter 2023-07-18T03:16:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-18T03:16:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-18T03:16:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-18T03:16:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-18T03:16:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-18T03:16:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-18T03:16:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-18T03:16:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-18T03:16:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-18T03:16:20.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-18T03:16:20.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-18T03:16:20.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-18T03:16:20.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-18T03:16:20.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-18T03:16:20.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-18T03:16:20.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-18T03:16:20.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-18T03:16:20.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-18T03:16:20.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-18T03:16:20.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-18T03:16:20.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-18T03:16:20.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-18T03:16:20.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-18T03:16:20.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-18T03:16:20.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph 2023-07-18T03:16:20.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-18T03:16:20.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core 2023-07-18T03:16:20.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/failover.ko 2023-07-18T03:16:20.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-18T03:16:20.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp 2023-07-18T03:16:20.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-18T03:16:20.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-18T03:16:20.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-18T03:16:20.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-18T03:16:20.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver 2023-07-18T03:16:20.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver/dns_resolver.ko 2023-07-18T03:16:20.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa 2023-07-18T03:16:20.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-18T03:16:20.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-18T03:16:20.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-18T03:16:20.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-18T03:16:20.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-18T03:16:20.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154 2023-07-18T03:16:20.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan 2023-07-18T03:16:20.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-18T03:16:20.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-18T03:16:20.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-18T03:16:20.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4 2023-07-18T03:16:20.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-18T03:16:20.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-18T03:16:20.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-18T03:16:20.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-18T03:16:20.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-18T03:16:20.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-18T03:16:20.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-18T03:16:20.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-18T03:16:20.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-18T03:16:20.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-18T03:16:20.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter 2023-07-18T03:16:20.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-18T03:16:20.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-18T03:16:20.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-18T03:16:20.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-18T03:16:20.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-18T03:16:20.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-18T03:16:20.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-18T03:16:20.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-18T03:16:20.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-18T03:16:20.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-18T03:16:20.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-18T03:16:20.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-18T03:16:20.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-18T03:16:20.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-18T03:16:20.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-18T03:16:20.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-18T03:16:20.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-18T03:16:20.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-18T03:16:20.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-18T03:16:20.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-18T03:16:20.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-18T03:16:20.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-18T03:16:20.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-18T03:16:20.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-18T03:16:20.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-18T03:16:20.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-18T03:16:20.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-18T03:16:20.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-18T03:16:20.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-18T03:16:20.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-18T03:16:20.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-18T03:16:20.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-18T03:16:20.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-18T03:16:20.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-18T03:16:20.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-18T03:16:20.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-18T03:16:20.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-18T03:16:20.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-18T03:16:20.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6 2023-07-18T03:16:20.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-18T03:16:20.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-18T03:16:20.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-18T03:16:20.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-18T03:16:20.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-18T03:16:20.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-18T03:16:20.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-18T03:16:20.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter 2023-07-18T03:16:20.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-18T03:16:20.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-18T03:16:20.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-18T03:16:20.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-18T03:16:20.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-18T03:16:20.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-18T03:16:20.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-18T03:16:20.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-18T03:16:20.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-18T03:16:20.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-18T03:16:20.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-18T03:16:20.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-18T03:16:20.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-18T03:16:20.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-18T03:16:20.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-18T03:16:20.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-18T03:16:20.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-18T03:16:20.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-18T03:16:20.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-18T03:16:20.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-18T03:16:20.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-18T03:16:20.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-18T03:16:20.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-18T03:16:20.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-18T03:16:20.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-18T03:16:20.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-18T03:16:20.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-18T03:16:20.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key 2023-07-18T03:16:20.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-18T03:16:20.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp 2023-07-18T03:16:20.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-18T03:16:20.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-18T03:16:20.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-18T03:16:20.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-18T03:16:20.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-18T03:16:20.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-18T03:16:20.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-18T03:16:20.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc 2023-07-18T03:16:20.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-18T03:16:20.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154 2023-07-18T03:16:20.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-18T03:16:20.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls 2023-07-18T03:16:20.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-18T03:16:20.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter 2023-07-18T03:16:20.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset 2023-07-18T03:16:20.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-18T03:16:20.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-18T03:16:20.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-18T03:16:20.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-18T03:16:20.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-18T03:16:20.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-18T03:16:20.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-18T03:16:20.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-18T03:16:20.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-18T03:16:20.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-18T03:16:20.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-18T03:16:20.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-18T03:16:20.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-18T03:16:20.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-18T03:16:20.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-18T03:16:20.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-18T03:16:20.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs 2023-07-18T03:16:20.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-18T03:16:20.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-18T03:16:20.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-18T03:16:20.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-18T03:16:20.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-18T03:16:20.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-18T03:16:20.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-18T03:16:20.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-18T03:16:20.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-18T03:16:20.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-18T03:16:20.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-18T03:16:20.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-18T03:16:20.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-18T03:16:20.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-18T03:16:20.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-18T03:16:20.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-18T03:16:20.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-18T03:16:20.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-18T03:16:20.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-18T03:16:20.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-18T03:16:20.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-18T03:16:20.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-18T03:16:20.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-18T03:16:20.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-18T03:16:20.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-18T03:16:20.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-18T03:16:20.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-18T03:16:20.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-18T03:16:20.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-18T03:16:20.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-18T03:16:20.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-18T03:16:20.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-18T03:16:20.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-18T03:16:20.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-18T03:16:20.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-18T03:16:20.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-18T03:16:20.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-18T03:16:20.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-18T03:16:20.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-18T03:16:20.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-18T03:16:20.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-18T03:16:20.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-18T03:16:20.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-18T03:16:20.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-18T03:16:20.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-18T03:16:20.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-18T03:16:20.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-18T03:16:20.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-18T03:16:20.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-18T03:16:20.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-18T03:16:20.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-18T03:16:20.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-18T03:16:20.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-18T03:16:20.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-18T03:16:20.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-18T03:16:20.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-18T03:16:20.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-18T03:16:20.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-18T03:16:20.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_quota.ko 2023-07-18T03:16:20.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-18T03:16:20.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-18T03:16:20.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-18T03:16:20.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-18T03:16:20.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tunnel.ko 2023-07-18T03:16:20.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-18T03:16:20.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-18T03:16:20.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-18T03:16:20.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-18T03:16:20.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-18T03:16:20.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-18T03:16:20.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-18T03:16:20.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-18T03:16:20.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-18T03:16:20.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-18T03:16:20.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-18T03:16:20.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-18T03:16:20.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-18T03:16:20.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-18T03:16:20.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-18T03:16:20.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-18T03:16:20.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-18T03:16:20.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-18T03:16:20.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-18T03:16:20.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-18T03:16:20.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-18T03:16:20.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-18T03:16:20.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-18T03:16:20.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-18T03:16:20.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-18T03:16:20.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-18T03:16:20.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-18T03:16:20.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-18T03:16:20.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-18T03:16:20.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-18T03:16:20.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-18T03:16:20.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-18T03:16:20.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-18T03:16:20.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-18T03:16:20.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-18T03:16:20.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-18T03:16:20.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-18T03:16:20.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-18T03:16:20.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-18T03:16:20.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-18T03:16:20.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-18T03:16:20.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-18T03:16:20.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-18T03:16:20.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-18T03:16:20.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-18T03:16:20.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-18T03:16:20.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-18T03:16:20.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-18T03:16:20.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-18T03:16:20.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-18T03:16:20.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-18T03:16:20.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-18T03:16:20.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-18T03:16:20.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-18T03:16:20.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-18T03:16:20.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-18T03:16:20.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-18T03:16:20.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-18T03:16:20.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-18T03:16:20.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-18T03:16:20.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-18T03:16:20.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-18T03:16:20.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-18T03:16:20.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-18T03:16:20.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-18T03:16:20.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-18T03:16:20.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-18T03:16:20.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-18T03:16:20.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-18T03:16:20.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-18T03:16:20.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-18T03:16:20.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink 2023-07-18T03:16:20.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-18T03:16:20.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh 2023-07-18T03:16:20.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-18T03:16:20.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch 2023-07-18T03:16:20.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-18T03:16:20.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-18T03:16:20.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-18T03:16:20.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-18T03:16:20.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet 2023-07-18T03:16:20.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-18T03:16:20.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds 2023-07-18T03:16:20.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-18T03:16:20.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-18T03:16:20.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-18T03:16:20.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched 2023-07-18T03:16:20.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-18T03:16:20.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-18T03:16:20.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-18T03:16:20.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-18T03:16:20.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-18T03:16:20.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-18T03:16:20.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-18T03:16:20.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-18T03:16:20.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-18T03:16:20.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-18T03:16:20.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-18T03:16:20.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-18T03:16:20.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-18T03:16:20.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-18T03:16:20.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-18T03:16:20.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-18T03:16:20.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-18T03:16:20.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-18T03:16:20.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-18T03:16:20.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-18T03:16:20.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-18T03:16:20.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-18T03:16:20.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-18T03:16:20.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-18T03:16:20.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-18T03:16:20.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-18T03:16:20.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-18T03:16:20.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-18T03:16:20.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-18T03:16:20.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-18T03:16:20.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-18T03:16:20.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-18T03:16:20.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-18T03:16:20.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-18T03:16:20.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-18T03:16:20.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-18T03:16:20.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-18T03:16:20.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-18T03:16:20.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-18T03:16:20.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-18T03:16:20.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-18T03:16:20.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-18T03:16:20.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-18T03:16:20.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-18T03:16:20.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp 2023-07-18T03:16:20.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-18T03:16:20.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-18T03:16:20.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc 2023-07-18T03:16:20.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss 2023-07-18T03:16:20.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-18T03:16:20.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-18T03:16:20.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-18T03:16:20.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma 2023-07-18T03:16:20.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-18T03:16:20.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc 2023-07-18T03:16:20.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-18T03:16:20.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-18T03:16:20.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix 2023-07-18T03:16:20.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-18T03:16:20.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock 2023-07-18T03:16:20.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-18T03:16:20.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-18T03:16:20.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-18T03:16:20.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-18T03:16:20.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-18T03:16:20.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm 2023-07-18T03:16:20.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-18T03:16:20.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security 2023-07-18T03:16:20.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys 2023-07-18T03:16:20.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys 2023-07-18T03:16:20.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-18T03:16:20.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys 2023-07-18T03:16:20.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys/trusted.ko 2023-07-18T03:16:20.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt 2023-07-18T03:16:20.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib 2023-07-18T03:16:20.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-18T03:16:20.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-18T03:16:20.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-18T03:16:20.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-18T03:16:20.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-18T03:16:20.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-18T03:16:20.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-18T03:16:20.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-18T03:16:20.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-18T03:16:20.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-18T03:16:20.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-18T03:16:20.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-18T03:16:20.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-18T03:16:20.596 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:20.596 DEBUG:teuthology.orchestra.run.smithi107:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-18T03:16:20.624 INFO:teuthology.orchestra.run.smithi107.stdout:postinstall scriptlet (using /bin/sh): 2023-07-18T03:16:20.624 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 -a -r /boot/System.map-6.5.0-rc1-ga440c79eee02 ]; then 2023-07-18T03:16:20.624 INFO:teuthology.orchestra.run.smithi107.stdout:cp /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm 2023-07-18T03:16:20.624 INFO:teuthology.orchestra.run.smithi107.stdout:cp /boot/System.map-6.5.0-rc1-ga440c79eee02 /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-18T03:16:20.624 INFO:teuthology.orchestra.run.smithi107.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:20.624 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/installkernel 6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-18T03:16:20.625 INFO:teuthology.orchestra.run.smithi107.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-18T03:16:20.625 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2023-07-18T03:16:20.625 INFO:teuthology.orchestra.run.smithi107.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-18T03:16:20.625 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-18T03:16:20.625 INFO:teuthology.orchestra.run.smithi107.stdout:new-kernel-pkg --remove 6.5.0-rc1-ga440c79eee02 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-ga440c79eee02.img 2023-07-18T03:16:20.625 INFO:teuthology.orchestra.run.smithi107.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-18T03:16:20.625 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-install remove 6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:20.626 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2023-07-18T03:16:20.626 INFO:teuthology.orchestra.run.smithi107.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-18T03:16:20.626 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-18T03:16:20.626 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:20.626 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2023-07-18T03:16:20.627 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2023-07-18T03:16:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2023-07-18T03:16:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2023-07-18T03:16:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.02 2023-07-18T03:16:20.706 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 123.el8 2023-07-18T03:16:20.707 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2023-07-18T03:16:20.707 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-18T03:16:20.707 INFO:teuthology.orchestra.run.smithi107.stdout:Group : System Environment/Base 2023-07-18T03:16:20.707 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 9265450 2023-07-18T03:16:20.707 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2023-07-18T03:16:20.707 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-18T03:16:20.707 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-18T03:16:20.707 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-18T03:16:20.708 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-18T03:16:20.708 INFO:teuthology.orchestra.run.smithi107.stdout:Relocations : (not relocatable) 2023-07-18T03:16:20.708 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : Red Hat, Inc. 2023-07-18T03:16:20.708 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : Red Hat, Inc. 2023-07-18T03:16:20.708 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-18T03:16:20.708 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2023-07-18T03:16:20.708 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2023-07-18T03:16:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:16:20.709 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-18T03:16:20.709 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-18T03:16:20.709 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-18T03:16:20.709 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2023-07-18T03:16:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:16:20.709 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2023-07-18T03:16:20.710 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-18T03:16:20.710 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:20.711 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-18T03:16:22.914 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2023-07-18T03:16:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:done 2023-07-18T03:16:25.097 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2023-07-18T03:16:25.113 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.uM4otHej6Q 2023-07-18T03:16:25.114 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uM4otHej6Q 2023-07-18T03:16:25.186 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.uM4otHej6Q 2023-07-18T03:16:25.328 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.uM4otHej6Q is 6KB 2023-07-18T03:16:25.348 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.uM4otHej6Q 2023-07-18T03:16:25.363 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2023-07-18T03:16:25.427 INFO:teuthology.orchestra.run.smithi107.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-18T03:16:25.427 INFO:teuthology.orchestra.run.smithi107.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-18T03:16:25.428 INFO:teuthology.orchestra.run.smithi107.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02.conf 2023-07-18T03:16:25.429 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02 2023-07-18T03:16:25.582 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2023-07-18T03:16:40.133 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-07-18T03:16:40.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-07-18T03:16:40.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:16:55.600 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-07-18T03:16:55.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-18T03:16:55.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:17:14.736 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2023-07-18T03:17:23.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-07-18T03:17:23.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:17:26.800 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2023-07-18T03:17:30.096 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2023-07-18T03:17:38.808 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-07-18T03:17:38.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:17:39.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-18T03:17:39.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:17:41.876 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2023-07-18T03:17:56.888 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-07-18T03:17:56.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:17:57.488 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2023-07-18T03:18:09.491 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-18T03:18:09.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:18:12.701 DEBUG:teuthology.orchestra.run.smithi107:> true 2023-07-18T03:18:13.128 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-18T03:18:13.129 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.5.0-rc1-ga440c79eee02"... 2023-07-18T03:18:13.129 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2023-07-18T03:18:13.173 INFO:teuthology.orchestra.run.smithi107.stdout:6.5.0-rc1-ga440c79eee02 2023-07-18T03:18:13.173 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-18T03:18:13.174 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-18T03:18:13.174 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-18T03:18:14.175 DEBUG:teuthology.parallel:result is None 2023-07-18T03:18:28.829 DEBUG:teuthology.orchestra.run.smithi049:> true 2023-07-18T03:18:29.260 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-07-18T03:18:29.260 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc1-ga440c79eee02"... 2023-07-18T03:18:29.261 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2023-07-18T03:18:29.303 INFO:teuthology.orchestra.run.smithi049.stdout:6.5.0-rc1-ga440c79eee02 2023-07-18T03:18:29.304 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-18T03:18:29.304 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-18T03:18:29.304 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-18T03:18:30.305 DEBUG:teuthology.parallel:result is None 2023-07-18T03:18:30.306 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-18T03:18:30.315 INFO:teuthology.task.internal:Creating test directory... 2023-07-18T03:18:30.316 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-18T03:18:30.321 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-18T03:18:30.373 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-18T03:18:30.380 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-18T03:18:30.391 INFO:teuthology.task.internal:Creating archive directory... 2023-07-18T03:18:30.391 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-18T03:18:30.405 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-18T03:18:30.461 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-18T03:18:30.471 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-18T03:18:30.472 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-18T03:18:30.476 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-18T03:18:30.631 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T03:18:30.669 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T03:18:30.699 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T03:18:30.734 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T03:18:30.737 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-18T03:18:30.749 INFO:teuthology.task.internal:Configuring sudo... 2023-07-18T03:18:30.750 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-18T03:18:30.754 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-18T03:18:30.823 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-18T03:18:30.834 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-18T03:18:30.835 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-18T03:18:30.839 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-18T03:18:30.880 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T03:18:30.997 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T03:18:31.079 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T03:18:31.156 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T03:18:31.228 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:18:31.229 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-18T03:18:31.302 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T03:18:31.378 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T03:18:31.456 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T03:18:31.531 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T03:18:31.598 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:18:31.598 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-18T03:18:31.665 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2023-07-18T03:18:31.670 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2023-07-18T03:18:31.796 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:18:31.839 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:18:32.295 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-18T03:18:32.304 INFO:teuthology.task.internal:Starting timer... 2023-07-18T03:18:32.304 INFO:teuthology.run_tasks:Running task pcp... 2023-07-18T03:18:32.319 INFO:teuthology.run_tasks:Running task selinux... 2023-07-18T03:18:32.337 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-18T03:18:32.338 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2023-07-18T03:18:32.461 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2023-07-18T03:18:32.465 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2023-07-18T03:18:32.584 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2023-07-18T03:18:32.587 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-18T03:18:32.588 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2023-07-18T03:18:32.639 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2023-07-18T03:18:32.639 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2023-07-18T03:18:32.693 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2023-07-18T03:18:32.693 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi107.front.sepia.ceph.com': 'permissive'} 2023-07-18T03:18:32.694 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:18:32.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:18:32.766 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:18:32.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:18:32.835 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-18T03:18:32.835 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2023-07-18T03:18:32.918 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2023-07-18T03:18:32.990 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-18T03:18:33.001 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-18T03:18:33.007 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-18T03:18:33.139 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-18T03:18:33.162 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-18T03:18:33.163 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi107.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-18T03:30:31.698 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com')] 2023-07-18T03:30:31.700 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-07-18T03:30:31.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:30:31.792 DEBUG:teuthology.orchestra.run.smithi049:> true 2023-07-18T03:30:31.904 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-07-18T03:30:31.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-18T03:30:31.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:30:31.993 DEBUG:teuthology.orchestra.run.smithi107:> true 2023-07-18T03:30:32.102 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-18T03:30:32.103 INFO:teuthology.run_tasks:Running task clock... 2023-07-18T03:30:32.114 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-18T03:30:32.114 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-18T03:30:32.115 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:30:32.120 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-18T03:30:32.120 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:30:32.194 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-18T03:30:32.214 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-18T03:30:32.231 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-18T03:30:32.248 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-18T03:30:32.331 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2023-07-18T03:30:32.335 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2023-07-18T03:30:32.362 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2023-07-18T03:30:32.364 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2023-07-18T03:30:32.398 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-18T03:30:32.399 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-18T03:30:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-18T03:30:32.435 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-18T03:30:32.527 INFO:teuthology.orchestra.run.smithi107.stderr:bash: ntpq: command not found 2023-07-18T03:30:32.535 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:30:32.535 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2023-07-18T03:30:32.535 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:30:32.535 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:30:32.535 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:30:32.536 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:30:32.542 INFO:teuthology.orchestra.run.smithi049.stderr:bash: ntpq: command not found 2023-07-18T03:30:32.550 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:30:32.550 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2023-07-18T03:30:32.550 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:30:32.550 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:30:32.550 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:30:32.550 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:30:32.551 INFO:teuthology.run_tasks:Running task install... 2023-07-18T03:30:32.559 DEBUG:teuthology.task.install:project ceph 2023-07-18T03:30:32.560 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'}} 2023-07-18T03:30:32.560 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'} 2023-07-18T03:30:32.560 INFO:teuthology.task.install:Using flavor: default 2023-07-18T03:30:32.566 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-18T03:30:32.566 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-18T03:30:32.566 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-18T03:30:32.567 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:30:32.569 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-18T03:30:32.569 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:30:32.787 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-18T03:30:32.787 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-18T03:30:32.798 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-18T03:30:32.798 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-18T03:30:32.960 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-18T03:30:32.961 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:30:32.961 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-18T03:30:32.970 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-18T03:30:32.971 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:30:32.971 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-18T03:30:33.030 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-18T03:30:33.030 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-18T03:30:33.042 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-18T03:30:33.042 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-18T03:30:33.139 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-18T03:30:33.153 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-18T03:30:33.233 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-18T03:30:33.252 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-18T03:30:33.300 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2023-07-18T03:30:33.303 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2023-07-18T03:30:33.322 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2023-07-18T03:30:33.325 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2023-07-18T03:30:33.746 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-18T03:30:33.818 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2023-07-18T03:30:34.711 INFO:teuthology.orchestra.run.smithi107.stdout:44 files removed 2023-07-18T03:30:34.766 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-18T03:30:34.820 INFO:teuthology.orchestra.run.smithi049.stdout:44 files removed 2023-07-18T03:30:34.882 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-18T03:30:35.215 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-18T03:30:35.365 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2023-07-18T03:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 191 kB/s | 81 kB 00:00 2023-07-18T03:30:36.749 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 198 kB/s | 81 kB 00:00 2023-07-18T03:30:37.142 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 26 kB/s | 9.4 kB 00:00 2023-07-18T03:30:37.534 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 4.0 kB/s | 1.5 kB 00:00 2023-07-18T03:30:37.693 INFO:teuthology.orchestra.run.smithi107.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-07-18T03:30:39.233 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 3.7 kB/s | 9.4 kB 00:02 2023-07-18T03:30:39.634 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 4.0 kB/s | 1.5 kB 00:00 2023-07-18T03:30:39.801 INFO:teuthology.orchestra.run.smithi049.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-07-18T03:30:42.060 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 16 MB 00:02 2023-07-18T03:30:46.318 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 510 kB/s | 24 kB 00:00 2023-07-18T03:30:47.512 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-07-18T03:30:56.837 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 47 MB 00:01 2023-07-18T03:31:09.123 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T03:31:09.124 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T03:31:09.140 INFO:teuthology.orchestra.run.smithi049.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-18T03:31:09.141 INFO:teuthology.orchestra.run.smithi049.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-18T03:31:09.142 INFO:teuthology.orchestra.run.smithi049.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-18T03:31:09.143 INFO:teuthology.orchestra.run.smithi049.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-18T03:31:09.146 INFO:teuthology.orchestra.run.smithi049.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-18T03:31:09.149 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-18T03:31:09.151 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-18T03:31:09.298 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2023-07-18T03:31:09.312 INFO:teuthology.orchestra.run.smithi049.stdout:=========================================================================================================================== 2023-07-18T03:31:09.312 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2023-07-18T03:31:09.313 INFO:teuthology.orchestra.run.smithi049.stdout:=========================================================================================================================== 2023-07-18T03:31:09.313 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2023-07-18T03:31:09.313 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-18T03:31:09.313 INFO:teuthology.orchestra.run.smithi049.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-18T03:31:09.313 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-18T03:31:09.313 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-18T03:31:09.313 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-18T03:31:09.314 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-18T03:31:09.314 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-18T03:31:09.314 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-18T03:31:09.314 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-18T03:31:09.314 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-18T03:31:09.314 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-18T03:31:09.314 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-18T03:31:09.314 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-18T03:31:09.315 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-18T03:31:09.315 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-18T03:31:09.315 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-18T03:31:09.315 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-18T03:31:09.315 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-18T03:31:09.315 INFO:teuthology.orchestra.run.smithi049.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-18T03:31:09.315 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-18T03:31:09.316 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-18T03:31:09.316 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-18T03:31:09.316 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-18T03:31:09.316 INFO:teuthology.orchestra.run.smithi049.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-18T03:31:09.316 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-18T03:31:09.316 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-18T03:31:09.316 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-18T03:31:09.317 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-18T03:31:09.317 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-18T03:31:09.317 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-18T03:31:09.317 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-18T03:31:09.317 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-18T03:31:09.317 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-18T03:31:09.317 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-18T03:31:09.318 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-18T03:31:09.318 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-18T03:31:09.318 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-18T03:31:09.318 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2023-07-18T03:31:09.318 INFO:teuthology.orchestra.run.smithi049.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-18T03:31:09.318 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-18T03:31:09.319 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-18T03:31:09.319 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2023-07-18T03:31:09.319 INFO:teuthology.orchestra.run.smithi049.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-18T03:31:09.319 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-18T03:31:09.319 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-18T03:31:09.319 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-18T03:31:09.320 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-18T03:31:09.320 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-18T03:31:09.320 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-18T03:31:09.320 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-18T03:31:09.321 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-18T03:31:09.321 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-18T03:31:09.321 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-18T03:31:09.321 INFO:teuthology.orchestra.run.smithi049.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-18T03:31:09.322 INFO:teuthology.orchestra.run.smithi049.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-18T03:31:09.322 INFO:teuthology.orchestra.run.smithi049.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-18T03:31:09.322 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-18T03:31:09.322 INFO:teuthology.orchestra.run.smithi049.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-18T03:31:09.323 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-18T03:31:09.323 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-18T03:31:09.323 INFO:teuthology.orchestra.run.smithi049.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-18T03:31:09.324 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-18T03:31:09.324 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-18T03:31:09.324 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-18T03:31:09.325 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-18T03:31:09.325 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-18T03:31:09.325 INFO:teuthology.orchestra.run.smithi049.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-18T03:31:09.326 INFO:teuthology.orchestra.run.smithi049.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-18T03:31:09.326 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-18T03:31:09.326 INFO:teuthology.orchestra.run.smithi049.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-18T03:31:09.326 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-18T03:31:09.327 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-18T03:31:09.327 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-18T03:31:09.327 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-18T03:31:09.327 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-18T03:31:09.328 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-18T03:31:09.328 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-18T03:31:09.328 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-18T03:31:09.328 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-18T03:31:09.328 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-18T03:31:09.329 INFO:teuthology.orchestra.run.smithi049.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-18T03:31:09.329 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-18T03:31:09.329 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-18T03:31:09.330 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-18T03:31:09.330 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-18T03:31:09.330 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-18T03:31:09.330 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-18T03:31:09.330 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-18T03:31:09.331 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-18T03:31:09.331 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-18T03:31:09.331 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-18T03:31:09.331 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-18T03:31:09.331 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-18T03:31:09.331 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-18T03:31:09.332 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-18T03:31:09.332 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T03:31:09.332 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-18T03:31:09.332 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-18T03:31:09.332 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-18T03:31:09.332 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-18T03:31:09.333 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-18T03:31:09.333 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-18T03:31:09.333 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-18T03:31:09.333 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-18T03:31:09.333 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-18T03:31:09.333 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-18T03:31:09.333 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-18T03:31:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-18T03:31:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-18T03:31:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-18T03:31:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-18T03:31:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-18T03:31:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-18T03:31:09.334 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-18T03:31:09.335 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T03:31:09.335 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-18T03:31:09.335 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-18T03:31:09.335 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-18T03:31:09.335 INFO:teuthology.orchestra.run.smithi049.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-18T03:31:09.335 INFO:teuthology.orchestra.run.smithi049.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-18T03:31:09.335 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-18T03:31:09.336 INFO:teuthology.orchestra.run.smithi049.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-18T03:31:09.336 INFO:teuthology.orchestra.run.smithi049.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-18T03:31:09.336 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-18T03:31:09.336 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-18T03:31:09.336 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-18T03:31:09.336 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-18T03:31:09.336 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-18T03:31:09.336 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-18T03:31:09.337 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-18T03:31:09.337 INFO:teuthology.orchestra.run.smithi049.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-18T03:31:09.337 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-18T03:31:09.337 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-18T03:31:09.337 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling module streams: 2023-07-18T03:31:09.337 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql 10 2023-07-18T03:31:09.338 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:31:09.338 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2023-07-18T03:31:09.338 INFO:teuthology.orchestra.run.smithi049.stdout:=========================================================================================================================== 2023-07-18T03:31:09.338 INFO:teuthology.orchestra.run.smithi049.stdout:Install 128 Packages 2023-07-18T03:31:09.339 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 3 Packages 2023-07-18T03:31:09.339 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:31:09.340 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 205 M 2023-07-18T03:31:09.340 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2023-07-18T03:31:09.562 INFO:teuthology.orchestra.run.smithi049.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 32 kB/s | 6.4 kB 00:00 2023-07-18T03:31:10.029 INFO:teuthology.orchestra.run.smithi049.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 1.8 MB/s | 848 kB 00:00 2023-07-18T03:31:10.162 INFO:teuthology.orchestra.run.smithi049.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 1.2 MB/s | 165 kB 00:00 2023-07-18T03:31:10.446 INFO:teuthology.orchestra.run.smithi049.stdout:(4/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 5.0 MB/s | 5.4 MB 00:01 2023-07-18T03:31:10.655 INFO:teuthology.orchestra.run.smithi049.stdout:(5/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 7.4 MB/s | 1.5 MB 00:00 2023-07-18T03:31:10.897 INFO:teuthology.orchestra.run.smithi049.stdout:(6/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 3.1 MB/s | 2.3 MB 00:00 2023-07-18T03:31:11.347 INFO:teuthology.orchestra.run.smithi049.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 6.7 MB/s | 4.6 MB 00:00 2023-07-18T03:31:11.889 INFO:teuthology.orchestra.run.smithi049.stdout:(8/131): ceph-common-18.0.0-4746.g81cf785a.el8. 8.7 MB/s | 22 MB 00:02 2023-07-18T03:31:11.956 INFO:teuthology.orchestra.run.smithi049.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 367 kB/s | 24 kB 00:00 2023-07-18T03:31:12.848 INFO:teuthology.orchestra.run.smithi049.stdout:(10/131): ceph-radosgw-18.0.0-4746.g81cf785a.el 9.0 MB/s | 13 MB 00:01 2023-07-18T03:31:12.942 INFO:teuthology.orchestra.run.smithi049.stdout:(11/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 2.7 MB/s | 258 kB 00:00 2023-07-18T03:31:13.059 INFO:teuthology.orchestra.run.smithi049.stdout:(12/131): libcephfs-devel-18.0.0-4746.g81cf785a 266 kB/s | 31 kB 00:00 2023-07-18T03:31:13.201 INFO:teuthology.orchestra.run.smithi049.stdout:(13/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 5.3 MB/s | 772 kB 00:00 2023-07-18T03:31:13.461 INFO:teuthology.orchestra.run.smithi049.stdout:(14/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x8 6.9 MB/s | 18 MB 00:02 2023-07-18T03:31:13.485 INFO:teuthology.orchestra.run.smithi049.stdout:(15/131): libcephsqlite-18.0.0-4746.g81cf785a.e 484 kB/s | 137 kB 00:00 2023-07-18T03:31:13.527 INFO:teuthology.orchestra.run.smithi049.stdout:(16/131): librados-devel-18.0.0-4746.g81cf785a. 2.1 MB/s | 139 kB 00:00 2023-07-18T03:31:13.594 INFO:teuthology.orchestra.run.smithi049.stdout:(17/131): libradosstriper1-18.0.0-4746.g81cf785 4.2 MB/s | 473 kB 00:00 2023-07-18T03:31:13.686 INFO:teuthology.orchestra.run.smithi049.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 490 kB/s | 45 kB 00:00 2023-07-18T03:31:13.778 INFO:teuthology.orchestra.run.smithi049.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 1.3 MB/s | 119 kB 00:00 2023-07-18T03:31:13.895 INFO:teuthology.orchestra.run.smithi049.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 1.7 MB/s | 197 kB 00:00 2023-07-18T03:31:14.020 INFO:teuthology.orchestra.run.smithi049.stdout:(21/131): python3-rados-18.0.0-4746.g81cf785a.e 2.9 MB/s | 370 kB 00:00 2023-07-18T03:31:14.154 INFO:teuthology.orchestra.run.smithi049.stdout:(22/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 11 MB/s | 6.7 MB 00:00 2023-07-18T03:31:14.187 INFO:teuthology.orchestra.run.smithi049.stdout:(23/131): python3-rbd-18.0.0-4746.g81cf785a.el8 2.1 MB/s | 364 kB 00:00 2023-07-18T03:31:14.229 INFO:teuthology.orchestra.run.smithi049.stdout:(24/131): python3-rgw-18.0.0-4746.g81cf785a.el8 1.4 MB/s | 107 kB 00:00 2023-07-18T03:31:14.288 INFO:teuthology.orchestra.run.smithi049.stdout:(25/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 941 kB/s | 94 kB 00:00 2023-07-18T03:31:14.396 INFO:teuthology.orchestra.run.smithi049.stdout:(26/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 1.6 MB/s | 179 kB 00:00 2023-07-18T03:31:14.530 INFO:teuthology.orchestra.run.smithi049.stdout:(27/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 11 MB/s | 3.4 MB 00:00 2023-07-18T03:31:14.555 INFO:teuthology.orchestra.run.smithi049.stdout:(28/131): ceph-grafana-dashboards-18.0.0-4746.g 145 kB/s | 23 kB 00:00 2023-07-18T03:31:14.606 INFO:teuthology.orchestra.run.smithi049.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 1.7 MB/s | 129 kB 00:00 2023-07-18T03:31:14.865 INFO:teuthology.orchestra.run.smithi049.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 5.5 MB/s | 1.7 MB 00:00 2023-07-18T03:31:14.973 INFO:teuthology.orchestra.run.smithi049.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 2.2 MB/s | 240 kB 00:00 2023-07-18T03:31:15.090 INFO:teuthology.orchestra.run.smithi049.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 407 kB/s | 47 kB 00:00 2023-07-18T03:31:15.157 INFO:teuthology.orchestra.run.smithi049.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 218 kB/s | 15 kB 00:00 2023-07-18T03:31:15.249 INFO:teuthology.orchestra.run.smithi049.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-07-18T03:31:15.274 INFO:teuthology.orchestra.run.smithi049.stdout:(35/131): ceph-volume-18.0.0-4746.g81cf785a.el8 2.2 MB/s | 258 kB 00:00 2023-07-18T03:31:15.333 INFO:teuthology.orchestra.run.smithi049.stdout:(36/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 1.9 MB/s | 166 kB 00:00 2023-07-18T03:31:15.367 INFO:teuthology.orchestra.run.smithi049.stdout:(37/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 194 kB/s | 18 kB 00:00 2023-07-18T03:31:16.076 INFO:teuthology.orchestra.run.smithi049.stdout:(38/131): ceph-test-18.0.0-4746.g81cf785a.el8.x 12 MB/s | 48 MB 00:04 2023-07-18T03:31:16.101 INFO:teuthology.orchestra.run.smithi049.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 80 kB/s | 59 kB 00:00 2023-07-18T03:31:16.479 INFO:teuthology.orchestra.run.smithi049.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 267 kB/s | 306 kB 00:01 2023-07-18T03:31:16.502 INFO:teuthology.orchestra.run.smithi049.stdout:(41/131): libunwind-1.3.1-3.el8.x86_64.rpm 176 kB/s | 75 kB 00:00 2023-07-18T03:31:16.552 INFO:teuthology.orchestra.run.smithi049.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 600 kB/s | 44 kB 00:00 2023-07-18T03:31:16.635 INFO:teuthology.orchestra.run.smithi049.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 891 kB/s | 476 kB 00:00 2023-07-18T03:31:16.652 INFO:teuthology.orchestra.run.smithi049.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 329 kB/s | 33 kB 00:00 2023-07-18T03:31:16.686 INFO:teuthology.orchestra.run.smithi049.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.0 MB/s | 185 kB 00:00 2023-07-18T03:31:16.703 INFO:teuthology.orchestra.run.smithi049.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 243 kB/s | 16 kB 00:00 2023-07-18T03:31:16.770 INFO:teuthology.orchestra.run.smithi049.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2023-07-18T03:31:16.787 INFO:teuthology.orchestra.run.smithi049.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 972 kB/s | 81 kB 00:00 2023-07-18T03:31:16.845 INFO:teuthology.orchestra.run.smithi049.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 148 kB/s | 11 kB 00:00 2023-07-18T03:31:16.896 INFO:teuthology.orchestra.run.smithi049.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-07-18T03:31:16.979 INFO:teuthology.orchestra.run.smithi049.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.3 MB/s | 384 kB 00:00 2023-07-18T03:31:17.005 INFO:teuthology.orchestra.run.smithi049.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 450 kB/s | 49 kB 00:00 2023-07-18T03:31:17.055 INFO:teuthology.orchestra.run.smithi049.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 782 kB/s | 59 kB 00:00 2023-07-18T03:31:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 803 kB/s | 60 kB 00:00 2023-07-18T03:31:17.156 INFO:teuthology.orchestra.run.smithi049.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 208 kB/s | 16 kB 00:00 2023-07-18T03:31:17.231 INFO:teuthology.orchestra.run.smithi049.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 450 kB/s | 34 kB 00:00 2023-07-18T03:31:17.285 INFO:teuthology.orchestra.run.smithi049.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.2 MB/s | 283 kB 00:00 2023-07-18T03:31:17.365 INFO:teuthology.orchestra.run.smithi049.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 777 kB/s | 61 kB 00:00 2023-07-18T03:31:17.390 INFO:teuthology.orchestra.run.smithi049.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2023-07-18T03:31:17.440 INFO:teuthology.orchestra.run.smithi049.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2023-07-18T03:31:17.465 INFO:teuthology.orchestra.run.smithi049.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 318 kB/s | 24 kB 00:00 2023-07-18T03:31:17.516 INFO:teuthology.orchestra.run.smithi049.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2023-07-18T03:31:17.541 INFO:teuthology.orchestra.run.smithi049.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 366 kB/s | 27 kB 00:00 2023-07-18T03:31:17.717 INFO:teuthology.orchestra.run.smithi049.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.2 MB/s | 255 kB 00:00 2023-07-18T03:31:17.742 INFO:teuthology.orchestra.run.smithi049.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.2 MB/s | 251 kB 00:00 2023-07-18T03:31:17.792 INFO:teuthology.orchestra.run.smithi049.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 818 kB/s | 61 kB 00:00 2023-07-18T03:31:17.818 INFO:teuthology.orchestra.run.smithi049.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-07-18T03:31:17.868 INFO:teuthology.orchestra.run.smithi049.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 306 kB/s | 23 kB 00:00 2023-07-18T03:31:17.901 INFO:teuthology.orchestra.run.smithi049.stdout:(69/131): python3-kubernetes-11.0.0-6.el8.noarc 1.1 MB/s | 1.2 MB 00:01 2023-07-18T03:31:17.927 INFO:teuthology.orchestra.run.smithi049.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-07-18T03:31:17.977 INFO:teuthology.orchestra.run.smithi049.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 625 kB/s | 47 kB 00:00 2023-07-18T03:31:18.002 INFO:teuthology.orchestra.run.smithi049.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.9 MB/s | 200 kB 00:00 2023-07-18T03:31:18.028 INFO:teuthology.orchestra.run.smithi049.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 851 kB/s | 43 kB 00:00 2023-07-18T03:31:18.061 INFO:teuthology.orchestra.run.smithi049.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-07-18T03:31:18.112 INFO:teuthology.orchestra.run.smithi049.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 577 kB/s | 48 kB 00:00 2023-07-18T03:31:18.129 INFO:teuthology.orchestra.run.smithi049.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2023-07-18T03:31:18.154 INFO:teuthology.orchestra.run.smithi049.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-07-18T03:31:18.179 INFO:teuthology.orchestra.run.smithi049.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-18T03:31:18.213 INFO:teuthology.orchestra.run.smithi049.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-07-18T03:31:18.264 INFO:teuthology.orchestra.run.smithi049.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 6.6 MB/s | 560 kB 00:00 2023-07-18T03:31:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 249 kB/s | 19 kB 00:00 2023-07-18T03:31:18.314 INFO:teuthology.orchestra.run.smithi049.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-07-18T03:31:18.356 INFO:teuthology.orchestra.run.smithi049.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 8.2 MB/s | 559 kB 00:00 2023-07-18T03:31:18.373 INFO:teuthology.orchestra.run.smithi049.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-07-18T03:31:18.415 INFO:teuthology.orchestra.run.smithi049.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-18T03:31:18.448 INFO:teuthology.orchestra.run.smithi049.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-07-18T03:31:18.507 INFO:teuthology.orchestra.run.smithi049.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-07-18T03:31:18.549 INFO:teuthology.orchestra.run.smithi049.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-07-18T03:31:18.583 INFO:teuthology.orchestra.run.smithi049.stdout:(89/131): elfutils-libelf-devel-0.186-1.el8.x86 1.8 MB/s | 60 kB 00:00 2023-07-18T03:31:18.633 INFO:teuthology.orchestra.run.smithi049.stdout:(90/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 11 MB/s | 2.3 MB 00:00 2023-07-18T03:31:18.659 INFO:teuthology.orchestra.run.smithi049.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.3 MB/s | 98 kB 00:00 2023-07-18T03:31:18.684 INFO:teuthology.orchestra.run.smithi049.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-18T03:31:18.709 INFO:teuthology.orchestra.run.smithi049.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 769 kB/s | 39 kB 00:00 2023-07-18T03:31:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-07-18T03:31:18.776 INFO:teuthology.orchestra.run.smithi049.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-07-18T03:31:18.793 INFO:teuthology.orchestra.run.smithi049.stdout:(96/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-07-18T03:31:18.835 INFO:teuthology.orchestra.run.smithi049.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-18T03:31:18.869 INFO:teuthology.orchestra.run.smithi049.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-07-18T03:31:18.927 INFO:teuthology.orchestra.run.smithi049.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-07-18T03:31:18.970 INFO:teuthology.orchestra.run.smithi049.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-18T03:31:19.003 INFO:teuthology.orchestra.run.smithi049.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.1 MB/s | 39 kB 00:00 2023-07-18T03:31:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-07-18T03:31:19.120 INFO:teuthology.orchestra.run.smithi049.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-07-18T03:31:19.163 INFO:teuthology.orchestra.run.smithi049.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-18T03:31:19.196 INFO:teuthology.orchestra.run.smithi049.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2023-07-18T03:31:19.247 INFO:teuthology.orchestra.run.smithi049.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-07-18T03:31:19.347 INFO:teuthology.orchestra.run.smithi049.stdout:(107/131): thrift-0.13.0-2.el8.x86_64.rpm 1.1 MB/s | 1.7 MB 00:01 2023-07-18T03:31:19.514 INFO:teuthology.orchestra.run.smithi049.stdout:(108/131): python3-scipy-1.0.0-21.module+el8.5. 19 MB/s | 14 MB 00:00 2023-07-18T03:31:19.539 INFO:teuthology.orchestra.run.smithi049.stdout:(109/131): python3-jsonpatch-1.21-2.el8.noarch. 138 kB/s | 27 kB 00:00 2023-07-18T03:31:19.573 INFO:teuthology.orchestra.run.smithi049.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2023-07-18T03:31:19.598 INFO:teuthology.orchestra.run.smithi049.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 748 kB/s | 44 kB 00:00 2023-07-18T03:31:19.673 INFO:teuthology.orchestra.run.smithi049.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 6.0 MB/s | 457 kB 00:00 2023-07-18T03:31:19.765 INFO:teuthology.orchestra.run.smithi049.stdout:(113/131): python3-babel-2.5.1-7.el8.noarch.rpm 9.2 MB/s | 4.8 MB 00:00 2023-07-18T03:31:19.799 INFO:teuthology.orchestra.run.smithi049.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.8 MB/s | 225 kB 00:00 2023-07-18T03:31:19.824 INFO:teuthology.orchestra.run.smithi049.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-07-18T03:31:19.849 INFO:teuthology.orchestra.run.smithi049.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-07-18T03:31:19.867 INFO:teuthology.orchestra.run.smithi049.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 473 kB/s | 20 kB 00:00 2023-07-18T03:31:19.900 INFO:teuthology.orchestra.run.smithi049.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-18T03:31:19.942 INFO:teuthology.orchestra.run.smithi049.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-07-18T03:31:20.018 INFO:teuthology.orchestra.run.smithi049.stdout:(120/131): python3-numpy-1.14.3-10.el8.x86_64.r 8.4 MB/s | 3.7 MB 00:00 2023-07-18T03:31:20.043 INFO:teuthology.orchestra.run.smithi049.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.9 MB/s | 198 kB 00:00 2023-07-18T03:31:20.085 INFO:teuthology.orchestra.run.smithi049.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-07-18T03:31:20.143 INFO:teuthology.orchestra.run.smithi049.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.0 MB/s | 354 kB 00:00 2023-07-18T03:31:20.261 INFO:teuthology.orchestra.run.smithi049.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-07-18T03:31:20.352 INFO:teuthology.orchestra.run.smithi049.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2023-07-18T03:31:20.395 INFO:teuthology.orchestra.run.smithi049.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.9 MB/s | 808 kB 00:00 2023-07-18T03:31:20.461 INFO:teuthology.orchestra.run.smithi049.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-07-18T03:31:20.570 INFO:teuthology.orchestra.run.smithi049.stdout:(128/131): postgresql-server-10.21-2.module+el8 12 MB/s | 5.1 MB 00:00 2023-07-18T03:31:20.620 INFO:teuthology.orchestra.run.smithi049.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-18T03:31:20.962 INFO:teuthology.orchestra.run.smithi049.stdout:(130/131): librados2-18.0.0-4746.g81cf785a.el8. 6.5 MB/s | 3.7 MB 00:00 2023-07-18T03:31:21.021 INFO:teuthology.orchestra.run.smithi049.stdout:(131/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 6.4 MB/s | 3.6 MB 00:00 2023-07-18T03:31:21.025 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2023-07-18T03:31:21.025 INFO:teuthology.orchestra.run.smithi049.stdout:Total 18 MB/s | 205 MB 00:11 2023-07-18T03:31:22.720 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2023-07-18T03:31:22.867 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2023-07-18T03:31:22.868 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2023-07-18T03:31:24.697 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2023-07-18T03:31:24.703 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2023-07-18T03:31:30.058 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2023-07-18T03:31:30.247 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-18T03:31:30.476 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-18T03:31:30.631 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-18T03:31:30.819 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-18T03:31:31.031 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-18T03:31:31.534 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-18T03:31:31.856 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-18T03:31:32.341 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-18T03:31:32.556 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:31:32.699 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:31:32.931 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:31:33.041 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:31:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T03:31:33.445 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-18T03:31:33.742 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-18T03:31:33.962 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-18T03:31:34.461 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-18T03:31:34.717 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-18T03:31:34.939 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T03:31:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T03:31:35.272 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T03:31:35.470 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T03:31:35.721 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-18T03:31:35.899 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-18T03:31:36.121 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-18T03:31:36.263 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-18T03:31:36.445 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-18T03:31:36.622 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-18T03:31:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-18T03:31:37.546 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-18T03:31:38.277 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-18T03:31:38.471 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-18T03:31:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-18T03:31:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-18T03:31:39.937 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-18T03:31:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-18T03:31:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-18T03:31:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-18T03:31:40.872 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-18T03:31:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-18T03:31:43.306 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-18T03:31:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-18T03:31:44.959 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-18T03:31:45.274 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-18T03:31:45.562 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-18T03:31:45.777 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-18T03:31:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-18T03:31:46.229 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-18T03:31:46.445 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-18T03:31:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-18T03:31:46.815 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-18T03:31:46.995 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-18T03:31:47.090 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-18T03:31:47.380 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-18T03:31:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-18T03:31:47.811 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-18T03:31:47.910 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-18T03:31:48.176 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-18T03:31:48.367 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-18T03:31:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-18T03:31:48.750 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-18T03:31:48.965 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-18T03:31:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-18T03:31:49.371 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-18T03:31:49.596 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-18T03:31:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-18T03:31:50.327 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-18T03:31:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-18T03:31:50.880 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-18T03:31:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-18T03:31:51.488 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-18T03:31:51.763 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-18T03:31:51.952 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-18T03:31:52.266 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-18T03:31:52.467 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-18T03:31:52.685 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-18T03:31:53.031 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-18T03:31:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-18T03:31:53.604 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-18T03:31:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-18T03:31:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-18T03:31:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-18T03:31:54.583 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-18T03:31:54.777 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-18T03:31:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-18T03:31:55.141 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-18T03:31:55.345 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-18T03:31:55.570 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-18T03:31:55.802 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-18T03:31:55.973 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-18T03:31:56.127 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-18T03:31:56.433 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-18T03:31:56.693 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-18T03:31:56.902 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-18T03:31:57.234 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-18T03:31:57.479 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-18T03:31:57.656 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-18T03:31:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-18T03:31:59.352 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-18T03:31:59.784 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-18T03:32:00.069 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-18T03:32:00.396 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-18T03:32:00.718 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-18T03:32:00.936 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-18T03:32:01.875 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-18T03:32:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T03:32:02.157 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T03:32:02.777 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-18T03:32:07.973 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:32:08.251 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:32:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:32:09.247 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T03:32:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T03:32:09.541 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T03:32:30.709 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T03:32:30.710 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2023-07-18T03:32:30.710 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2023-07-18T03:32:30.710 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /dev 2023-07-18T03:32:30.710 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /run 2023-07-18T03:32:30.710 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2023-07-18T03:32:30.710 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2023-07-18T03:32:30.710 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2023-07-18T03:32:30.711 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2023-07-18T03:32:30.711 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /dev 2023-07-18T03:32:30.711 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:32:31.138 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T03:32:32.997 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T03:32:33.202 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T03:32:33.487 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T03:32:33.688 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T03:32:34.253 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T03:32:34.513 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T03:32:34.557 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T03:32:37.481 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T03:32:37.696 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T03:32:37.845 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-18T03:32:38.093 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-18T03:32:38.303 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T03:32:38.579 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T03:32:38.841 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T03:32:40.033 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T03:32:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T03:32:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T03:32:40.786 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-18T03:32:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T03:32:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T03:32:41.294 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T03:32:41.323 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T03:32:42.811 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-18T03:32:43.072 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T03:32:43.143 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T03:32:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T03:32:43.427 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T03:32:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T03:32:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T03:32:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T03:32:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T03:32:50.983 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-18T03:32:51.311 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-18T03:32:51.552 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 117/134 2023-07-18T03:32:51.822 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-18T03:32:51.992 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-18T03:32:52.262 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-18T03:32:52.482 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-18T03:32:52.690 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-18T03:32:52.835 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-18T03:32:56.543 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:32:56.901 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:32:56.939 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:32:57.131 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-18T03:32:57.336 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-18T03:32:57.698 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-18T03:32:58.074 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-18T03:32:58.241 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-18T03:32:58.362 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-18T03:32:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-18T03:32:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-18T03:32:58.808 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-18T03:32:58.831 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-18T03:32:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-18T03:32:58.987 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:32:59.215 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:32:59.215 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T03:32:59.458 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T03:32:59.684 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:32:59.957 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:32:59.957 INFO:teuthology.orchestra.run.smithi049.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-18T03:32:59.957 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:33:00.584 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-18T03:33:00.585 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-18T03:33:00.585 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-18T03:33:00.585 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-18T03:33:00.585 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-18T03:33:00.585 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:33:00.585 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:33:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T03:33:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-18T03:33:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-18T03:33:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-18T03:33:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-18T03:33:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-18T03:33:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-18T03:33:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-18T03:33:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-18T03:33:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-18T03:33:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-18T03:33:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-18T03:33:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-18T03:33:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-18T03:33:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-18T03:33:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-18T03:33:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-18T03:33:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-18T03:33:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-18T03:33:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-18T03:33:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-18T03:33:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-18T03:33:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-18T03:33:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-18T03:33:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-18T03:33:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-18T03:33:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-18T03:33:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-18T03:33:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-18T03:33:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-18T03:33:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-18T03:33:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-18T03:33:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-18T03:33:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-18T03:33:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-18T03:33:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-18T03:33:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-18T03:33:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-18T03:33:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-18T03:33:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-18T03:33:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-18T03:33:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-18T03:33:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-18T03:33:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-18T03:33:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-18T03:33:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-18T03:33:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-18T03:33:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-18T03:33:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-18T03:33:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-18T03:33:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-18T03:33:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-18T03:33:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-18T03:33:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-18T03:33:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-18T03:33:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-18T03:33:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-18T03:33:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-18T03:33:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-18T03:33:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-18T03:33:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-18T03:33:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-18T03:33:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-18T03:33:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-18T03:33:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-18T03:33:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-18T03:33:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-18T03:33:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-18T03:33:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-18T03:33:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-18T03:33:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-18T03:33:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-18T03:33:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-18T03:33:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-18T03:33:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-18T03:33:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-18T03:33:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-18T03:33:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-18T03:33:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-18T03:33:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-18T03:33:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-18T03:33:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-18T03:33:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-18T03:33:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-18T03:33:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-18T03:33:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-18T03:33:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-18T03:33:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-18T03:33:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-18T03:33:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-18T03:33:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-18T03:33:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-18T03:33:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-18T03:33:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-18T03:33:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-18T03:33:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-18T03:33:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-18T03:33:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-18T03:33:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-18T03:33:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-18T03:33:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-18T03:33:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-18T03:33:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-18T03:33:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-18T03:33:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-18T03:33:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-18T03:33:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-18T03:33:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-18T03:33:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-18T03:33:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-18T03:33:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-18T03:33:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-18T03:33:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-18T03:33:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-18T03:33:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-18T03:33:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-18T03:33:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-18T03:33:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-18T03:33:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-18T03:33:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-18T03:33:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-18T03:33:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-18T03:33:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-18T03:33:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-18T03:33:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:33:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-18T03:33:03.529 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:33:03.529 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:02:07 ago on Tue 18 Jul 2023 03:30:56 AM UTC. 2023-07-18T03:33:06.262 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2023-07-18T03:33:06.519 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:33:06.519 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2023-07-18T03:33:06.519 INFO:teuthology.orchestra.run.smithi049.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-18T03:33:06.519 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.519 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.520 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2023-07-18T03:33:06.520 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-18T03:33:06.520 INFO:teuthology.orchestra.run.smithi049.stdout: automake-1.16.1-7.el8.noarch 2023-07-18T03:33:06.520 INFO:teuthology.orchestra.run.smithi049.stdout: bison-3.0.4-10.el8.x86_64 2023-07-18T03:33:06.520 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.520 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.520 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.521 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.521 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:33:06.521 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.521 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.521 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.521 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:33:06.521 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:33:06.521 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:33:06.522 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:33:06.522 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:33:06.522 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.522 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.522 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:33:06.522 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.522 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.523 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.523 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:33:06.523 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:33:06.523 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.523 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:33:06.523 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-18T03:33:06.523 INFO:teuthology.orchestra.run.smithi049.stdout: flex-2.6.1-9.el8.x86_64 2023-07-18T03:33:06.523 INFO:teuthology.orchestra.run.smithi049.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-18T03:33:06.524 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-18T03:33:06.524 INFO:teuthology.orchestra.run.smithi049.stdout: jq-1.6-3.el8.x86_64 2023-07-18T03:33:06.524 INFO:teuthology.orchestra.run.smithi049.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-18T03:33:06.524 INFO:teuthology.orchestra.run.smithi049.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-18T03:33:06.524 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-18T03:33:06.524 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.524 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.525 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.525 INFO:teuthology.orchestra.run.smithi049.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-18T03:33:06.525 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-18T03:33:06.525 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-18T03:33:06.525 INFO:teuthology.orchestra.run.smithi049.stdout: libpq-13.5-1.el8.x86_64 2023-07-18T03:33:06.525 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-18T03:33:06.525 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-18T03:33:06.525 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.526 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.526 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-18T03:33:06.526 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.526 INFO:teuthology.orchestra.run.smithi049.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-18T03:33:06.526 INFO:teuthology.orchestra.run.smithi049.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-18T03:33:06.526 INFO:teuthology.orchestra.run.smithi049.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-18T03:33:06.526 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-18T03:33:06.526 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-18T03:33:06.527 INFO:teuthology.orchestra.run.smithi049.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-18T03:33:06.527 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-18T03:33:06.527 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-18T03:33:06.527 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-18T03:33:06.527 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-18T03:33:06.527 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-18T03:33:06.527 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-18T03:33:06.528 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-18T03:33:06.528 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-18T03:33:06.528 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:33:06.528 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:33:06.528 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:33:06.528 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-18T03:33:06.528 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-18T03:33:06.528 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-18T03:33:06.529 INFO:teuthology.orchestra.run.smithi049.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-18T03:33:06.529 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-18T03:33:06.529 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.529 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.529 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.529 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-18T03:33:06.529 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-18T03:33:06.530 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-18T03:33:06.530 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-18T03:33:06.530 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-18T03:33:06.530 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-18T03:33:06.530 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-18T03:33:06.530 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-18T03:33:06.530 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-18T03:33:06.530 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-18T03:33:06.531 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-18T03:33:06.531 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-18T03:33:06.531 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-18T03:33:06.531 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-18T03:33:06.531 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-18T03:33:06.531 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-18T03:33:06.531 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-18T03:33:06.531 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-18T03:33:06.532 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-18T03:33:06.532 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-18T03:33:06.532 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-18T03:33:06.532 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-18T03:33:06.532 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-2.6-1.el8.noarch 2023-07-18T03:33:06.532 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-18T03:33:06.532 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-18T03:33:06.532 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-18T03:33:06.533 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-18T03:33:06.533 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-18T03:33:06.533 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-18T03:33:06.533 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.533 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.533 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-18T03:33:06.533 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-18T03:33:06.534 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.534 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-18T03:33:06.534 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-18T03:33:06.534 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-18T03:33:06.534 INFO:teuthology.orchestra.run.smithi049.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-18T03:33:06.534 INFO:teuthology.orchestra.run.smithi049.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-18T03:33:06.534 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-18T03:33:06.534 INFO:teuthology.orchestra.run.smithi049.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-18T03:33:06.535 INFO:teuthology.orchestra.run.smithi049.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-18T03:33:06.535 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-18T03:33:06.535 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-18T03:33:06.535 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-18T03:33:06.535 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-18T03:33:06.535 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-18T03:33:06.535 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.535 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.536 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:33:06.536 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-18T03:33:06.536 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-18T03:33:06.536 INFO:teuthology.orchestra.run.smithi049.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-18T03:33:06.536 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-18T03:33:06.536 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-18T03:33:06.537 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-18T03:33:06.537 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:33:06.537 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2023-07-18T03:33:06.821 DEBUG:teuthology.parallel:result is None 2023-07-18T03:35:12.389 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 58 kB/s | 16 MB 04:34 2023-07-18T03:35:16.406 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 414 kB/s | 24 kB 00:00 2023-07-18T03:35:17.507 INFO:teuthology.orchestra.run.smithi107.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 52 MB/s | 53 MB 00:01 2023-07-18T03:35:26.198 INFO:teuthology.orchestra.run.smithi107.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 47 MB 00:01 2023-07-18T03:35:37.783 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T03:35:37.784 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T03:35:37.799 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-18T03:35:37.800 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-18T03:35:37.801 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-18T03:35:37.802 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-18T03:35:37.805 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-18T03:35:37.808 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-18T03:35:37.809 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-18T03:35:37.945 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2023-07-18T03:35:37.958 INFO:teuthology.orchestra.run.smithi107.stdout:=========================================================================================================================== 2023-07-18T03:35:37.958 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2023-07-18T03:35:37.958 INFO:teuthology.orchestra.run.smithi107.stdout:=========================================================================================================================== 2023-07-18T03:35:37.958 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2023-07-18T03:35:37.958 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-18T03:35:37.959 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-18T03:35:37.959 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-18T03:35:37.959 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-18T03:35:37.959 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-18T03:35:37.959 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-18T03:35:37.959 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-18T03:35:37.959 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-18T03:35:37.959 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-18T03:35:37.960 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-18T03:35:37.960 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-18T03:35:37.960 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-18T03:35:37.960 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-18T03:35:37.960 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-18T03:35:37.960 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-18T03:35:37.960 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-18T03:35:37.961 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-18T03:35:37.961 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-18T03:35:37.961 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-18T03:35:37.961 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-18T03:35:37.961 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-18T03:35:37.961 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-18T03:35:37.962 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-18T03:35:37.962 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-18T03:35:37.962 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-18T03:35:37.962 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-18T03:35:37.962 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-18T03:35:37.962 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-18T03:35:37.962 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-18T03:35:37.963 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-18T03:35:37.963 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-18T03:35:37.963 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-18T03:35:37.963 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-18T03:35:37.963 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-18T03:35:37.963 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-18T03:35:37.963 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-18T03:35:37.964 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-18T03:35:37.964 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2023-07-18T03:35:37.964 INFO:teuthology.orchestra.run.smithi107.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-18T03:35:37.964 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-18T03:35:37.964 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-18T03:35:37.964 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2023-07-18T03:35:37.964 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-18T03:35:37.965 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-18T03:35:37.965 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-18T03:35:37.965 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-18T03:35:37.965 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-18T03:35:37.965 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-18T03:35:37.965 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-18T03:35:37.965 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-18T03:35:37.965 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-18T03:35:37.966 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-18T03:35:37.966 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-18T03:35:37.966 INFO:teuthology.orchestra.run.smithi107.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-18T03:35:37.966 INFO:teuthology.orchestra.run.smithi107.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-18T03:35:37.966 INFO:teuthology.orchestra.run.smithi107.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-18T03:35:37.966 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-18T03:35:37.966 INFO:teuthology.orchestra.run.smithi107.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-18T03:35:37.967 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-18T03:35:37.967 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-18T03:35:37.967 INFO:teuthology.orchestra.run.smithi107.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-18T03:35:37.967 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-18T03:35:37.967 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-18T03:35:37.967 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-18T03:35:37.967 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-18T03:35:37.967 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-18T03:35:37.968 INFO:teuthology.orchestra.run.smithi107.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-18T03:35:37.968 INFO:teuthology.orchestra.run.smithi107.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-18T03:35:37.968 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-18T03:35:37.968 INFO:teuthology.orchestra.run.smithi107.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-18T03:35:37.968 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-18T03:35:37.968 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-18T03:35:37.969 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-18T03:35:37.969 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-18T03:35:37.969 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-18T03:35:37.969 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-18T03:35:37.969 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-18T03:35:37.969 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-18T03:35:37.969 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-18T03:35:37.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-18T03:35:37.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-18T03:35:37.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-18T03:35:37.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-18T03:35:37.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-18T03:35:37.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-18T03:35:37.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-18T03:35:37.970 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-18T03:35:37.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-18T03:35:37.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-18T03:35:37.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-18T03:35:37.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-18T03:35:37.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-18T03:35:37.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-18T03:35:37.971 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-18T03:35:37.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-18T03:35:37.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T03:35:37.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-18T03:35:37.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-18T03:35:37.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-18T03:35:37.972 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-18T03:35:37.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-18T03:35:37.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-18T03:35:37.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-18T03:35:37.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-18T03:35:37.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-18T03:35:37.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-18T03:35:37.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-18T03:35:37.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-18T03:35:37.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-18T03:35:37.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-18T03:35:37.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-18T03:35:37.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-18T03:35:37.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-18T03:35:37.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-18T03:35:37.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T03:35:37.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-18T03:35:37.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-18T03:35:37.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-18T03:35:37.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-18T03:35:37.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-18T03:35:37.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-18T03:35:37.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-18T03:35:37.976 INFO:teuthology.orchestra.run.smithi107.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-18T03:35:37.976 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-18T03:35:37.976 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-18T03:35:37.976 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-18T03:35:37.976 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-18T03:35:37.976 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-18T03:35:37.976 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-18T03:35:37.977 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-18T03:35:37.977 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-18T03:35:37.977 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-18T03:35:37.977 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-18T03:35:37.977 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling module streams: 2023-07-18T03:35:37.977 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql 10 2023-07-18T03:35:37.977 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:35:37.977 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2023-07-18T03:35:37.978 INFO:teuthology.orchestra.run.smithi107.stdout:=========================================================================================================================== 2023-07-18T03:35:37.978 INFO:teuthology.orchestra.run.smithi107.stdout:Install 128 Packages 2023-07-18T03:35:37.978 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 3 Packages 2023-07-18T03:35:37.978 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:35:37.978 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 205 M 2023-07-18T03:35:37.979 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2023-07-18T03:35:38.381 INFO:teuthology.orchestra.run.smithi107.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 28 kB/s | 6.4 kB 00:00 2023-07-18T03:35:39.181 INFO:teuthology.orchestra.run.smithi107.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 1.0 MB/s | 848 kB 00:00 2023-07-18T03:35:39.323 INFO:teuthology.orchestra.run.smithi107.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 1.1 MB/s | 165 kB 00:00 2023-07-18T03:35:40.107 INFO:teuthology.orchestra.run.smithi107.stdout:(4/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 2.8 MB/s | 5.4 MB 00:01 2023-07-18T03:35:40.215 INFO:teuthology.orchestra.run.smithi107.stdout:(5/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-07-18T03:35:40.374 INFO:teuthology.orchestra.run.smithi107.stdout:(6/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 5.8 MB/s | 1.5 MB 00:00 2023-07-18T03:35:41.090 INFO:teuthology.orchestra.run.smithi107.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 5.3 MB/s | 4.6 MB 00:00 2023-07-18T03:35:41.741 INFO:teuthology.orchestra.run.smithi107.stdout:(8/131): ceph-common-18.0.0-4746.g81cf785a.el8. 6.2 MB/s | 22 MB 00:03 2023-07-18T03:35:41.808 INFO:teuthology.orchestra.run.smithi107.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 367 kB/s | 24 kB 00:00 2023-07-18T03:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout:(10/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x8 10 MB/s | 18 MB 00:01 2023-07-18T03:35:42.208 INFO:teuthology.orchestra.run.smithi107.stdout:(11/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 1.9 MB/s | 258 kB 00:00 2023-07-18T03:35:42.333 INFO:teuthology.orchestra.run.smithi107.stdout:(12/131): libcephfs-devel-18.0.0-4746.g81cf785a 249 kB/s | 31 kB 00:00 2023-07-18T03:35:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:(13/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 5.7 MB/s | 772 kB 00:00 2023-07-18T03:35:42.567 INFO:teuthology.orchestra.run.smithi107.stdout:(14/131): libcephsqlite-18.0.0-4746.g81cf785a.e 1.3 MB/s | 137 kB 00:00 2023-07-18T03:35:42.742 INFO:teuthology.orchestra.run.smithi107.stdout:(15/131): ceph-radosgw-18.0.0-4746.g81cf785a.el 8.2 MB/s | 13 MB 00:01 2023-07-18T03:35:42.768 INFO:teuthology.orchestra.run.smithi107.stdout:(16/131): librados-devel-18.0.0-4746.g81cf785a. 695 kB/s | 139 kB 00:00 2023-07-18T03:35:42.843 INFO:teuthology.orchestra.run.smithi107.stdout:(17/131): libradosstriper1-18.0.0-4746.g81cf785 4.6 MB/s | 473 kB 00:00 2023-07-18T03:35:42.935 INFO:teuthology.orchestra.run.smithi107.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 488 kB/s | 45 kB 00:00 2023-07-18T03:35:43.052 INFO:teuthology.orchestra.run.smithi107.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 1.0 MB/s | 119 kB 00:00 2023-07-18T03:35:43.185 INFO:teuthology.orchestra.run.smithi107.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 1.4 MB/s | 197 kB 00:00 2023-07-18T03:35:43.286 INFO:teuthology.orchestra.run.smithi107.stdout:(21/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 13 MB/s | 6.7 MB 00:00 2023-07-18T03:35:43.319 INFO:teuthology.orchestra.run.smithi107.stdout:(22/131): python3-rados-18.0.0-4746.g81cf785a.e 2.7 MB/s | 370 kB 00:00 2023-07-18T03:35:43.394 INFO:teuthology.orchestra.run.smithi107.stdout:(23/131): python3-rbd-18.0.0-4746.g81cf785a.el8 3.3 MB/s | 364 kB 00:00 2023-07-18T03:35:43.420 INFO:teuthology.orchestra.run.smithi107.stdout:(24/131): python3-rgw-18.0.0-4746.g81cf785a.el8 1.0 MB/s | 107 kB 00:00 2023-07-18T03:35:43.478 INFO:teuthology.orchestra.run.smithi107.stdout:(25/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 1.1 MB/s | 94 kB 00:00 2023-07-18T03:35:43.578 INFO:teuthology.orchestra.run.smithi107.stdout:(26/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 1.8 MB/s | 179 kB 00:00 2023-07-18T03:35:43.671 INFO:teuthology.orchestra.run.smithi107.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4746.g 251 kB/s | 23 kB 00:00 2023-07-18T03:35:43.771 INFO:teuthology.orchestra.run.smithi107.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 1.3 MB/s | 129 kB 00:00 2023-07-18T03:35:43.846 INFO:teuthology.orchestra.run.smithi107.stdout:(29/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 8.1 MB/s | 3.4 MB 00:00 2023-07-18T03:35:44.005 INFO:teuthology.orchestra.run.smithi107.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 7.3 MB/s | 1.7 MB 00:00 2023-07-18T03:35:44.105 INFO:teuthology.orchestra.run.smithi107.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 2.4 MB/s | 240 kB 00:00 2023-07-18T03:35:44.189 INFO:teuthology.orchestra.run.smithi107.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 570 kB/s | 47 kB 00:00 2023-07-18T03:35:44.264 INFO:teuthology.orchestra.run.smithi107.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 194 kB/s | 15 kB 00:00 2023-07-18T03:35:44.348 INFO:teuthology.orchestra.run.smithi107.stdout:(34/131): ceph-volume-18.0.0-4746.g81cf785a.el8 3.0 MB/s | 258 kB 00:00 2023-07-18T03:35:44.440 INFO:teuthology.orchestra.run.smithi107.stdout:(35/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 1.8 MB/s | 166 kB 00:00 2023-07-18T03:35:44.532 INFO:teuthology.orchestra.run.smithi107.stdout:(36/131): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-07-18T03:35:44.548 INFO:teuthology.orchestra.run.smithi107.stdout:(37/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 166 kB/s | 18 kB 00:00 2023-07-18T03:35:44.866 INFO:teuthology.orchestra.run.smithi107.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 186 kB/s | 59 kB 00:00 2023-07-18T03:35:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 564 kB/s | 75 kB 00:00 2023-07-18T03:35:45.033 INFO:teuthology.orchestra.run.smithi107.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 612 kB/s | 306 kB 00:00 2023-07-18T03:35:45.100 INFO:teuthology.orchestra.run.smithi107.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 655 kB/s | 44 kB 00:00 2023-07-18T03:35:45.292 INFO:teuthology.orchestra.run.smithi107.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 966 kB/s | 185 kB 00:00 2023-07-18T03:35:45.892 INFO:teuthology.orchestra.run.smithi107.stdout:(43/131): ceph-test-18.0.0-4746.g81cf785a.el8.x 12 MB/s | 48 MB 00:04 2023-07-18T03:35:45.917 INFO:teuthology.orchestra.run.smithi107.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 53 kB/s | 33 kB 00:00 2023-07-18T03:35:45.942 INFO:teuthology.orchestra.run.smithi107.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 504 kB/s | 476 kB 00:00 2023-07-18T03:35:46.093 INFO:teuthology.orchestra.run.smithi107.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 81 kB/s | 16 kB 00:00 2023-07-18T03:35:46.135 INFO:teuthology.orchestra.run.smithi107.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 796 kB/s | 173 kB 00:00 2023-07-18T03:35:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 148 kB/s | 11 kB 00:00 2023-07-18T03:35:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 423 kB/s | 81 kB 00:00 2023-07-18T03:35:46.311 INFO:teuthology.orchestra.run.smithi107.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 176 kB/s | 18 kB 00:00 2023-07-18T03:35:46.386 INFO:teuthology.orchestra.run.smithi107.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 652 kB/s | 49 kB 00:00 2023-07-18T03:35:46.436 INFO:teuthology.orchestra.run.smithi107.stdout:(52/131): python3-cherrypy-18.4.0-1.el8.noarch. 787 kB/s | 384 kB 00:00 2023-07-18T03:35:46.461 INFO:teuthology.orchestra.run.smithi107.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 782 kB/s | 59 kB 00:00 2023-07-18T03:35:46.512 INFO:teuthology.orchestra.run.smithi107.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 803 kB/s | 60 kB 00:00 2023-07-18T03:35:46.595 INFO:teuthology.orchestra.run.smithi107.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 187 kB/s | 16 kB 00:00 2023-07-18T03:35:46.671 INFO:teuthology.orchestra.run.smithi107.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 449 kB/s | 34 kB 00:00 2023-07-18T03:35:46.696 INFO:teuthology.orchestra.run.smithi107.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.2 MB/s | 283 kB 00:00 2023-07-18T03:35:46.763 INFO:teuthology.orchestra.run.smithi107.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 925 kB/s | 61 kB 00:00 2023-07-18T03:35:46.847 INFO:teuthology.orchestra.run.smithi107.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2023-07-18T03:35:46.864 INFO:teuthology.orchestra.run.smithi107.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2023-07-18T03:35:46.914 INFO:teuthology.orchestra.run.smithi107.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 359 kB/s | 24 kB 00:00 2023-07-18T03:35:46.931 INFO:teuthology.orchestra.run.smithi107.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 439 kB/s | 29 kB 00:00 2023-07-18T03:35:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 413 kB/s | 27 kB 00:00 2023-07-18T03:35:47.023 INFO:teuthology.orchestra.run.smithi107.stdout:(64/131): python3-kubernetes-11.0.0-6.el8.noarc 1.6 MB/s | 1.2 MB 00:00 2023-07-18T03:35:47.098 INFO:teuthology.orchestra.run.smithi107.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 815 kB/s | 61 kB 00:00 2023-07-18T03:35:47.124 INFO:teuthology.orchestra.run.smithi107.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 1.3 MB/s | 255 kB 00:00 2023-07-18T03:35:47.174 INFO:teuthology.orchestra.run.smithi107.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-07-18T03:35:47.199 INFO:teuthology.orchestra.run.smithi107.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 306 kB/s | 23 kB 00:00 2023-07-18T03:35:47.224 INFO:teuthology.orchestra.run.smithi107.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 681 kB/s | 85 kB 00:00 2023-07-18T03:35:47.241 INFO:teuthology.orchestra.run.smithi107.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-07-18T03:35:47.291 INFO:teuthology.orchestra.run.smithi107.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 702 kB/s | 47 kB 00:00 2023-07-18T03:35:47.317 INFO:teuthology.orchestra.run.smithi107.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.8 MB/s | 200 kB 00:00 2023-07-18T03:35:47.342 INFO:teuthology.orchestra.run.smithi107.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 855 kB/s | 43 kB 00:00 2023-07-18T03:35:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-18T03:35:47.392 INFO:teuthology.orchestra.run.smithi107.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 959 kB/s | 48 kB 00:00 2023-07-18T03:35:47.443 INFO:teuthology.orchestra.run.smithi107.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2023-07-18T03:35:47.476 INFO:teuthology.orchestra.run.smithi107.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 513 kB/s | 43 kB 00:00 2023-07-18T03:35:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.8 MB/s | 109 kB 00:00 2023-07-18T03:35:47.527 INFO:teuthology.orchestra.run.smithi107.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-07-18T03:35:47.560 INFO:teuthology.orchestra.run.smithi107.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-07-18T03:35:47.586 INFO:teuthology.orchestra.run.smithi107.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 321 kB/s | 19 kB 00:00 2023-07-18T03:35:47.611 INFO:teuthology.orchestra.run.smithi107.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-07-18T03:35:47.645 INFO:teuthology.orchestra.run.smithi107.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-07-18T03:35:47.670 INFO:teuthology.orchestra.run.smithi107.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 307 kB/s | 18 kB 00:00 2023-07-18T03:35:47.712 INFO:teuthology.orchestra.run.smithi107.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2023-07-18T03:35:47.737 INFO:teuthology.orchestra.run.smithi107.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-07-18T03:35:47.779 INFO:teuthology.orchestra.run.smithi107.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-07-18T03:35:47.821 INFO:teuthology.orchestra.run.smithi107.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-07-18T03:35:47.871 INFO:teuthology.orchestra.run.smithi107.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 15 MB/s | 2.3 MB 00:00 2023-07-18T03:35:47.897 INFO:teuthology.orchestra.run.smithi107.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 800 kB/s | 60 kB 00:00 2023-07-18T03:35:47.914 INFO:teuthology.orchestra.run.smithi107.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-07-18T03:35:47.939 INFO:teuthology.orchestra.run.smithi107.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2023-07-18T03:35:47.964 INFO:teuthology.orchestra.run.smithi107.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 766 kB/s | 39 kB 00:00 2023-07-18T03:35:47.998 INFO:teuthology.orchestra.run.smithi107.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-07-18T03:35:48.023 INFO:teuthology.orchestra.run.smithi107.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-07-18T03:35:48.049 INFO:teuthology.orchestra.run.smithi107.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-18T03:35:48.091 INFO:teuthology.orchestra.run.smithi107.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-18T03:35:48.124 INFO:teuthology.orchestra.run.smithi107.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-07-18T03:35:48.166 INFO:teuthology.orchestra.run.smithi107.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-18T03:35:48.208 INFO:teuthology.orchestra.run.smithi107.stdout:(100/131): thrift-0.13.0-2.el8.x86_64.rpm 1.7 MB/s | 1.7 MB 00:01 2023-07-18T03:35:48.258 INFO:teuthology.orchestra.run.smithi107.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.4 MB/s | 323 kB 00:00 2023-07-18T03:35:48.292 INFO:teuthology.orchestra.run.smithi107.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 467 kB/s | 39 kB 00:00 2023-07-18T03:35:48.342 INFO:teuthology.orchestra.run.smithi107.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 8.7 MB/s | 688 kB 00:00 2023-07-18T03:35:48.376 INFO:teuthology.orchestra.run.smithi107.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-07-18T03:35:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-07-18T03:35:48.427 INFO:teuthology.orchestra.run.smithi107.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-07-18T03:35:48.477 INFO:teuthology.orchestra.run.smithi107.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-07-18T03:35:48.510 INFO:teuthology.orchestra.run.smithi107.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 800 kB/s | 27 kB 00:00 2023-07-18T03:35:48.552 INFO:teuthology.orchestra.run.smithi107.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-18T03:35:48.586 INFO:teuthology.orchestra.run.smithi107.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-07-18T03:35:48.711 INFO:teuthology.orchestra.run.smithi107.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-07-18T03:35:48.770 INFO:teuthology.orchestra.run.smithi107.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-07-18T03:35:48.845 INFO:teuthology.orchestra.run.smithi107.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-07-18T03:35:48.879 INFO:teuthology.orchestra.run.smithi107.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.0 MB/s | 225 kB 00:00 2023-07-18T03:35:48.904 INFO:teuthology.orchestra.run.smithi107.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-07-18T03:35:49.054 INFO:teuthology.orchestra.run.smithi107.stdout:(116/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-07-18T03:35:49.080 INFO:teuthology.orchestra.run.smithi107.stdout:(117/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 630 kB/s | 126 kB 00:00 2023-07-18T03:35:49.105 INFO:teuthology.orchestra.run.smithi107.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 100 kB/s | 20 kB 00:00 2023-07-18T03:35:49.130 INFO:teuthology.orchestra.run.smithi107.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-18T03:35:49.189 INFO:teuthology.orchestra.run.smithi107.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 6.4 MB/s | 709 kB 00:00 2023-07-18T03:35:49.214 INFO:teuthology.orchestra.run.smithi107.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2023-07-18T03:35:49.256 INFO:teuthology.orchestra.run.smithi107.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-18T03:35:49.306 INFO:teuthology.orchestra.run.smithi107.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-07-18T03:35:49.423 INFO:teuthology.orchestra.run.smithi107.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 15 MB/s | 4.9 MB 00:00 2023-07-18T03:35:49.499 INFO:teuthology.orchestra.run.smithi107.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-07-18T03:35:49.540 INFO:teuthology.orchestra.run.smithi107.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-07-18T03:35:49.607 INFO:teuthology.orchestra.run.smithi107.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-07-18T03:35:49.683 INFO:teuthology.orchestra.run.smithi107.stdout:(128/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-07-18T03:35:49.741 INFO:teuthology.orchestra.run.smithi107.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-18T03:35:50.116 INFO:teuthology.orchestra.run.smithi107.stdout:(130/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 7.0 MB/s | 3.6 MB 00:00 2023-07-18T03:35:50.174 INFO:teuthology.orchestra.run.smithi107.stdout:(131/131): librados2-18.0.0-4746.g81cf785a.el8. 5.8 MB/s | 3.7 MB 00:00 2023-07-18T03:35:50.179 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2023-07-18T03:35:50.179 INFO:teuthology.orchestra.run.smithi107.stdout:Total 17 MB/s | 205 MB 00:12 2023-07-18T03:35:51.790 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2023-07-18T03:35:51.922 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2023-07-18T03:35:51.923 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2023-07-18T03:35:53.641 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2023-07-18T03:35:53.645 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2023-07-18T03:35:58.677 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2023-07-18T03:35:58.856 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-18T03:35:59.041 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-18T03:35:59.199 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-18T03:35:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-18T03:35:59.557 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-18T03:36:00.021 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-18T03:36:00.240 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-18T03:36:00.676 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-18T03:36:00.849 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:36:00.982 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:36:01.232 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:36:01.352 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:36:01.563 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T03:36:01.746 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-18T03:36:01.998 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-18T03:36:02.204 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-18T03:36:02.639 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-18T03:36:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-18T03:36:03.056 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T03:36:03.216 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T03:36:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T03:36:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T03:36:03.754 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-18T03:36:03.932 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-18T03:36:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-18T03:36:04.231 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-18T03:36:04.403 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-18T03:36:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-18T03:36:05.237 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-18T03:36:05.414 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-18T03:36:06.098 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-18T03:36:06.539 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-18T03:36:07.390 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-18T03:36:07.671 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-18T03:36:07.889 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-18T03:36:08.125 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-18T03:36:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-18T03:36:08.477 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-18T03:36:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-18T03:36:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-18T03:36:10.952 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-18T03:36:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-18T03:36:12.425 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-18T03:36:12.703 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-18T03:36:12.931 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-18T03:36:13.140 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-18T03:36:13.344 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-18T03:36:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-18T03:36:13.720 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-18T03:36:13.880 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-18T03:36:14.032 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-18T03:36:14.178 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-18T03:36:14.273 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-18T03:36:14.516 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-18T03:36:14.718 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-18T03:36:14.871 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-18T03:36:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-18T03:36:15.172 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-18T03:36:15.366 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-18T03:36:15.558 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-18T03:36:15.723 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-18T03:36:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-18T03:36:16.042 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-18T03:36:16.239 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-18T03:36:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-18T03:36:16.850 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-18T03:36:17.168 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-18T03:36:17.377 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-18T03:36:17.630 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-18T03:36:17.864 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-18T03:36:18.195 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-18T03:36:18.396 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-18T03:36:18.554 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-18T03:36:18.813 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-18T03:36:19.003 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-18T03:36:19.166 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-18T03:36:19.482 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-18T03:36:19.738 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-18T03:36:19.976 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-18T03:36:20.205 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-18T03:36:20.370 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-18T03:36:20.601 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-18T03:36:20.876 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-18T03:36:21.085 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-18T03:36:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-18T03:36:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-18T03:36:21.523 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-18T03:36:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-18T03:36:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-18T03:36:22.072 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-18T03:36:22.225 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-18T03:36:22.438 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-18T03:36:22.617 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-18T03:36:22.800 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-18T03:36:23.061 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-18T03:36:23.286 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-18T03:36:23.444 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-18T03:36:23.648 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-18T03:36:24.957 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-18T03:36:25.213 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-18T03:36:25.473 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-18T03:36:25.822 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-18T03:36:26.096 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-18T03:36:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-18T03:36:27.109 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-18T03:36:27.275 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T03:36:27.380 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T03:36:27.534 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-18T03:36:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:36:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:36:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:36:33.501 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T03:36:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T03:36:33.791 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T03:36:54.081 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T03:36:54.081 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2023-07-18T03:36:54.082 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2023-07-18T03:36:54.082 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /dev 2023-07-18T03:36:54.082 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /run 2023-07-18T03:36:54.082 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2023-07-18T03:36:54.082 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2023-07-18T03:36:54.082 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2023-07-18T03:36:54.082 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2023-07-18T03:36:54.083 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /dev 2023-07-18T03:36:54.083 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:36:54.462 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T03:36:56.202 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T03:36:56.372 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T03:36:56.639 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T03:36:56.786 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T03:36:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T03:36:57.511 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T03:36:57.552 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T03:37:00.182 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T03:37:00.323 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T03:37:00.485 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-18T03:37:00.674 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-18T03:37:00.843 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T03:37:01.109 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T03:37:01.319 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T03:37:02.433 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T03:37:02.592 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T03:37:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T03:37:03.094 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-18T03:37:03.253 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T03:37:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T03:37:03.493 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T03:37:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T03:37:04.921 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-18T03:37:05.158 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T03:37:05.228 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T03:37:05.359 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T03:37:05.430 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T03:37:05.575 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T03:37:05.984 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T03:37:06.134 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T03:37:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T03:37:12.382 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-18T03:37:12.711 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-18T03:37:12.893 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 117/134 2023-07-18T03:37:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-18T03:37:13.340 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-18T03:37:13.597 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-18T03:37:13.821 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-18T03:37:13.979 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-18T03:37:14.125 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-18T03:37:17.381 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:37:17.631 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:37:17.671 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:37:17.924 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-18T03:37:18.149 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-18T03:37:18.441 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-18T03:37:18.830 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-18T03:37:19.005 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-18T03:37:19.113 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-18T03:37:19.297 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-18T03:37:19.326 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-18T03:37:19.497 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-18T03:37:19.519 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-18T03:37:19.692 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-18T03:37:19.693 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:37:19.903 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:37:19.904 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T03:37:20.138 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T03:37:20.322 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:37:20.599 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:37:20.599 INFO:teuthology.orchestra.run.smithi107.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-18T03:37:20.599 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-18T03:37:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-18T03:37:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-18T03:37:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-18T03:37:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-18T03:37:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:37:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:37:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T03:37:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-18T03:37:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-18T03:37:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-18T03:37:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-18T03:37:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-18T03:37:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-18T03:37:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-18T03:37:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-18T03:37:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-18T03:37:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-18T03:37:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-18T03:37:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-18T03:37:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-18T03:37:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-18T03:37:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-18T03:37:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-18T03:37:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-18T03:37:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-18T03:37:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-18T03:37:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-18T03:37:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-18T03:37:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-18T03:37:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-18T03:37:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-18T03:37:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-18T03:37:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-18T03:37:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-18T03:37:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-18T03:37:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-18T03:37:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-18T03:37:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-18T03:37:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-18T03:37:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-18T03:37:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-18T03:37:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-18T03:37:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-18T03:37:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-18T03:37:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-18T03:37:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-18T03:37:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-18T03:37:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-18T03:37:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-18T03:37:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-18T03:37:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-18T03:37:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-18T03:37:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-18T03:37:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-18T03:37:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-18T03:37:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-18T03:37:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-18T03:37:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-18T03:37:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-18T03:37:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-18T03:37:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-18T03:37:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-18T03:37:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-18T03:37:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-18T03:37:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-18T03:37:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-18T03:37:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-18T03:37:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-18T03:37:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-18T03:37:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-18T03:37:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-18T03:37:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-18T03:37:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-18T03:37:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-18T03:37:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-18T03:37:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-18T03:37:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-18T03:37:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-18T03:37:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-18T03:37:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-18T03:37:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-18T03:37:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-18T03:37:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-18T03:37:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-18T03:37:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-18T03:37:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-18T03:37:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-18T03:37:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-18T03:37:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-18T03:37:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-18T03:37:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-18T03:37:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-18T03:37:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-18T03:37:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-18T03:37:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-18T03:37:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-18T03:37:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-18T03:37:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-18T03:37:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-18T03:37:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-18T03:37:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-18T03:37:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-18T03:37:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-18T03:37:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-18T03:37:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-18T03:37:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-18T03:37:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-18T03:37:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-18T03:37:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-18T03:37:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-18T03:37:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-18T03:37:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-18T03:37:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-18T03:37:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-18T03:37:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-18T03:37:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-18T03:37:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-18T03:37:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-18T03:37:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-18T03:37:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-18T03:37:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-18T03:37:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-18T03:37:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-18T03:37:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-18T03:37:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-18T03:37:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-18T03:37:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-18T03:37:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-18T03:37:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-18T03:37:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-18T03:37:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:37:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-18T03:37:23.934 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:37:23.935 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:58 ago on Tue 18 Jul 2023 03:35:25 AM UTC. 2023-07-18T03:37:26.579 INFO:teuthology.orchestra.run.smithi107.stdout:Installed products updated. 2023-07-18T03:37:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:26.820 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2023-07-18T03:37:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-18T03:37:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.821 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2023-07-18T03:37:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-18T03:37:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.1-7.el8.noarch 2023-07-18T03:37:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.0.4-10.el8.x86_64 2023-07-18T03:37:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:37:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:37:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:37:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:37:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:37:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:37:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:37:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:37:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:37:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:37:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-18T03:37:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.1-9.el8.x86_64 2023-07-18T03:37:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-18T03:37:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-18T03:37:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: jq-1.6-3.el8.x86_64 2023-07-18T03:37:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-18T03:37:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-18T03:37:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-18T03:37:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-18T03:37:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-18T03:37:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-18T03:37:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: libpq-13.5-1.el8.x86_64 2023-07-18T03:37:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-18T03:37:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-18T03:37:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-18T03:37:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-18T03:37:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-18T03:37:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-18T03:37:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-18T03:37:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-18T03:37:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-18T03:37:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-18T03:37:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-18T03:37:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-18T03:37:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-18T03:37:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-18T03:37:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-18T03:37:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-18T03:37:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-18T03:37:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:37:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:37:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:37:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-18T03:37:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-18T03:37:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-18T03:37:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-18T03:37:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-18T03:37:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-18T03:37:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-18T03:37:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-18T03:37:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-18T03:37:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-18T03:37:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-18T03:37:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-18T03:37:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-18T03:37:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-18T03:37:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-18T03:37:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-18T03:37:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-18T03:37:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-18T03:37:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-18T03:37:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-18T03:37:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-18T03:37:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-18T03:37:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-18T03:37:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-18T03:37:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-18T03:37:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-18T03:37:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-18T03:37:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-2.6-1.el8.noarch 2023-07-18T03:37:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-18T03:37:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-18T03:37:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-18T03:37:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-18T03:37:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-18T03:37:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-18T03:37:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-18T03:37:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-18T03:37:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-18T03:37:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-18T03:37:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-18T03:37:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-18T03:37:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-18T03:37:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-18T03:37:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-18T03:37:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-18T03:37:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-18T03:37:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-18T03:37:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-18T03:37:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-18T03:37:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-18T03:37:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:37:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-18T03:37:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-18T03:37:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-18T03:37:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-18T03:37:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-18T03:37:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-18T03:37:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:26.846 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2023-07-18T03:37:27.134 DEBUG:teuthology.parallel:result is None 2023-07-18T03:37:27.134 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:37:27.395 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-18T03:37:27.460 INFO:teuthology.orchestra.run.smithi049.stdout:18.0.0-4746.g81cf785a.el8 2023-07-18T03:37:27.461 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-18T03:37:27.462 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-18T03:37:27.463 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:37:27.714 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-18T03:37:27.774 INFO:teuthology.orchestra.run.smithi107.stdout:18.0.0-4746.g81cf785a.el8 2023-07-18T03:37:27.775 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-18T03:37:27.775 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-18T03:37:27.776 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-18T03:37:27.776 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:37:27.777 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-18T03:37:27.876 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:37:27.877 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-18T03:37:27.969 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-18T03:37:27.969 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:37:27.970 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2023-07-18T03:37:28.042 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-18T03:37:28.152 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:37:28.152 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2023-07-18T03:37:28.223 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-18T03:37:28.337 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-18T03:37:28.338 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:37:28.338 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-18T03:37:28.408 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-18T03:37:28.517 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:37:28.518 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-18T03:37:28.586 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-18T03:37:28.697 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-18T03:37:28.698 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:37:28.698 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2023-07-18T03:37:28.768 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-18T03:37:28.878 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:37:28.878 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2023-07-18T03:37:28.950 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-18T03:37:29.061 INFO:teuthology.run_tasks:Running task ceph... 2023-07-18T03:37:29.133 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-07-18T03:37:29.134 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /var/log/ceph 2023-07-18T03:37:29.138 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /var/log/ceph 2023-07-18T03:37:29.207 INFO:tasks.ceph:Disabling ceph logrotate... 2023-07-18T03:37:29.207 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-18T03:37:29.212 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-18T03:37:29.281 INFO:tasks.ceph:Creating extra log directories... 2023-07-18T03:37:29.281 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-18T03:37:29.286 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-18T03:37:29.364 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-07-18T03:37:29.364 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-07-18T03:37:29.364 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-07-18T03:37:29.365 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:37:29.365 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-18T03:37:29.408 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-18T03:37:29.630 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-18T03:37:29.701 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:37:29.701 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-18T03:37:29.743 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-18T03:37:29.960 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-18T03:37:30.035 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-07-18T03:37:30.036 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-07-18T03:37:30.036 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747', 'branch': 'wip-rishabh-2023Jul13-m-quota', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}', 'email': None, 'first_in_suite': False, 'job_id': '7342747', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-2023Jul13-m-quota'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-2023Jul13-m-quota', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-2023Jul13-m-quota', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi049.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi107.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots', 'tasks.cephfs.test_volumes.TestSubvolumeSnapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b', 'timestamp': '2023-07-18_02:11:50', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-07-18T03:37:30.037 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-18T03:37:30.041 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-18T03:37:30.086 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/run/ceph 2023-07-18T03:37:30.127 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/run/ceph 2023-07-18T03:37:30.197 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:37:30.197 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2023-07-18T03:37:30.240 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-18T03:37:30.241 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2023-07-18T03:37:30.325 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-18T03:37:30.325 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:37:30.325 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 450 Links: 1 2023-07-18T03:37:30.326 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:37:30.326 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:37:30.326 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2023-07-18 03:33:00.095538859 +0000 2023-07-18T03:37:30.326 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2023-07-18 03:29:49.279978998 +0000 2023-07-18T03:37:30.326 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2023-07-18 03:29:49.279978998 +0000 2023-07-18T03:37:30.326 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2023-07-18 03:29:49.254979676 +0000 2023-07-18T03:37:30.326 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-18T03:37:30.432 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2023-07-18T03:37:30.432 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2023-07-18T03:37:30.432 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000240407 s, 2.1 MB/s 2023-07-18T03:37:30.435 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-18T03:37:30.481 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2023-07-18T03:37:30.565 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-18T03:37:30.566 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:37:30.566 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 456 Links: 1 2023-07-18T03:37:30.566 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:37:30.566 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:37:30.566 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2023-07-18 03:33:00.095538859 +0000 2023-07-18T03:37:30.566 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2023-07-18 03:29:50.115956304 +0000 2023-07-18T03:37:30.566 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2023-07-18 03:29:50.115956304 +0000 2023-07-18T03:37:30.567 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2023-07-18 03:29:50.093956901 +0000 2023-07-18T03:37:30.567 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-18T03:37:30.671 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2023-07-18T03:37:30.672 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2023-07-18T03:37:30.672 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000195325 s, 2.6 MB/s 2023-07-18T03:37:30.674 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-18T03:37:30.720 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2023-07-18T03:37:30.805 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-18T03:37:30.806 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:37:30.806 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 462 Links: 1 2023-07-18T03:37:30.806 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:37:30.806 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:37:30.806 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2023-07-18 03:33:00.096538832 +0000 2023-07-18T03:37:30.806 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2023-07-18 03:29:50.965933230 +0000 2023-07-18T03:37:30.807 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2023-07-18 03:29:50.965933230 +0000 2023-07-18T03:37:30.807 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2023-07-18 03:29:50.941933882 +0000 2023-07-18T03:37:30.807 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-18T03:37:30.911 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2023-07-18T03:37:30.911 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2023-07-18T03:37:30.911 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000189078 s, 2.7 MB/s 2023-07-18T03:37:30.914 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-18T03:37:30.959 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2023-07-18T03:37:31.044 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-18T03:37:31.044 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:37:31.044 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 468 Links: 1 2023-07-18T03:37:31.044 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:37:31.045 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:37:31.045 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2023-07-18 03:33:00.096538832 +0000 2023-07-18T03:37:31.045 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2023-07-18 03:29:51.839909505 +0000 2023-07-18T03:37:31.045 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2023-07-18 03:29:51.839909505 +0000 2023-07-18T03:37:31.045 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2023-07-18 03:29:51.816910129 +0000 2023-07-18T03:37:31.045 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-18T03:37:31.150 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2023-07-18T03:37:31.151 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2023-07-18T03:37:31.151 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000187237 s, 2.7 MB/s 2023-07-18T03:37:31.153 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-18T03:37:31.198 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-18T03:37:31.199 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:37:31.199 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2023-07-18T03:37:31.242 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-18T03:37:31.242 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2023-07-18T03:37:31.325 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-18T03:37:31.326 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:37:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 450 Links: 1 2023-07-18T03:37:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:37:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:37:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-18 03:37:20.733503639 +0000 2023-07-18T03:37:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-18 03:29:23.449362025 +0000 2023-07-18T03:37:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-18 03:29:23.449362025 +0000 2023-07-18T03:37:31.327 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2023-07-18 03:29:23.422362467 +0000 2023-07-18T03:37:31.327 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-18T03:37:31.431 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2023-07-18T03:37:31.432 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2023-07-18T03:37:31.432 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000372475 s, 1.4 MB/s 2023-07-18T03:37:31.434 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-18T03:37:31.480 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2023-07-18T03:37:31.525 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-18T03:37:31.525 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:37:31.525 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 456 Links: 1 2023-07-18T03:37:31.525 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:37:31.525 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:37:31.525 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-18 03:37:20.733503639 +0000 2023-07-18T03:37:31.526 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-18 03:29:24.276348481 +0000 2023-07-18T03:37:31.526 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-18 03:29:24.276348481 +0000 2023-07-18T03:37:31.526 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2023-07-18 03:29:24.247348956 +0000 2023-07-18T03:37:31.526 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-18T03:37:31.632 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2023-07-18T03:37:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2023-07-18T03:37:31.633 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.00027821 s, 1.8 MB/s 2023-07-18T03:37:31.634 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-18T03:37:31.679 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2023-07-18T03:37:31.763 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-18T03:37:31.763 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:37:31.763 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 462 Links: 1 2023-07-18T03:37:31.763 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:37:31.764 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:37:31.764 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-18 03:37:20.734503622 +0000 2023-07-18T03:37:31.764 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-18 03:29:25.091335134 +0000 2023-07-18T03:37:31.764 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-18 03:29:25.091335134 +0000 2023-07-18T03:37:31.764 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2023-07-18 03:29:25.065335560 +0000 2023-07-18T03:37:31.764 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-18T03:37:31.870 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2023-07-18T03:37:31.870 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2023-07-18T03:37:31.870 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000276376 s, 1.9 MB/s 2023-07-18T03:37:31.872 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-18T03:37:31.918 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2023-07-18T03:37:32.002 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-18T03:37:32.002 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:37:32.002 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 468 Links: 1 2023-07-18T03:37:32.003 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:37:32.004 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:37:32.005 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-18 03:37:20.734503622 +0000 2023-07-18T03:37:32.005 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-18 03:29:25.921321541 +0000 2023-07-18T03:37:32.005 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-18 03:29:25.921321541 +0000 2023-07-18T03:37:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2023-07-18 03:29:25.892322016 +0000 2023-07-18T03:37:32.005 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-18T03:37:32.109 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2023-07-18T03:37:32.110 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2023-07-18T03:37:32.110 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000299762 s, 1.7 MB/s 2023-07-18T03:37:32.112 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-18T03:37:32.161 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-18T03:37:32.161 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi107.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-18T03:37:32.162 INFO:tasks.ceph:Generating config... 2023-07-18T03:37:32.164 INFO:tasks.ceph:[client] client mount timeout = 600 2023-07-18T03:37:32.164 INFO:tasks.ceph:[client] debug client = 20 2023-07-18T03:37:32.164 INFO:tasks.ceph:[client] debug ms = 1 2023-07-18T03:37:32.165 INFO:tasks.ceph:[client] pid file = 2023-07-18T03:37:32.165 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-07-18T03:37:32.165 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-07-18T03:37:32.166 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-07-18T03:37:32.166 INFO:tasks.ceph:[mds] debug mds = 20 2023-07-18T03:37:32.166 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-07-18T03:37:32.166 INFO:tasks.ceph:[mds] debug ms = 1 2023-07-18T03:37:32.167 INFO:tasks.ceph:[mds] mds debug frag = True 2023-07-18T03:37:32.167 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-07-18T03:37:32.167 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-07-18T03:37:32.167 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-07-18T03:37:32.167 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-07-18T03:37:32.168 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-07-18T03:37:32.168 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-07-18T03:37:32.168 INFO:tasks.ceph:[mgr] debug client = 20 2023-07-18T03:37:32.168 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-07-18T03:37:32.168 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-07-18T03:37:32.169 INFO:tasks.ceph:[mgr] debug ms = 1 2023-07-18T03:37:32.169 INFO:tasks.ceph:[mon] debug mon = 20 2023-07-18T03:37:32.169 INFO:tasks.ceph:[mon] debug ms = 1 2023-07-18T03:37:32.169 INFO:tasks.ceph:[mon] debug paxos = 20 2023-07-18T03:37:32.169 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-07-18T03:37:32.170 INFO:tasks.ceph:[osd] bdev async discard = True 2023-07-18T03:37:32.170 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-07-18T03:37:32.170 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-07-18T03:37:32.170 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-07-18T03:37:32.170 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-07-18T03:37:32.171 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-07-18T03:37:32.171 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-07-18T03:37:32.171 INFO:tasks.ceph:[osd] debug ms = 1 2023-07-18T03:37:32.171 INFO:tasks.ceph:[osd] debug osd = 20 2023-07-18T03:37:32.171 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-07-18T03:37:32.172 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-07-18T03:37:32.172 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-07-18T03:37:32.172 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-07-18T03:37:32.173 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-07-18T03:37:32.173 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-07-18T03:37:32.173 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-07-18T03:37:32.173 INFO:tasks.ceph:Setting up mon.a... 2023-07-18T03:37:32.174 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-07-18T03:37:32.295 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.keyring 2023-07-18T03:37:32.301 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-07-18T03:37:32.407 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-18T03:37:32.477 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.49'), ('mon.b', '172.21.15.107'), ('mon.c', '[v2:172.21.15.107:3301,v1:172.21.15.107:6790]')] 2023-07-18T03:37:32.478 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.49,172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-07-18T03:37:32.480 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:37:32.480 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-07-18T03:37:32.524 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.49 --add b 172.21.15.107 --addv c '[v2:172.21.15.107:3301,v1:172.21.15.107:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:37:32.738 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:37:32.738 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: generated fsid e859a751-7613-40d7-973c-3e47a5e301cb 2023-07-18T03:37:32.738 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = pacific 2023-07-18T03:37:32.738 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 0 2023-07-18T03:37:32.739 INFO:teuthology.orchestra.run.smithi049.stdout:fsid e859a751-7613-40d7-973c-3e47a5e301cb 2023-07-18T03:37:32.739 INFO:teuthology.orchestra.run.smithi049.stdout:last_changed 2023-07-18T03:37:32.743482+0000 2023-07-18T03:37:32.739 INFO:teuthology.orchestra.run.smithi049.stdout:created 2023-07-18T03:37:32.743482+0000 2023-07-18T03:37:32.739 INFO:teuthology.orchestra.run.smithi049.stdout:min_mon_release 16 (pacific) 2023-07-18T03:37:32.739 INFO:teuthology.orchestra.run.smithi049.stdout:election_strategy: 1 2023-07-18T03:37:32.739 INFO:teuthology.orchestra.run.smithi049.stdout:0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2023-07-18T03:37:32.739 INFO:teuthology.orchestra.run.smithi049.stdout:1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2023-07-18T03:37:32.739 INFO:teuthology.orchestra.run.smithi049.stdout:2: [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] mon.c 2023-07-18T03:37:32.740 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-07-18T03:37:32.741 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-07-18T03:37:32.784 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e859a751-7613-40d7-973c-3e47a5e301cb... 2023-07-18T03:37:32.785 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-18T03:37:32.828 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-18T03:37:32.947 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2023-07-18T03:37:32.948 INFO:teuthology.orchestra.run.smithi049.stdout: chdir = "" 2023-07-18T03:37:32.948 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-18T03:37:32.948 INFO:teuthology.orchestra.run.smithi049.stdout: auth supported = cephx 2023-07-18T03:37:32.948 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.948 INFO:teuthology.orchestra.run.smithi049.stdout: filestore xattr use omap = true 2023-07-18T03:37:32.949 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.949 INFO:teuthology.orchestra.run.smithi049.stdout: mon clock drift allowed = 1.000 2023-07-18T03:37:32.949 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.949 INFO:teuthology.orchestra.run.smithi049.stdout: osd crush chooseleaf type = 0 2023-07-18T03:37:32.949 INFO:teuthology.orchestra.run.smithi049.stdout: auth debug = true 2023-07-18T03:37:32.949 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.949 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on old message = true 2023-07-18T03:37:32.949 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on bug = true 2023-07-18T03:37:32.950 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.950 INFO:teuthology.orchestra.run.smithi049.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-18T03:37:32.950 INFO:teuthology.orchestra.run.smithi049.stdout: mon pg warn max object skew = 0 2023-07-18T03:37:32.950 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.950 INFO:teuthology.orchestra.run.smithi049.stdout: # disable pg_autoscaler by default for new pools 2023-07-18T03:37:32.950 INFO:teuthology.orchestra.run.smithi049.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-18T03:37:32.950 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.951 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default size = 2 2023-07-18T03:37:32.951 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.951 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow primary affinity = true 2023-07-18T03:37:32.951 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow pg remap = true 2023-07-18T03:37:32.951 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on legacy crush tunables = false 2023-07-18T03:37:32.951 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on crush straw calc version zero = false 2023-07-18T03:37:32.951 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on no sortbitwise = false 2023-07-18T03:37:32.951 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on osd down out interval zero = false 2023-07-18T03:37:32.952 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on too few osds = false 2023-07-18T03:37:32.952 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-18T03:37:32.952 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-18T03:37:32.952 INFO:teuthology.orchestra.run.smithi049.stdout: mon_allow_pool_size_one = true 2023-07-18T03:37:32.952 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.952 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-18T03:37:32.952 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.952 INFO:teuthology.orchestra.run.smithi049.stdout: osd default data pool replay window = 5 2023-07-18T03:37:32.953 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.953 INFO:teuthology.orchestra.run.smithi049.stdout: mon allow pool delete = true 2023-07-18T03:37:32.953 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.953 INFO:teuthology.orchestra.run.smithi049.stdout: mon cluster log file level = debug 2023-07-18T03:37:32.953 INFO:teuthology.orchestra.run.smithi049.stdout: debug asserts on shutdown = true 2023-07-18T03:37:32.953 INFO:teuthology.orchestra.run.smithi049.stdout: mon health detail to clog = false 2023-07-18T03:37:32.953 INFO:teuthology.orchestra.run.smithi049.stdout: mon host = "172.21.15.49,172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790]" 2023-07-18T03:37:32.954 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on skipped message = False 2023-07-18T03:37:32.954 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = e859a751-7613-40d7-973c-3e47a5e301cb 2023-07-18T03:37:32.954 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.954 INFO:teuthology.orchestra.run.smithi049.stdout:[osd] 2023-07-18T03:37:32.954 INFO:teuthology.orchestra.run.smithi049.stdout: osd journal size = 100 2023-07-18T03:37:32.954 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.954 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub load threshold = 5.0 2023-07-18T03:37:32.954 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub max interval = 600 2023-07-18T03:37:32.955 INFO:teuthology.orchestra.run.smithi049.stdout: osd mclock profile = high_recovery_ops 2023-07-18T03:37:32.955 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.955 INFO:teuthology.orchestra.run.smithi049.stdout: osd recover clone overlap = true 2023-07-18T03:37:32.955 INFO:teuthology.orchestra.run.smithi049.stdout: osd recovery max chunk = 1048576 2023-07-18T03:37:32.955 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.955 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug shutdown = true 2023-07-18T03:37:32.955 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug op order = true 2023-07-18T03:37:32.955 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug verify stray on activate = true 2023-07-18T03:37:32.956 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.956 INFO:teuthology.orchestra.run.smithi049.stdout: osd open classes on start = true 2023-07-18T03:37:32.956 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug pg log writeout = true 2023-07-18T03:37:32.956 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.956 INFO:teuthology.orchestra.run.smithi049.stdout: osd deep scrub update digest min age = 30 2023-07-18T03:37:32.956 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.956 INFO:teuthology.orchestra.run.smithi049.stdout: osd map max advance = 10 2023-07-18T03:37:32.957 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.957 INFO:teuthology.orchestra.run.smithi049.stdout: journal zero on create = true 2023-07-18T03:37:32.957 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.957 INFO:teuthology.orchestra.run.smithi049.stdout: filestore ondisk finisher threads = 3 2023-07-18T03:37:32.957 INFO:teuthology.orchestra.run.smithi049.stdout: filestore apply finisher threads = 3 2023-07-18T03:37:32.957 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.957 INFO:teuthology.orchestra.run.smithi049.stdout: bdev debug aio = true 2023-07-18T03:37:32.957 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug misdirected ops = true 2023-07-18T03:37:32.958 INFO:teuthology.orchestra.run.smithi049.stdout: bdev async discard = True 2023-07-18T03:37:32.958 INFO:teuthology.orchestra.run.smithi049.stdout: bdev enable discard = True 2023-07-18T03:37:32.958 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore allocator = bitmap 2023-07-18T03:37:32.958 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore block size = 96636764160 2023-07-18T03:37:32.958 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore fsck on mount = True 2023-07-18T03:37:32.958 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluefs = 1/20 2023-07-18T03:37:32.958 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluestore = 1/20 2023-07-18T03:37:32.958 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-07-18T03:37:32.959 INFO:teuthology.orchestra.run.smithi049.stdout: debug osd = 20 2023-07-18T03:37:32.959 INFO:teuthology.orchestra.run.smithi049.stdout: debug rocksdb = 4/10 2023-07-18T03:37:32.959 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd backfillfull_ratio = 0.85 2023-07-18T03:37:32.959 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd full ratio = 0.9 2023-07-18T03:37:32.959 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd nearfull ratio = 0.8 2023-07-18T03:37:32.959 INFO:teuthology.orchestra.run.smithi049.stdout: osd failsafe full ratio = 0.95 2023-07-18T03:37:32.959 INFO:teuthology.orchestra.run.smithi049.stdout: osd objectstore = bluestore 2023-07-18T03:37:32.959 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2023-07-18T03:37:32.960 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.960 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr] 2023-07-18T03:37:32.960 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-07-18T03:37:32.960 INFO:teuthology.orchestra.run.smithi049.stdout: debug mgr = 20 2023-07-18T03:37:32.960 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2023-07-18T03:37:32.960 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2023-07-18T03:37:32.960 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2023-07-18T03:37:32.961 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2023-07-18T03:37:32.961 INFO:teuthology.orchestra.run.smithi049.stdout: mgr/telemetry/nag = false 2023-07-18T03:37:32.961 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2023-07-18T03:37:32.961 INFO:teuthology.orchestra.run.smithi049.stdout: debug finisher = 20 2023-07-18T03:37:32.961 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.961 INFO:teuthology.orchestra.run.smithi049.stdout:[mon] 2023-07-18T03:37:32.961 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-07-18T03:37:32.961 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2023-07-18T03:37:32.962 INFO:teuthology.orchestra.run.smithi049.stdout: debug paxos = 20 2023-07-18T03:37:32.962 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2023-07-18T03:37:32.962 INFO:teuthology.orchestra.run.smithi049.stdout: mon data avail warn = 5 2023-07-18T03:37:32.962 INFO:teuthology.orchestra.run.smithi049.stdout: mon mgr mkfs grace = 240 2023-07-18T03:37:32.962 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2023-07-18T03:37:32.962 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd reporter subtree level = osd 2023-07-18T03:37:32.962 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd prime pg temp = true 2023-07-18T03:37:32.962 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2023-07-18T03:37:32.963 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.963 INFO:teuthology.orchestra.run.smithi049.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-18T03:37:32.963 INFO:teuthology.orchestra.run.smithi049.stdout: auth mon ticket ttl = 660 # 11m 2023-07-18T03:37:32.963 INFO:teuthology.orchestra.run.smithi049.stdout: auth service ticket ttl = 240 # 4m 2023-07-18T03:37:32.963 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.963 INFO:teuthology.orchestra.run.smithi049.stdout: # don't complain about insecure global_id in the test suite 2023-07-18T03:37:32.963 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-18T03:37:32.963 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-18T03:37:32.964 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.964 INFO:teuthology.orchestra.run.smithi049.stdout: # 1m isn't quite enough 2023-07-18T03:37:32.964 INFO:teuthology.orchestra.run.smithi049.stdout: mon_down_mkfs_grace = 2m 2023-07-18T03:37:32.964 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.964 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_filestore_osds = false 2023-07-18T03:37:32.964 INFO:teuthology.orchestra.run.smithi049.stdout: mon op complaint time = 120 2023-07-18T03:37:32.964 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:37:32.965 INFO:teuthology.orchestra.run.smithi049.stdout:[client] 2023-07-18T03:37:32.965 INFO:teuthology.orchestra.run.smithi049.stdout: rgw cache enabled = true 2023-07-18T03:37:32.965 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable ops log = true 2023-07-18T03:37:32.965 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable usage log = true 2023-07-18T03:37:32.965 INFO:teuthology.orchestra.run.smithi049.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-18T03:37:32.965 INFO:teuthology.orchestra.run.smithi049.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-18T03:37:32.965 INFO:teuthology.orchestra.run.smithi049.stdout: client mount timeout = 600 2023-07-18T03:37:32.965 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2023-07-18T03:37:32.966 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-07-18T03:37:32.966 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = "" 2023-07-18T03:37:32.966 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2023-07-18T03:37:32.966 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2023-07-18T03:37:32.966 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.a] 2023-07-18T03:37:32.966 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.b] 2023-07-18T03:37:32.966 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.c] 2023-07-18T03:37:32.966 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2023-07-18T03:37:32.967 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2023-07-18T03:37:32.967 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2023-07-18T03:37:32.967 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2023-07-18T03:37:32.967 INFO:teuthology.orchestra.run.smithi049.stdout:[mds] 2023-07-18T03:37:32.967 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds = 20 2023-07-18T03:37:32.967 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds balancer = 20 2023-07-18T03:37:32.967 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-07-18T03:37:32.967 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug frag = True 2023-07-18T03:37:32.968 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug scatterstat = True 2023-07-18T03:37:32.968 INFO:teuthology.orchestra.run.smithi049.stdout: mds op complaint time = 180 2023-07-18T03:37:32.968 INFO:teuthology.orchestra.run.smithi049.stdout: mds verify scatter = True 2023-07-18T03:37:32.968 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2023-07-18T03:37:32.968 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2023-07-18T03:37:32.968 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2023-07-18T03:37:32.969 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2023-07-18T03:37:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: chdir = "" 2023-07-18T03:37:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-18T03:37:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: auth supported = cephx 2023-07-18T03:37:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: filestore xattr use omap = true 2023-07-18T03:37:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: mon clock drift allowed = 1.000 2023-07-18T03:37:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: osd crush chooseleaf type = 0 2023-07-18T03:37:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: auth debug = true 2023-07-18T03:37:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on old message = true 2023-07-18T03:37:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on bug = true 2023-07-18T03:37:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-18T03:37:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: mon pg warn max object skew = 0 2023-07-18T03:37:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: # disable pg_autoscaler by default for new pools 2023-07-18T03:37:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-18T03:37:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default size = 2 2023-07-18T03:37:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow primary affinity = true 2023-07-18T03:37:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow pg remap = true 2023-07-18T03:37:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on legacy crush tunables = false 2023-07-18T03:37:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on crush straw calc version zero = false 2023-07-18T03:37:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on no sortbitwise = false 2023-07-18T03:37:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on osd down out interval zero = false 2023-07-18T03:37:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on too few osds = false 2023-07-18T03:37:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-18T03:37:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-18T03:37:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: mon_allow_pool_size_one = true 2023-07-18T03:37:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-18T03:37:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: osd default data pool replay window = 5 2023-07-18T03:37:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: mon allow pool delete = true 2023-07-18T03:37:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: mon cluster log file level = debug 2023-07-18T03:37:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: debug asserts on shutdown = true 2023-07-18T03:37:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: mon health detail to clog = false 2023-07-18T03:37:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: mon host = "172.21.15.49,172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790]" 2023-07-18T03:37:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on skipped message = False 2023-07-18T03:37:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = e859a751-7613-40d7-973c-3e47a5e301cb 2023-07-18T03:37:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.975 INFO:teuthology.orchestra.run.smithi107.stdout:[osd] 2023-07-18T03:37:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: osd journal size = 100 2023-07-18T03:37:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub load threshold = 5.0 2023-07-18T03:37:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub max interval = 600 2023-07-18T03:37:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: osd mclock profile = high_recovery_ops 2023-07-18T03:37:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: osd recover clone overlap = true 2023-07-18T03:37:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: osd recovery max chunk = 1048576 2023-07-18T03:37:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug shutdown = true 2023-07-18T03:37:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug op order = true 2023-07-18T03:37:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug verify stray on activate = true 2023-07-18T03:37:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: osd open classes on start = true 2023-07-18T03:37:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug pg log writeout = true 2023-07-18T03:37:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: osd deep scrub update digest min age = 30 2023-07-18T03:37:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: osd map max advance = 10 2023-07-18T03:37:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: journal zero on create = true 2023-07-18T03:37:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: filestore ondisk finisher threads = 3 2023-07-18T03:37:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: filestore apply finisher threads = 3 2023-07-18T03:37:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: bdev debug aio = true 2023-07-18T03:37:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug misdirected ops = true 2023-07-18T03:37:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: bdev async discard = True 2023-07-18T03:37:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: bdev enable discard = True 2023-07-18T03:37:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore allocator = bitmap 2023-07-18T03:37:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore block size = 96636764160 2023-07-18T03:37:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore fsck on mount = True 2023-07-18T03:37:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluefs = 1/20 2023-07-18T03:37:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluestore = 1/20 2023-07-18T03:37:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2023-07-18T03:37:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: debug osd = 20 2023-07-18T03:37:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: debug rocksdb = 4/10 2023-07-18T03:37:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd backfillfull_ratio = 0.85 2023-07-18T03:37:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd full ratio = 0.9 2023-07-18T03:37:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd nearfull ratio = 0.8 2023-07-18T03:37:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: osd failsafe full ratio = 0.95 2023-07-18T03:37:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: osd objectstore = bluestore 2023-07-18T03:37:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2023-07-18T03:37:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.981 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr] 2023-07-18T03:37:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2023-07-18T03:37:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: debug mgr = 20 2023-07-18T03:37:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2023-07-18T03:37:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2023-07-18T03:37:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2023-07-18T03:37:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2023-07-18T03:37:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: mgr/telemetry/nag = false 2023-07-18T03:37:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: debug client = 20 2023-07-18T03:37:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: debug finisher = 20 2023-07-18T03:37:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.982 INFO:teuthology.orchestra.run.smithi107.stdout:[mon] 2023-07-18T03:37:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2023-07-18T03:37:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2023-07-18T03:37:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: debug paxos = 20 2023-07-18T03:37:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2023-07-18T03:37:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: mon data avail warn = 5 2023-07-18T03:37:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: mon mgr mkfs grace = 240 2023-07-18T03:37:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2023-07-18T03:37:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd reporter subtree level = osd 2023-07-18T03:37:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd prime pg temp = true 2023-07-18T03:37:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2023-07-18T03:37:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-18T03:37:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: auth mon ticket ttl = 660 # 11m 2023-07-18T03:37:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: auth service ticket ttl = 240 # 4m 2023-07-18T03:37:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: # don't complain about insecure global_id in the test suite 2023-07-18T03:37:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-18T03:37:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-18T03:37:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: # 1m isn't quite enough 2023-07-18T03:37:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: mon_down_mkfs_grace = 2m 2023-07-18T03:37:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_filestore_osds = false 2023-07-18T03:37:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: mon op complaint time = 120 2023-07-18T03:37:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-18T03:37:32.985 INFO:teuthology.orchestra.run.smithi107.stdout:[client] 2023-07-18T03:37:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: rgw cache enabled = true 2023-07-18T03:37:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable ops log = true 2023-07-18T03:37:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable usage log = true 2023-07-18T03:37:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-18T03:37:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-18T03:37:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: client mount timeout = 600 2023-07-18T03:37:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: debug client = 20 2023-07-18T03:37:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2023-07-18T03:37:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = "" 2023-07-18T03:37:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 900 2023-07-18T03:37:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 900 2023-07-18T03:37:32.987 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.a] 2023-07-18T03:37:32.987 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.b] 2023-07-18T03:37:32.987 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.c] 2023-07-18T03:37:32.987 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a] 2023-07-18T03:37:32.987 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c] 2023-07-18T03:37:32.988 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b] 2023-07-18T03:37:32.988 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d] 2023-07-18T03:37:32.988 INFO:teuthology.orchestra.run.smithi107.stdout:[mds] 2023-07-18T03:37:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: debug mds = 20 2023-07-18T03:37:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: debug mds balancer = 20 2023-07-18T03:37:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2023-07-18T03:37:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug frag = True 2023-07-18T03:37:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug scatterstat = True 2023-07-18T03:37:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: mds op complaint time = 180 2023-07-18T03:37:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: mds verify scatter = True 2023-07-18T03:37:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2023-07-18T03:37:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 900 2023-07-18T03:37:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 900 2023-07-18T03:37:32.990 INFO:tasks.ceph:Creating admin key on mon.a... 2023-07-18T03:37:32.991 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-07-18T03:37:33.106 INFO:tasks.ceph:Copying monmap to all nodes... 2023-07-18T03:37:33.107 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:37:33.107 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-07-18T03:37:33.151 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:37:33.151 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-07-18T03:37:33.234 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:37:33.234 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:37:33.235 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-18T03:37:33.235 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-18T03:37:33.371 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:37:33.372 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:37:33.454 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:37:33.455 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:37:33.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-18T03:37:33.456 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-18T03:37:33.559 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:37:33.559 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:37:33.641 INFO:tasks.ceph:Setting up mon nodes... 2023-07-18T03:37:33.642 INFO:tasks.ceph:Setting up mgr nodes... 2023-07-18T03:37:33.642 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-07-18T03:37:33.770 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-07-18T03:37:33.776 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-07-18T03:37:33.916 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-07-18T03:37:33.923 INFO:tasks.ceph:Setting up mds nodes... 2023-07-18T03:37:33.923 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-07-18T03:37:34.052 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-07-18T03:37:34.057 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-07-18T03:37:34.128 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-07-18T03:37:34.256 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-07-18T03:37:34.262 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-07-18T03:37:34.332 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-07-18T03:37:34.455 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-07-18T03:37:34.460 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-07-18T03:37:34.529 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-07-18T03:37:34.666 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-07-18T03:37:34.671 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-07-18T03:37:34.745 INFO:tasks.ceph_client:Setting up client nodes... 2023-07-18T03:37:34.746 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-18T03:37:34.846 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-07-18T03:37:34.879 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-07-18T03:37:34.978 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-07-18T03:37:35.012 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-07-18T03:37:35.108 INFO:teuthology.orchestra.run.smithi107.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-07-18T03:37:35.142 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-07-18T03:37:35.238 INFO:teuthology.orchestra.run.smithi107.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-07-18T03:37:35.276 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-07-18T03:37:35.277 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi107.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-18T03:37:35.277 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-07-18T03:37:35.348 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-18T03:37:35.348 INFO:tasks.ceph:role: osd.0 2023-07-18T03:37:35.348 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:37:35.349 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-18T03:37:35.419 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-18T03:37:35.420 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-18T03:37:35.420 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-18T03:37:35.420 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-18T03:37:35.420 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-18T03:37:35.420 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2023-07-18T03:37:35.420 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-18T03:37:35.421 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-18T03:37:35.421 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-18T03:37:35.421 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-18T03:37:35.444 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2023-07-18T03:37:35.448 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2023-07-18T03:37:35.448 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-07-18T03:37:35.557 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-07-18T03:37:35.637 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-07-18T03:37:35.707 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-18T03:37:35.707 INFO:tasks.ceph:role: osd.1 2023-07-18T03:37:35.708 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:37:35.708 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-18T03:37:35.778 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-18T03:37:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-18T03:37:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-18T03:37:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-18T03:37:35.779 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-18T03:37:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2023-07-18T03:37:35.779 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-18T03:37:35.779 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-18T03:37:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-18T03:37:35.780 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-18T03:37:35.802 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2023-07-18T03:37:35.806 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2023-07-18T03:37:35.807 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-07-18T03:37:35.914 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-07-18T03:37:35.991 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-07-18T03:37:36.062 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-18T03:37:36.063 INFO:tasks.ceph:role: osd.2 2023-07-18T03:37:36.063 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:37:36.063 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-18T03:37:36.133 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-18T03:37:36.133 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-18T03:37:36.133 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-18T03:37:36.133 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-18T03:37:36.133 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-18T03:37:36.134 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2023-07-18T03:37:36.134 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-18T03:37:36.134 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-18T03:37:36.134 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-18T03:37:36.134 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-18T03:37:36.155 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2023-07-18T03:37:36.159 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2023-07-18T03:37:36.159 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-07-18T03:37:36.270 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-07-18T03:37:36.346 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-07-18T03:37:36.417 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-18T03:37:36.417 INFO:tasks.ceph:role: osd.3 2023-07-18T03:37:36.418 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:37:36.418 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-18T03:37:36.487 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-18T03:37:36.488 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-18T03:37:36.488 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-18T03:37:36.488 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-18T03:37:36.488 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-18T03:37:36.489 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2023-07-18T03:37:36.489 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-18T03:37:36.489 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-18T03:37:36.489 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-18T03:37:36.490 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-18T03:37:36.511 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2023-07-18T03:37:36.515 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2023-07-18T03:37:36.516 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-07-18T03:37:36.623 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-07-18T03:37:36.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:37:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:36.839+0000 7fcdcd7c4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-07-18T03:37:36.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:36.840+0000 7fcdcd7c4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-07-18T03:37:36.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:36.840+0000 7fcdcd7c4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-18T03:37:36.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:36.840+0000 7fcdcd7c4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-18T03:37:36.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:36.840+0000 7fcdcd7c4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-18T03:37:36.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:36.842+0000 7fcdcd7c4640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-07-18T03:37:39.722 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-07-18T03:37:39.799 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:37:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:39.913+0000 7f2870355640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-07-18T03:37:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:39.914+0000 7f2870355640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-07-18T03:37:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:39.914+0000 7f2870355640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-18T03:37:39.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:39.914+0000 7f2870355640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-18T03:37:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:39.914+0000 7f2870355640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-18T03:37:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:39.915+0000 7f2870355640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-07-18T03:37:42.799 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-07-18T03:37:42.875 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:37:42.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:42.990+0000 7efe2bb40640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-07-18T03:37:42.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:42.990+0000 7efe2bb40640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-07-18T03:37:42.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:42.990+0000 7efe2bb40640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-18T03:37:42.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:42.991+0000 7efe2bb40640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-18T03:37:42.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:42.991+0000 7efe2bb40640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-18T03:37:42.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:42.992+0000 7efe2bb40640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-07-18T03:37:45.866 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-07-18T03:37:45.942 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:37:46.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:46.057+0000 7f77915ec640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-07-18T03:37:46.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:46.057+0000 7f77915ec640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-07-18T03:37:46.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:46.057+0000 7f77915ec640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-18T03:37:46.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:46.057+0000 7f77915ec640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-18T03:37:46.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:46.057+0000 7f77915ec640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-18T03:37:46.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:37:46.059+0000 7f77915ec640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-07-18T03:37:48.932 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-07-18T03:37:49.006 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-07-18T03:37:49.078 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-18T03:37:49.078 INFO:tasks.ceph:role: osd.4 2023-07-18T03:37:49.078 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:37:49.079 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-18T03:37:49.155 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-18T03:37:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-18T03:37:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-18T03:37:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-18T03:37:49.156 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-18T03:37:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2023-07-18T03:37:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-18T03:37:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-18T03:37:49.157 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-18T03:37:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-18T03:37:49.583 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2023-07-18T03:37:49.587 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2023-07-18T03:37:49.588 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-07-18T03:37:49.777 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-07-18T03:37:49.854 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-07-18T03:37:49.920 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-18T03:37:49.920 INFO:tasks.ceph:role: osd.5 2023-07-18T03:37:49.921 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:37:49.921 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-18T03:37:49.994 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-18T03:37:49.994 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-18T03:37:49.995 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-18T03:37:49.995 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-18T03:37:49.995 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-18T03:37:49.995 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2023-07-18T03:37:49.995 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-18T03:37:49.995 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-18T03:37:49.995 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-18T03:37:49.996 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-18T03:37:50.400 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2023-07-18T03:37:50.405 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2023-07-18T03:37:50.405 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-07-18T03:37:50.545 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-07-18T03:37:50.620 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-07-18T03:37:50.689 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-18T03:37:50.690 INFO:tasks.ceph:role: osd.6 2023-07-18T03:37:50.690 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:37:50.690 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-18T03:37:50.765 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-18T03:37:50.766 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-18T03:37:50.766 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-18T03:37:50.766 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-18T03:37:50.766 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-18T03:37:50.766 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2023-07-18T03:37:50.766 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-18T03:37:50.767 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-18T03:37:50.767 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-18T03:37:50.767 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-18T03:37:51.209 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2023-07-18T03:37:51.213 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2023-07-18T03:37:51.213 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-07-18T03:37:51.382 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-07-18T03:37:51.458 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-07-18T03:37:51.528 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-18T03:37:51.528 INFO:tasks.ceph:role: osd.7 2023-07-18T03:37:51.528 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:37:51.529 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-18T03:37:51.605 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-18T03:37:51.605 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-18T03:37:51.605 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-18T03:37:51.606 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-18T03:37:51.606 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-18T03:37:51.606 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2023-07-18T03:37:51.606 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-18T03:37:51.606 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-18T03:37:51.606 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-18T03:37:51.606 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-18T03:37:52.045 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2023-07-18T03:37:52.049 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2023-07-18T03:37:52.049 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-07-18T03:37:52.190 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-07-18T03:37:52.274 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:37:52.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:52.434+0000 7fab6f8cb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-07-18T03:37:52.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:52.434+0000 7fab6f8cb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-07-18T03:37:52.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:52.434+0000 7fab6f8cb640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-18T03:37:52.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:52.434+0000 7fab6f8cb640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-18T03:37:52.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:52.435+0000 7fab6f8cb640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-18T03:37:52.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:52.436+0000 7fab6f8cb640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-07-18T03:37:55.365 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-07-18T03:37:55.435 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:37:55.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:55.545+0000 7fb026ef5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-07-18T03:37:55.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:55.546+0000 7fb026ef5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-07-18T03:37:55.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:55.546+0000 7fb026ef5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-18T03:37:55.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:55.546+0000 7fb026ef5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-18T03:37:55.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:55.546+0000 7fb026ef5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-18T03:37:55.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:55.548+0000 7fb026ef5640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-07-18T03:37:58.496 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-07-18T03:37:58.568 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:37:58.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:58.679+0000 7f9d65161640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-07-18T03:37:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:58.679+0000 7f9d65161640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-07-18T03:37:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:58.679+0000 7f9d65161640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-18T03:37:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:58.679+0000 7f9d65161640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-18T03:37:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:58.679+0000 7f9d65161640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-18T03:37:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:37:58.681+0000 7f9d65161640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-07-18T03:38:00.039 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:38:00.044 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:38:01.666 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-07-18T03:38:01.735 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:38:01.841 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:01.844+0000 7fa563a4a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-07-18T03:38:01.841 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:01.845+0000 7fa563a4a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-07-18T03:38:01.841 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:01.845+0000 7fa563a4a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-18T03:38:01.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:01.845+0000 7fa563a4a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-18T03:38:01.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:01.845+0000 7fa563a4a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-18T03:38:01.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:01.846+0000 7fa563a4a640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-07-18T03:38:04.777 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-07-18T03:38:04.850 INFO:tasks.ceph:Reading keys from all nodes... 2023-07-18T03:38:04.851 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:04.851 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-07-18T03:38:04.929 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:04.929 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-07-18T03:38:05.041 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:05.041 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-07-18T03:38:05.151 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:05.152 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-07-18T03:38:05.262 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:05.262 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-07-18T03:38:05.371 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:05.371 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-07-18T03:38:05.481 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:05.482 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-07-18T03:38:05.590 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:05.590 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-07-18T03:38:05.659 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:05.659 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-07-18T03:38:05.767 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:05.768 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-07-18T03:38:05.875 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:05.876 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-07-18T03:38:05.986 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:05.986 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-07-18T03:38:06.097 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:06.098 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-07-18T03:38:06.208 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:06.208 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-07-18T03:38:06.319 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:06.320 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-07-18T03:38:06.366 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:06.366 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-07-18T03:38:06.450 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:06.451 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-07-18T03:38:06.494 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:06.494 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-07-18T03:38:06.577 INFO:tasks.ceph:Adding keys to all mons... 2023-07-18T03:38:06.577 DEBUG:teuthology.orchestra.run.smithi049:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-18T03:38:06.582 DEBUG:teuthology.orchestra.run.smithi107:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-18T03:38:06.646 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.y] 2023-07-18T03:38:06.646 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQD9CLZkATFALhAAqRgQzaCzvqgqsvTwhZ/+rw== 2023-07-18T03:38:06.647 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2023-07-18T03:38:06.647 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQD+CLZk0A9wAxAAPyxogWvN53TNCPyBYAgGgQ== 2023-07-18T03:38:06.647 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2023-07-18T03:38:06.647 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQD+CLZkCKiYDxAAhmsGhQF1Uusa7eSwZKWIWw== 2023-07-18T03:38:06.648 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.0] 2023-07-18T03:38:06.648 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAACbZkV1wVMhAAqSqjoEtRctkKCcFiUUOTcw== 2023-07-18T03:38:06.648 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.1] 2023-07-18T03:38:06.649 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQADCbZk9S6TNhAAOl9IxSR58LnxX58i/sjMvg== 2023-07-18T03:38:06.649 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.2] 2023-07-18T03:38:06.649 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAGCbZke4glOxAAASCGrrHrKsam5fv4gjdZ/A== 2023-07-18T03:38:06.649 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.3] 2023-07-18T03:38:06.650 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAKCbZkP4iFAxAA/nPkDfrSOZeJm2QyEAGNiQ== 2023-07-18T03:38:06.650 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.x] 2023-07-18T03:38:06.650 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQD9CLZk6hvxNhAAMALkYp+kAehBGNNCw/BJfg== 2023-07-18T03:38:06.651 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2023-07-18T03:38:06.651 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQD+CLZkWB96GxAAepmZ6nF9qh75BBmZTY1W4w== 2023-07-18T03:38:06.651 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2023-07-18T03:38:06.651 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQD+CLZkKJwHKBAAusYdoX3eXJdWmNNsa7RuZA== 2023-07-18T03:38:06.651 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.4] 2023-07-18T03:38:06.652 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAQCbZkxYX2GRAA1Y4sxGA6H05aa5jydM38WQ== 2023-07-18T03:38:06.652 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.5] 2023-07-18T03:38:06.652 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQATCbZkt8qVIBAAeaCS5rmMJEgwZUjh3siVTw== 2023-07-18T03:38:06.653 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.6] 2023-07-18T03:38:06.653 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAWCbZkK/qRKBAA7wpJMzxg/K+YMxEJHUeRPw== 2023-07-18T03:38:06.653 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.7] 2023-07-18T03:38:06.653 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAZCbZk2ltrMhAATvhB8r+uw5+LDGlMtJzEbw== 2023-07-18T03:38:06.654 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2023-07-18T03:38:06.654 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQD+CLZkI5fMMhAAisfdJO7kEPCDAngS1ywP+g== 2023-07-18T03:38:06.654 INFO:teuthology.orchestra.run.smithi049.stdout:[client.2] 2023-07-18T03:38:06.654 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQD+CLZk1mmtOhAAYC8ALN9saaob9naZYRC6Vg== 2023-07-18T03:38:06.655 INFO:teuthology.orchestra.run.smithi049.stdout:[client.1] 2023-07-18T03:38:06.655 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQD/CLZkgQfPBhAAIkgoL4P4YFEqTZqP/8hYHQ== 2023-07-18T03:38:06.655 INFO:teuthology.orchestra.run.smithi049.stdout:[client.3] 2023-07-18T03:38:06.656 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQD/CLZk1OaMDhAARMs8iCxSvTEiQomODnCtIg== 2023-07-18T03:38:06.682 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr.y] 2023-07-18T03:38:06.682 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD9CLZkATFALhAAqRgQzaCzvqgqsvTwhZ/+rw== 2023-07-18T03:38:06.682 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a] 2023-07-18T03:38:06.683 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD+CLZk0A9wAxAAPyxogWvN53TNCPyBYAgGgQ== 2023-07-18T03:38:06.683 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c] 2023-07-18T03:38:06.683 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD+CLZkCKiYDxAAhmsGhQF1Uusa7eSwZKWIWw== 2023-07-18T03:38:06.683 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.0] 2023-07-18T03:38:06.683 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAACbZkV1wVMhAAqSqjoEtRctkKCcFiUUOTcw== 2023-07-18T03:38:06.683 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.1] 2023-07-18T03:38:06.683 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQADCbZk9S6TNhAAOl9IxSR58LnxX58i/sjMvg== 2023-07-18T03:38:06.684 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.2] 2023-07-18T03:38:06.684 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAGCbZke4glOxAAASCGrrHrKsam5fv4gjdZ/A== 2023-07-18T03:38:06.684 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.3] 2023-07-18T03:38:06.684 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAKCbZkP4iFAxAA/nPkDfrSOZeJm2QyEAGNiQ== 2023-07-18T03:38:06.685 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr.x] 2023-07-18T03:38:06.685 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD9CLZk6hvxNhAAMALkYp+kAehBGNNCw/BJfg== 2023-07-18T03:38:06.685 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b] 2023-07-18T03:38:06.685 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD+CLZkWB96GxAAepmZ6nF9qh75BBmZTY1W4w== 2023-07-18T03:38:06.685 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d] 2023-07-18T03:38:06.685 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD+CLZkKJwHKBAAusYdoX3eXJdWmNNsa7RuZA== 2023-07-18T03:38:06.685 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.4] 2023-07-18T03:38:06.685 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAQCbZkxYX2GRAA1Y4sxGA6H05aa5jydM38WQ== 2023-07-18T03:38:06.686 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.5] 2023-07-18T03:38:06.686 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQATCbZkt8qVIBAAeaCS5rmMJEgwZUjh3siVTw== 2023-07-18T03:38:06.686 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.6] 2023-07-18T03:38:06.686 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAWCbZkK/qRKBAA7wpJMzxg/K+YMxEJHUeRPw== 2023-07-18T03:38:06.686 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.7] 2023-07-18T03:38:06.686 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAZCbZk2ltrMhAATvhB8r+uw5+LDGlMtJzEbw== 2023-07-18T03:38:06.686 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2023-07-18T03:38:06.687 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD+CLZkI5fMMhAAisfdJO7kEPCDAngS1ywP+g== 2023-07-18T03:38:06.687 INFO:teuthology.orchestra.run.smithi107.stdout:[client.2] 2023-07-18T03:38:06.687 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD+CLZk1mmtOhAAYC8ALN9saaob9naZYRC6Vg== 2023-07-18T03:38:06.687 INFO:teuthology.orchestra.run.smithi107.stdout:[client.1] 2023-07-18T03:38:06.687 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD/CLZkgQfPBhAAIkgoL4P4YFEqTZqP/8hYHQ== 2023-07-18T03:38:06.687 INFO:teuthology.orchestra.run.smithi107.stdout:[client.3] 2023-07-18T03:38:06.687 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD/CLZk1OaMDhAARMs8iCxSvTEiQomODnCtIg== 2023-07-18T03:38:06.688 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-18T03:38:06.701 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-18T03:38:06.843 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-18T03:38:06.847 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-18T03:38:06.960 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-18T03:38:06.965 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-18T03:38:07.079 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:07.084 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:07.193 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:07.197 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:07.307 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:07.312 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:07.420 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:07.425 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:07.537 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-18T03:38:07.541 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-18T03:38:07.652 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-18T03:38:07.656 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-18T03:38:07.770 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-18T03:38:07.774 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-18T03:38:07.887 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:07.892 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:08.002 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:08.006 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:08.115 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:08.120 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:08.229 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:08.233 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-18T03:38:08.343 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-18T03:38:08.348 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-18T03:38:08.463 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-18T03:38:08.467 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-18T03:38:08.579 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-18T03:38:08.584 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-18T03:38:08.695 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-18T03:38:08.700 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-18T03:38:08.812 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-07-18T03:38:08.813 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-07-18T03:38:08.882 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-18T03:38:09.009 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-07-18T03:38:09.080 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-07-18T03:38:09.149 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-18T03:38:09.275 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-07-18T03:38:09.346 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-07-18T03:38:09.416 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-18T03:38:09.538 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-07-18T03:38:09.608 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:38:09.613 DEBUG:teuthology.orchestra.run.smithi107:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-18T03:38:09.656 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-07-18T03:38:09.657 INFO:tasks.ceph.mon.a:Restarting daemon 2023-07-18T03:38:09.658 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-07-18T03:38:09.698 INFO:tasks.ceph.mon.a:Started 2023-07-18T03:38:09.699 INFO:tasks.ceph.mon.b:Restarting daemon 2023-07-18T03:38:09.699 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-07-18T03:38:09.704 INFO:tasks.ceph.mon.b:Started 2023-07-18T03:38:09.705 INFO:tasks.ceph.mon.c:Restarting daemon 2023-07-18T03:38:09.705 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-07-18T03:38:09.710 INFO:tasks.ceph.mon.c:Started 2023-07-18T03:38:09.710 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-07-18T03:38:09.711 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-07-18T03:38:09.711 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-07-18T03:38:09.715 INFO:tasks.ceph.mgr.y:Started 2023-07-18T03:38:09.716 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-07-18T03:38:09.716 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-07-18T03:38:09.720 INFO:tasks.ceph.mgr.x:Started 2023-07-18T03:38:09.720 INFO:tasks.ceph:Setting crush tunables to default 2023-07-18T03:38:09.721 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd crush tunables default 2023-07-18T03:38:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.930+0000 7efc79fff700 1 Processor -- start 2023-07-18T03:38:09.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.930+0000 7efc79fff700 1 -- start start 2023-07-18T03:38:09.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.931+0000 7efc79fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:09.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.931+0000 7efc79fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:09.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.931+0000 7efc79fff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc740b8e30 con 0x7efc740d1050 2023-07-18T03:38:09.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.931+0000 7efc79fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc740d14d0 con 0x7efc740b8a60 2023-07-18T03:38:09.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.931+0000 7efc79fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc740b1e30 con 0x7efc740b4530 2023-07-18T03:38:09.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.931+0000 7efc73fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7efc740d1050 0x7efc740b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56888/0 (socket says 172.21.15.49:56888) 2023-07-18T03:38:09.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.931+0000 7efc73fff700 1 -- 172.21.15.49:0/3541418725 learned_addr learned my addr 172.21.15.49:0/3541418725 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:09.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.931+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:09.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.932+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:09.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.932+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 msgr2=0x7efc740b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-18T03:38:09.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.932+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 msgr2=0x7efc740b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-18T03:38:09.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.932+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:38:09.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.932+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:38:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.932+0000 7efc6bfff700 1 -- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 msgr2=0x7efc740d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-07-18T03:38:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.932+0000 7efc6bfff700 1 -- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 msgr2=0x7efc740d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-18T03:38:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.932+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:38:09.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:09.932+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:38:10.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.133+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:10.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.133+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:10.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.133+0000 7efc6bfff700 1 -- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 msgr2=0x7efc740d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-18T03:38:10.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.133+0000 7efc6bfff700 1 -- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 msgr2=0x7efc740d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-18T03:38:10.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.133+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:38:10.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.133+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 msgr2=0x7efc740b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-07-18T03:38:10.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.133+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 msgr2=0x7efc740b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-18T03:38:10.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.133+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:38:10.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.133+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T03:38:10.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.133+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T03:38:10.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.534+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:10.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.534+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:10.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.534+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 msgr2=0x7efc740b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-07-18T03:38:10.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.534+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 msgr2=0x7efc740b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-18T03:38:10.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.534+0000 7efc6bfff700 1 -- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 msgr2=0x7efc740d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-18T03:38:10.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.534+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:38:10.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.534+0000 7efc6bfff700 1 -- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 msgr2=0x7efc740d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-18T03:38:10.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.534+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:38:10.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.534+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-18T03:38:10.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:10.534+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-18T03:38:11.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:11.336+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:11.336+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:11.336+0000 7efc6bfff700 1 -- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 msgr2=0x7efc740d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-18T03:38:11.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:11.336+0000 7efc6bfff700 1 -- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 msgr2=0x7efc740d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-18T03:38:11.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:11.336+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:38:11.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:11.336+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 msgr2=0x7efc740b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-07-18T03:38:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:11.336+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 msgr2=0x7efc740b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-18T03:38:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:11.336+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:38:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:11.336+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-18T03:38:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:11.336+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-18T03:38:12.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc71ffb700 1 -- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 msgr2=0x7efc740d0b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:38:12.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc71ffb700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc740b4530 0x7efc740d0b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:12.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc71ffb700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 msgr2=0x7efc740b3ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:38:12.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc71ffb700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.107:3300/0 conn(0x7efc740b8a60 0x7efc740b3ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:12.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc71ffb700 1 -- 172.21.15.49:0/3541418725 >> v1:172.21.15.49:6789/0 conn(0x7efc740d1050 legacy=0x7efc740b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:12.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc71ffb700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc71ffb700 1 -- 172.21.15.49:0/3541418725 --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc740afaa0 con 0x7efc54006280 2023-07-18T03:38:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc71ffb700 1 -- 172.21.15.49:0/3541418725 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc740d14d0 con 0x7efc54003770 2023-07-18T03:38:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 msgr2=0x7efc54005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-18T03:38:12.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 msgr2=0x7efc54005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-18T03:38:12.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:38:12.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:12.932+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:38:13.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:13.133+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:13.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:13.133+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 msgr2=0x7efc54005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-18T03:38:13.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:13.133+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 msgr2=0x7efc54005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-18T03:38:13.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:13.133+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:38:13.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:13.133+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T03:38:13.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:13.534+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:13.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:13.534+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 msgr2=0x7efc54005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-18T03:38:13.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:13.534+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 msgr2=0x7efc54005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-18T03:38:13.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:13.534+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:38:13.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:13.535+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-18T03:38:14.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:14.336+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:14.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:14.336+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 msgr2=0x7efc54005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-18T03:38:14.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:14.336+0000 7efc737fe700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 msgr2=0x7efc54005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-18T03:38:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:14.336+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:38:14.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:14.336+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-18T03:38:15.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.133+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225044440 0 0) 0x7efc740afaa0 con 0x7efc54006280 2023-07-18T03:38:15.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.133+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc600035b0 con 0x7efc54006280 2023-07-18T03:38:15.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3844732617 0 0) 0x7efc600035b0 con 0x7efc54006280 2023-07-18T03:38:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 msgr2=0x7efc54005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:38:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc72ffd700 1 --2- 172.21.15.49:0/3541418725 >> v2:172.21.15.49:3300/0 conn(0x7efc54003770 0x7efc54005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5400b010 con 0x7efc54006280 2023-07-18T03:38:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7efc5c00e590 con 0x7efc54006280 2023-07-18T03:38:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 >> v1:172.21.15.107:6789/0 conn(0x7efc54006280 legacy=0x7efc54008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc73fff700 1 -- 172.21.15.49:0/3541418725 reap_dead start 2023-07-18T03:38:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc72ffd700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc60003a50 0x7efc60003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc72ffd700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc60004850 0x7efc60004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc72ffd700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc60005400 0x7efc6000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc740d14d0 con 0x7efc60004850 2023-07-18T03:38:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6000c2a0 con 0x7efc60003a50 2023-07-18T03:38:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.138+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6000c460 con 0x7efc60005400 2023-07-18T03:38:15.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.139+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc60004850 0x7efc60004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.139+0000 7efc737fe700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc60003a50 0x7efc60003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.139+0000 7efc73fff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc60005400 0x7efc6000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.139+0000 7efc6bfff700 1 -- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc60005400 msgr2=0x7efc6000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.139+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc60005400 0x7efc6000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.139+0000 7efc6bfff700 1 -- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc60003a50 msgr2=0x7efc60003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.139+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc60003a50 0x7efc60003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.139+0000 7efc6bfff700 1 -- 172.21.15.49:0/3541418725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc600035b0 con 0x7efc60004850 2023-07-18T03:38:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.140+0000 7efc6bfff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc60004850 0x7efc60004cc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009100 tx=0x7efc60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6ba2eb6b7f5f38 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.140+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c014ba0 con 0x7efc60004850 2023-07-18T03:38:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.140+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5c00c070 con 0x7efc60004850 2023-07-18T03:38:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.140+0000 7efc72ffd700 1 -- 172.21.15.49:0/3541418725 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c017470 con 0x7efc60004850 2023-07-18T03:38:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.141+0000 7efc79fff700 1 -- 172.21.15.49:0/3541418725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc60004850 msgr2=0x7efc60004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.141+0000 7efc79fff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc60004850 0x7efc60004cc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009100 tx=0x7efc60005880 comp rx=0 tx=0).stop 2023-07-18T03:38:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.141+0000 7efc79fff700 1 -- 172.21.15.49:0/3541418725 shutdown_connections 2023-07-18T03:38:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.141+0000 7efc79fff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc60005400 0x7efc6000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.141+0000 7efc79fff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc60004850 0x7efc60004cc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.141+0000 7efc79fff700 1 --2- 172.21.15.49:0/3541418725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc60003a50 0x7efc60003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.141+0000 7efc79fff700 1 -- 172.21.15.49:0/3541418725 >> 172.21.15.49:0/3541418725 conn(0x7efc7409c1b0 msgr2=0x7efc740d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.141+0000 7efc79fff700 1 -- 172.21.15.49:0/3541418725 shutdown_connections 2023-07-18T03:38:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.141+0000 7efc79fff700 1 -- 172.21.15.49:0/3541418725 wait complete. 2023-07-18T03:38:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.143+0000 7efc79fff700 1 Processor -- start 2023-07-18T03:38:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.144+0000 7efc79fff700 1 -- start start 2023-07-18T03:38:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.145+0000 7efc79fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc60003a50 0x7efc7415a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.145+0000 7efc79fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc60004850 0x7efc7415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.145+0000 7efc79fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc60005400 0x7efc7415e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.145+0000 7efc79fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc741f1b70 con 0x7efc60003a50 2023-07-18T03:38:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.145+0000 7efc79fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc740a0c50 con 0x7efc60005400 2023-07-18T03:38:15.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.145+0000 7efc79fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc740ce6b0 con 0x7efc60004850 2023-07-18T03:38:15.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.145+0000 7efc73fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc60005400 0x7efc7415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:15.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.145+0000 7efc73fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc60005400 0x7efc7415e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:40562/0 (socket says 172.21.15.49:40562) 2023-07-18T03:38:15.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.145+0000 7efc73fff700 1 -- 172.21.15.49:0/1669748027 learned_addr learned my addr 172.21.15.49:0/1669748027 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:15.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.145+0000 7efc6bfff700 1 --2- 172.21.15.49:0/1669748027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc60004850 0x7efc7415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:15.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.145+0000 7efc737fe700 1 --2- 172.21.15.49:0/1669748027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc60003a50 0x7efc7415a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:15.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.146+0000 7efc73fff700 1 -- 172.21.15.49:0/1669748027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc60004850 msgr2=0x7efc7415e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:15.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.146+0000 7efc73fff700 1 --2- 172.21.15.49:0/1669748027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc60004850 0x7efc7415e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.146+0000 7efc73fff700 1 -- 172.21.15.49:0/1669748027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc60003a50 msgr2=0x7efc7415a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:15.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.146+0000 7efc73fff700 1 --2- 172.21.15.49:0/1669748027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc60003a50 0x7efc7415a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.146+0000 7efc73fff700 1 -- 172.21.15.49:0/1669748027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc740cab50 con 0x7efc60005400 2023-07-18T03:38:15.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.146+0000 7efc73fff700 1 --2- 172.21.15.49:0/1669748027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc60005400 0x7efc7415e750 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efc54009ed0 tx=0x7efc6c00f4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:15.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.146+0000 7efc70ff9700 1 -- 172.21.15.49:0/1669748027 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c001c20 con 0x7efc60005400 2023-07-18T03:38:15.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.146+0000 7efc70ff9700 1 -- 172.21.15.49:0/1669748027 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6c007b50 con 0x7efc60005400 2023-07-18T03:38:15.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.146+0000 7efc70ff9700 1 -- 172.21.15.49:0/1669748027 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c00fe10 con 0x7efc60005400 2023-07-18T03:38:15.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.146+0000 7efc79fff700 1 -- 172.21.15.49:0/1669748027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc740cae30 con 0x7efc60005400 2023-07-18T03:38:15.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.147+0000 7efc79fff700 1 -- 172.21.15.49:0/1669748027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc740cd1a0 con 0x7efc60005400 2023-07-18T03:38:15.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.147+0000 7efc70ff9700 1 -- 172.21.15.49:0/1669748027 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efc6c01d460 con 0x7efc60005400 2023-07-18T03:38:15.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.147+0000 7efc70ff9700 1 -- 172.21.15.49:0/1669748027 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efc6c013780 con 0x7efc60005400 2023-07-18T03:38:15.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.148+0000 7efc79fff700 1 -- 172.21.15.49:0/1669748027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc740ce880 con 0x7efc60005400 2023-07-18T03:38:15.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.151+0000 7efc70ff9700 1 -- 172.21.15.49:0/1669748027 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7efc6c013940 con 0x7efc60005400 2023-07-18T03:38:15.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.235+0000 7efc79fff700 1 -- 172.21.15.49:0/1669748027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7efc740b34b0 con 0x7efc60005400 2023-07-18T03:38:15.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.240+0000 7efc70ff9700 1 -- 172.21.15.49:0/1669748027 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7efc6c013ae0 con 0x7efc60005400 2023-07-18T03:38:15.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.241+0000 7efc79fff700 1 -- 172.21.15.49:0/1669748027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc60005400 msgr2=0x7efc7415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:15.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.241+0000 7efc79fff700 1 --2- 172.21.15.49:0/1669748027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc60005400 0x7efc7415e750 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efc54009ed0 tx=0x7efc6c00f4b0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.241+0000 7efc79fff700 1 -- 172.21.15.49:0/1669748027 shutdown_connections 2023-07-18T03:38:15.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.241+0000 7efc79fff700 1 --2- 172.21.15.49:0/1669748027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc60005400 0x7efc7415e750 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.241+0000 7efc79fff700 1 --2- 172.21.15.49:0/1669748027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc60003a50 0x7efc7415a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.241+0000 7efc79fff700 1 --2- 172.21.15.49:0/1669748027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc60004850 0x7efc7415e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.241+0000 7efc79fff700 1 -- 172.21.15.49:0/1669748027 >> 172.21.15.49:0/1669748027 conn(0x7efc7409c1b0 msgr2=0x7efc740b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:15.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.241+0000 7efc79fff700 1 -- 172.21.15.49:0/1669748027 shutdown_connections 2023-07-18T03:38:15.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:15.241+0000 7efc79fff700 1 -- 172.21.15.49:0/1669748027 wait complete. 2023-07-18T03:38:15.238 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2023-07-18T03:38:15.254 INFO:tasks.ceph:check_enable_crimson: False 2023-07-18T03:38:15.255 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-07-18T03:38:15.255 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:15.255 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-07-18T03:38:15.288 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:15.293+0000 7f465a047200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T03:38:15.325 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:15.326 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-07-18T03:38:15.372 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:15.376+0000 7f465a047200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T03:38:15.436 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:15.436 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-07-18T03:38:15.466 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:15.471+0000 7f465a047200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T03:38:15.545 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:15.546 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-07-18T03:38:15.563 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:15.567+0000 7f465a047200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T03:38:15.656 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:15.657 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-07-18T03:38:15.725 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:15.726 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-07-18T03:38:15.745 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:15.750+0000 7f465a047200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T03:38:15.814 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:15.814 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-07-18T03:38:15.923 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:38:15.923 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-07-18T03:38:16.024 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:16.029+0000 7f2738a92200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T03:38:16.029 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new a464d08b-7474-4b14-a691-ad484e4a088d 0 2023-07-18T03:38:16.114 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:16.119+0000 7f2738a92200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T03:38:16.180 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:16.185+0000 7f465a047200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T03:38:16.213 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:16.218+0000 7f2738a92200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T03:38:16.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.264+0000 7f01eebff700 1 Processor -- start 2023-07-18T03:38:16.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.265+0000 7f01eebff700 1 -- start start 2023-07-18T03:38:16.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01eebff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f01e80b8a60 0x7f01e80b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01eebff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f01e80d1220 0x7f01e80b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01eebff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e80b8e30 con 0x7f01e80b4530 2023-07-18T03:38:16.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01eebff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e80afaa0 con 0x7f01e80d1220 2023-07-18T03:38:16.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01eebff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e80afd80 con 0x7f01e80b8a60 2023-07-18T03:38:16.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01ec99b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f01e80b8a60 0x7f01e80b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01ec99b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f01e80b8a60 0x7f01e80b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47482/0 (socket says 172.21.15.107:47482) 2023-07-18T03:38:16.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01ec99b700 1 -- 172.21.15.107:0/1602405367 learned_addr learned my addr 172.21.15.107:0/1602405367 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:16.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01ed19c700 1 --2- 172.21.15.107:0/1602405367 >> v2:172.21.15.49:3300/0 conn(0x7f01e80d1220 0x7f01e80b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01ec99b700 1 -- 172.21.15.107:0/1602405367 >> v2:172.21.15.49:3300/0 conn(0x7f01e80d1220 msgr2=0x7f01e80b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:16.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01ec99b700 1 --2- 172.21.15.107:0/1602405367 >> v2:172.21.15.49:3300/0 conn(0x7f01e80d1220 0x7f01e80b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01ec99b700 1 -- 172.21.15.107:0/1602405367 >> v1:172.21.15.49:6789/0 conn(0x7f01e80b4530 legacy=0x7f01e80d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.266+0000 7f01ec99b700 1 -- 172.21.15.107:0/1602405367 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e80b0060 con 0x7f01e80b8a60 2023-07-18T03:38:16.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.267+0000 7f01ec99b700 1 --2- 172.21.15.107:0/1602405367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e80b8a60 0x7f01e80b3ff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01e80b5b00 tx=0x7f01dc009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54f04f91062f90a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:16.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.267+0000 7f01e77fe700 1 -- 172.21.15.107:0/1602405367 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc011830 con 0x7f01e80b8a60 2023-07-18T03:38:16.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.267+0000 7f01e77fe700 1 -- 172.21.15.107:0/1602405367 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc011990 con 0x7f01e80b8a60 2023-07-18T03:38:16.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.267+0000 7f01e77fe700 1 -- 172.21.15.107:0/1602405367 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc011d20 con 0x7f01e80b8a60 2023-07-18T03:38:16.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.268+0000 7f01eebff700 1 -- 172.21.15.107:0/1602405367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e80b8a60 msgr2=0x7f01e80b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.268+0000 7f01eebff700 1 --2- 172.21.15.107:0/1602405367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e80b8a60 0x7f01e80b3ff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01e80b5b00 tx=0x7f01dc009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.268+0000 7f01eebff700 1 -- 172.21.15.107:0/1602405367 shutdown_connections 2023-07-18T03:38:16.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.268+0000 7f01eebff700 1 --2- 172.21.15.107:0/1602405367 >> v2:172.21.15.49:3300/0 conn(0x7f01e80d1220 0x7f01e80b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.268+0000 7f01eebff700 1 --2- 172.21.15.107:0/1602405367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e80b8a60 0x7f01e80b3ff0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.268+0000 7f01eebff700 1 -- 172.21.15.107:0/1602405367 >> 172.21.15.107:0/1602405367 conn(0x7f01e809c1b0 msgr2=0x7f01e80d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:16.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.268+0000 7f01eebff700 1 -- 172.21.15.107:0/1602405367 shutdown_connections 2023-07-18T03:38:16.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.268+0000 7f01eebff700 1 -- 172.21.15.107:0/1602405367 wait complete. 2023-07-18T03:38:16.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.271+0000 7f01eebff700 1 Processor -- start 2023-07-18T03:38:16.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.271+0000 7f01eebff700 1 -- start start 2023-07-18T03:38:16.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01eebff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e80b4530 0x7f01e8152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01eebff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01e80b8a60 0x7f01e81f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01eebff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01e80d1220 0x7f01e81f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01eebff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e8155250 con 0x7f01e80b8a60 2023-07-18T03:38:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01eebff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81554c0 con 0x7f01e80b4530 2023-07-18T03:38:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01eebff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81557a0 con 0x7f01e80d1220 2023-07-18T03:38:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01ec99b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e80b4530 0x7f01e8152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01ec99b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e80b4530 0x7f01e8152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47492/0 (socket says 172.21.15.107:47492) 2023-07-18T03:38:16.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01ec99b700 1 -- 172.21.15.107:0/612044938 learned_addr learned my addr 172.21.15.107:0/612044938 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:16.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01e7fff700 1 --2- 172.21.15.107:0/612044938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01e80b8a60 0x7f01e81f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01ed19c700 1 --2- 172.21.15.107:0/612044938 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01e80d1220 0x7f01e81f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01e7fff700 1 -- 172.21.15.107:0/612044938 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01e80d1220 msgr2=0x7f01e81f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01e7fff700 1 --2- 172.21.15.107:0/612044938 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01e80d1220 0x7f01e81f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01e7fff700 1 -- 172.21.15.107:0/612044938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e80b4530 msgr2=0x7f01e8152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01e7fff700 1 --2- 172.21.15.107:0/612044938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e80b4530 0x7f01e8152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01e7fff700 1 -- 172.21.15.107:0/612044938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc009550 con 0x7f01e80b8a60 2023-07-18T03:38:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.273+0000 7f01ed19c700 1 --2- 172.21.15.107:0/612044938 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01e80d1220 0x7f01e81f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.274+0000 7f01e7fff700 1 --2- 172.21.15.107:0/612044938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01e80b8a60 0x7f01e81f5b70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01d4000f60 tx=0x7f01d400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.274+0000 7f01e57fa700 1 -- 172.21.15.107:0/612044938 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40052d0 con 0x7f01e80b8a60 2023-07-18T03:38:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.274+0000 7f01e57fa700 1 -- 172.21.15.107:0/612044938 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d4005430 con 0x7f01e80b8a60 2023-07-18T03:38:16.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.274+0000 7f01eebff700 1 -- 172.21.15.107:0/612044938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e80b5ef0 con 0x7f01e80b8a60 2023-07-18T03:38:16.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.274+0000 7f01e57fa700 1 -- 172.21.15.107:0/612044938 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40038c0 con 0x7f01e80b8a60 2023-07-18T03:38:16.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.274+0000 7f01eebff700 1 -- 172.21.15.107:0/612044938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e81530c0 con 0x7f01e80b8a60 2023-07-18T03:38:16.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.274+0000 7f01e57fa700 1 -- 172.21.15.107:0/612044938 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f01d4003d20 con 0x7f01e80b8a60 2023-07-18T03:38:16.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.274+0000 7f01e57fa700 1 -- 172.21.15.107:0/612044938 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f01d401c780 con 0x7f01e80b8a60 2023-07-18T03:38:16.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.275+0000 7f01eebff700 1 -- 172.21.15.107:0/612044938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e81532b0 con 0x7f01e80b8a60 2023-07-18T03:38:16.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.278+0000 7f01e57fa700 1 -- 172.21.15.107:0/612044938 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f01d4017070 con 0x7f01e80b8a60 2023-07-18T03:38:16.280 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:16.284+0000 7f465a047200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T03:38:16.315 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:16.320+0000 7f2738a92200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T03:38:16.357 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.362+0000 7f01eebff700 1 -- 172.21.15.107:0/612044938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a464d08b-7474-4b14-a691-ad484e4a088d", "id": 0} v 0) v1 -- 0x7f01e80b02f0 con 0x7f01e80b8a60 2023-07-18T03:38:16.360 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.365+0000 7f01e57fa700 1 -- 172.21.15.107:0/612044938 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a464d08b-7474-4b14-a691-ad484e4a088d", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f01d400ce30 con 0x7f01e80b8a60 2023-07-18T03:38:16.361 INFO:teuthology.orchestra.run.smithi107.stdout:0 2023-07-18T03:38:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.366+0000 7f01eebff700 1 -- 172.21.15.107:0/612044938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01e80b8a60 msgr2=0x7f01e81f5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.366+0000 7f01eebff700 1 --2- 172.21.15.107:0/612044938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01e80b8a60 0x7f01e81f5b70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01d4000f60 tx=0x7f01d400c730 comp rx=0 tx=0).stop 2023-07-18T03:38:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.366+0000 7f01eebff700 1 -- 172.21.15.107:0/612044938 shutdown_connections 2023-07-18T03:38:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.366+0000 7f01eebff700 1 --2- 172.21.15.107:0/612044938 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01e80d1220 0x7f01e81f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.366+0000 7f01eebff700 1 --2- 172.21.15.107:0/612044938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e80b4530 0x7f01e8152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.366+0000 7f01eebff700 1 --2- 172.21.15.107:0/612044938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01e80b8a60 0x7f01e81f5b70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.366+0000 7f01eebff700 1 -- 172.21.15.107:0/612044938 >> 172.21.15.107:0/612044938 conn(0x7f01e809c1b0 msgr2=0x7f01e80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.366+0000 7f01eebff700 1 -- 172.21.15.107:0/612044938 shutdown_connections 2023-07-18T03:38:16.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.367+0000 7f01eebff700 1 -- 172.21.15.107:0/612044938 wait complete. 2023-07-18T03:38:16.379 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 2f096bc0-34c5-4376-baec-56bb92dfdf8f 1 2023-07-18T03:38:16.434 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:16.439+0000 7f465a047200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T03:38:16.504 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:16.509+0000 7f2738a92200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T03:38:16.539 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:16.543+0000 7f465a047200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T03:38:16.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.572+0000 7f2cbcfff700 1 Processor -- start 2023-07-18T03:38:16.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.572+0000 7f2cbcfff700 1 -- start start 2023-07-18T03:38:16.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cbcfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80b8660 0x7f2cb80cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cbcfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cb80cd970 0x7f2cb80b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cbcfff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2cb80b4da0 0x7f2cb81601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cbcfff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb80d02a0 con 0x7f2cb80cd970 2023-07-18T03:38:16.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cbcfff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb80b51c0 con 0x7f2cb80b4da0 2023-07-18T03:38:16.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cbcfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80cfce0 con 0x7f2cb80b8660 2023-07-18T03:38:16.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cb659c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80b8660 0x7f2cb80cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cb6d9d700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2cb80b4da0 0x7f2cb81601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cb6d9d700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2cb80b4da0 0x7f2cb81601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47496/0 (socket says 172.21.15.107:47496) 2023-07-18T03:38:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cb6d9d700 1 -- 172.21.15.107:0/1441552021 learned_addr learned my addr 172.21.15.107:0/1441552021 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cb659c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80b8660 0x7f2cb80cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:33772/0 (socket says 172.21.15.107:33772) 2023-07-18T03:38:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2caffff700 1 --2- 172.21.15.107:0/1441552021 >> v2:172.21.15.49:3300/0 conn(0x7f2cb80cd970 0x7f2cb80b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cb6d9d700 1 -- 172.21.15.107:0/1441552021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80b8660 msgr2=0x7f2cb80cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:16.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cb6d9d700 1 --2- 172.21.15.107:0/1441552021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80b8660 0x7f2cb80cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cb6d9d700 1 -- 172.21.15.107:0/1441552021 >> v2:172.21.15.49:3300/0 conn(0x7f2cb80cd970 msgr2=0x7f2cb80b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:16.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cb6d9d700 1 --2- 172.21.15.107:0/1441552021 >> v2:172.21.15.49:3300/0 conn(0x7f2cb80cd970 0x7f2cb80b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cb6d9d700 1 -- 172.21.15.107:0/1441552021 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb80cffc0 con 0x7f2cb80b4da0 2023-07-18T03:38:16.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.574+0000 7f2cb659c700 1 --2- 172.21.15.107:0/1441552021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80b8660 0x7f2cb80cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:16.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.575+0000 7f2cb6d9d700 1 --2- 172.21.15.107:0/1441552021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cb80b4da0 0x7f2cb81601f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8160130 tx=0x7f2cb000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23ed9f2ffb263c07 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:16.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.575+0000 7f2cb5d9b700 1 -- 172.21.15.107:0/1441552021 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb000e070 con 0x7f2cb80b4da0 2023-07-18T03:38:16.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.575+0000 7f2cb5d9b700 1 -- 172.21.15.107:0/1441552021 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb000b910 con 0x7f2cb80b4da0 2023-07-18T03:38:16.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.575+0000 7f2cb5d9b700 1 -- 172.21.15.107:0/1441552021 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb000bcc0 con 0x7f2cb80b4da0 2023-07-18T03:38:16.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.575+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1441552021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cb80b4da0 msgr2=0x7f2cb81601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.575+0000 7f2cbcfff700 1 --2- 172.21.15.107:0/1441552021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cb80b4da0 0x7f2cb81601f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8160130 tx=0x7f2cb000b330 comp rx=0 tx=0).stop 2023-07-18T03:38:16.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.576+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1441552021 shutdown_connections 2023-07-18T03:38:16.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.576+0000 7f2cbcfff700 1 --2- 172.21.15.107:0/1441552021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cb80b4da0 0x7f2cb81601f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.576+0000 7f2cbcfff700 1 --2- 172.21.15.107:0/1441552021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80b8660 0x7f2cb80cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.576+0000 7f2cbcfff700 1 --2- 172.21.15.107:0/1441552021 >> v2:172.21.15.49:3300/0 conn(0x7f2cb80cd970 0x7f2cb80b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.576+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1441552021 >> 172.21.15.107:0/1441552021 conn(0x7f2cb8066860 msgr2=0x7f2cb806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:16.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.576+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1441552021 shutdown_connections 2023-07-18T03:38:16.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.576+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1441552021 wait complete. 2023-07-18T03:38:16.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.578+0000 7f2cbcfff700 1 Processor -- start 2023-07-18T03:38:16.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.578+0000 7f2cbcfff700 1 -- start start 2023-07-18T03:38:16.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.579+0000 7f2cbcfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cb80b4da0 0x7f2cb80c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.579+0000 7f2cbcfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cb80b8660 0x7f2cb81534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2cbcfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80cd970 0x7f2cb81f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2cbcfff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb81f5b50 con 0x7f2cb80b4da0 2023-07-18T03:38:16.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2cbcfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80c8e30 con 0x7f2cb80b8660 2023-07-18T03:38:16.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2cbcfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb815e320 con 0x7f2cb80cd970 2023-07-18T03:38:16.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2caffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cb80b8660 0x7f2cb81534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2caffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cb80b8660 0x7f2cb81534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47506/0 (socket says 172.21.15.107:47506) 2023-07-18T03:38:16.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2caffff700 1 -- 172.21.15.107:0/1048312193 learned_addr learned my addr 172.21.15.107:0/1048312193 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:16.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2cb6d9d700 1 --2- 172.21.15.107:0/1048312193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80cd970 0x7f2cb81f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2cb659c700 1 --2- 172.21.15.107:0/1048312193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cb80b4da0 0x7f2cb80c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2cb6d9d700 1 -- 172.21.15.107:0/1048312193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cb80b8660 msgr2=0x7f2cb81534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2cb6d9d700 1 --2- 172.21.15.107:0/1048312193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cb80b8660 0x7f2cb81534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2cb6d9d700 1 -- 172.21.15.107:0/1048312193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cb80b4da0 msgr2=0x7f2cb80c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2cb6d9d700 1 --2- 172.21.15.107:0/1048312193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cb80b4da0 0x7f2cb80c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2cb6d9d700 1 -- 172.21.15.107:0/1048312193 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80094a0 con 0x7f2cb80cd970 2023-07-18T03:38:16.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.580+0000 7f2caffff700 1 --2- 172.21.15.107:0/1048312193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cb80b8660 0x7f2cb81534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:16.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.581+0000 7f2cb6d9d700 1 --2- 172.21.15.107:0/1048312193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80cd970 0x7f2cb81f6310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0015000 tx=0x7f2cb0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:16.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.581+0000 7f2caeffd700 1 -- 172.21.15.107:0/1048312193 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00099b0 con 0x7f2cb80cd970 2023-07-18T03:38:16.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.581+0000 7f2caeffd700 1 -- 172.21.15.107:0/1048312193 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb000ce80 con 0x7f2cb80cd970 2023-07-18T03:38:16.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.581+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1048312193 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb0009b30 con 0x7f2cb80cd970 2023-07-18T03:38:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.581+0000 7f2caeffd700 1 -- 172.21.15.107:0/1048312193 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0004490 con 0x7f2cb80cd970 2023-07-18T03:38:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.581+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1048312193 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb81f5df0 con 0x7f2cb80cd970 2023-07-18T03:38:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.581+0000 7f2caeffd700 1 -- 172.21.15.107:0/1048312193 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2cb001c440 con 0x7f2cb80cd970 2023-07-18T03:38:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.581+0000 7f2caeffd700 1 -- 172.21.15.107:0/1048312193 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f2cb00257c0 con 0x7f2cb80cd970 2023-07-18T03:38:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.582+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1048312193 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb8074350 con 0x7f2cb80cd970 2023-07-18T03:38:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.585+0000 7f2caeffd700 1 -- 172.21.15.107:0/1048312193 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f2cb0044b20 con 0x7f2cb80cd970 2023-07-18T03:38:16.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.670+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1048312193 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2f096bc0-34c5-4376-baec-56bb92dfdf8f", "id": 1} v 0) v1 -- 0x7f2cb80b2150 con 0x7f2cb80cd970 2023-07-18T03:38:16.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.674+0000 7f2caeffd700 1 -- 172.21.15.107:0/1048312193 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2f096bc0-34c5-4376-baec-56bb92dfdf8f", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2cb0033030 con 0x7f2cb80cd970 2023-07-18T03:38:16.669 INFO:teuthology.orchestra.run.smithi107.stdout:1 2023-07-18T03:38:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.675+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1048312193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80cd970 msgr2=0x7f2cb81f6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.675+0000 7f2cbcfff700 1 --2- 172.21.15.107:0/1048312193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80cd970 0x7f2cb81f6310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0015000 tx=0x7f2cb0004ad0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.675+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1048312193 shutdown_connections 2023-07-18T03:38:16.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.675+0000 7f2cbcfff700 1 --2- 172.21.15.107:0/1048312193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cb80cd970 0x7f2cb81f6310 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.675+0000 7f2cbcfff700 1 --2- 172.21.15.107:0/1048312193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cb80b4da0 0x7f2cb80c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.675+0000 7f2cbcfff700 1 --2- 172.21.15.107:0/1048312193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cb80b8660 0x7f2cb81534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.675+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1048312193 >> 172.21.15.107:0/1048312193 conn(0x7f2cb8066860 msgr2=0x7f2cb80b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:16.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.675+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1048312193 shutdown_connections 2023-07-18T03:38:16.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.675+0000 7f2cbcfff700 1 -- 172.21.15.107:0/1048312193 wait complete. 2023-07-18T03:38:16.688 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 4389944f-ae37-4b2a-b9a8-c47911d7036c 2 2023-07-18T03:38:16.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.883+0000 7f009f1ff700 1 Processor -- start 2023-07-18T03:38:16.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.883+0000 7f009f1ff700 1 -- start start 2023-07-18T03:38:16.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.884+0000 7f009f1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00980c92d0 0x7f00980ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.884+0000 7f009f1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f00980adea0 0x7f00980cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.884+0000 7f009f1ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009815ce70 con 0x7f00980b12f0 2023-07-18T03:38:16.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.884+0000 7f009f1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009815da60 con 0x7f00980adea0 2023-07-18T03:38:16.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.884+0000 7f009f1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009815d0c0 con 0x7f00980c92d0 2023-07-18T03:38:16.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.884+0000 7f009d79c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f00980adea0 0x7f00980cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.884+0000 7f009d79c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f00980adea0 0x7f00980cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47508/0 (socket says 172.21.15.107:47508) 2023-07-18T03:38:16.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.884+0000 7f009d79c700 1 -- 172.21.15.107:0/1461073839 learned_addr learned my addr 172.21.15.107:0/1461073839 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.885+0000 7f009d79c700 1 -- 172.21.15.107:0/1461073839 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00980c92d0 msgr2=0x7f00980ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.885+0000 7f009d79c700 1 --2- 172.21.15.107:0/1461073839 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00980c92d0 0x7f00980ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.885+0000 7f009d79c700 1 -- 172.21.15.107:0/1461073839 >> v1:172.21.15.49:6789/0 conn(0x7f00980b12f0 legacy=0x7f00980c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.885+0000 7f009d79c700 1 -- 172.21.15.107:0/1461073839 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009815d3a0 con 0x7f00980adea0 2023-07-18T03:38:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.885+0000 7f009d79c700 1 --2- 172.21.15.107:0/1461073839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980adea0 0x7f00980cc690 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0090009e50 tx=0x7f009000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=887006c5c3db5d15 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:16.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.885+0000 7f00977fe700 1 -- 172.21.15.107:0/1461073839 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009000e070 con 0x7f00980adea0 2023-07-18T03:38:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.885+0000 7f00977fe700 1 -- 172.21.15.107:0/1461073839 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009000b9c0 con 0x7f00980adea0 2023-07-18T03:38:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.885+0000 7f00977fe700 1 -- 172.21.15.107:0/1461073839 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009000bd70 con 0x7f00980adea0 2023-07-18T03:38:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.885+0000 7f009f1ff700 1 -- 172.21.15.107:0/1461073839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980adea0 msgr2=0x7f00980cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.885+0000 7f009f1ff700 1 --2- 172.21.15.107:0/1461073839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980adea0 0x7f00980cc690 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0090009e50 tx=0x7f009000b330 comp rx=0 tx=0).stop 2023-07-18T03:38:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.886+0000 7f009f1ff700 1 -- 172.21.15.107:0/1461073839 shutdown_connections 2023-07-18T03:38:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.886+0000 7f009f1ff700 1 --2- 172.21.15.107:0/1461073839 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00980c92d0 0x7f00980ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.886+0000 7f009f1ff700 1 --2- 172.21.15.107:0/1461073839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980adea0 0x7f00980cc690 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.886+0000 7f009f1ff700 1 -- 172.21.15.107:0/1461073839 >> 172.21.15.107:0/1461073839 conn(0x7f00980b05d0 msgr2=0x7f00980b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.886+0000 7f009f1ff700 1 -- 172.21.15.107:0/1461073839 shutdown_connections 2023-07-18T03:38:16.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.886+0000 7f009f1ff700 1 -- 172.21.15.107:0/1461073839 wait complete. 2023-07-18T03:38:16.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.887+0000 7f009f1ff700 1 Processor -- start 2023-07-18T03:38:16.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.887+0000 7f009f1ff700 1 -- start start 2023-07-18T03:38:16.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.888+0000 7f009f1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00980adea0 0x7f0098157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.888+0000 7f009f1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980b12f0 0x7f00980b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.888+0000 7f009f1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00980c92d0 0x7f00980c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:16.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.888+0000 7f009f1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009806b5b0 con 0x7f00980adea0 2023-07-18T03:38:16.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.888+0000 7f009f1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0098158640 con 0x7f00980b12f0 2023-07-18T03:38:16.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.888+0000 7f009f1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0098158920 con 0x7f00980c92d0 2023-07-18T03:38:16.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.889+0000 7f0097fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980b12f0 0x7f00980b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.889+0000 7f009cf9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00980adea0 0x7f0098157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.889+0000 7f009d79c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00980c92d0 0x7f00980c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:16.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.889+0000 7f0097fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980b12f0 0x7f00980b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47518/0 (socket says 172.21.15.107:47518) 2023-07-18T03:38:16.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.889+0000 7f0097fff700 1 -- 172.21.15.107:0/3851915735 learned_addr learned my addr 172.21.15.107:0/3851915735 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:16.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.889+0000 7f009cf9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00980adea0 0x7f0098157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.107:59990/0 (socket says 172.21.15.107:59990) 2023-07-18T03:38:16.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.889+0000 7f009d79c700 1 -- 172.21.15.107:0/3851915735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980b12f0 msgr2=0x7f00980b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.889+0000 7f009d79c700 1 --2- 172.21.15.107:0/3851915735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980b12f0 0x7f00980b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.889+0000 7f009d79c700 1 -- 172.21.15.107:0/3851915735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00980adea0 msgr2=0x7f0098157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.889+0000 7f009d79c700 1 --2- 172.21.15.107:0/3851915735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00980adea0 0x7f0098157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.889+0000 7f009d79c700 1 -- 172.21.15.107:0/3851915735 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0090009b30 con 0x7f00980c92d0 2023-07-18T03:38:16.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.889+0000 7f009d79c700 1 --2- 172.21.15.107:0/3851915735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00980c92d0 0x7f00980c5d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0090015000 tx=0x7f0090004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:16.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.890+0000 7f00957fa700 1 -- 172.21.15.107:0/3851915735 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0090004800 con 0x7f00980c92d0 2023-07-18T03:38:16.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.890+0000 7f00957fa700 1 -- 172.21.15.107:0/3851915735 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00900092e0 con 0x7f00980c92d0 2023-07-18T03:38:16.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.890+0000 7f009f1ff700 1 -- 172.21.15.107:0/3851915735 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0098158c00 con 0x7f00980c92d0 2023-07-18T03:38:16.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.890+0000 7f00957fa700 1 -- 172.21.15.107:0/3851915735 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009001c680 con 0x7f00980c92d0 2023-07-18T03:38:16.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.890+0000 7f009f1ff700 1 -- 172.21.15.107:0/3851915735 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00980b42a0 con 0x7f00980c92d0 2023-07-18T03:38:16.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.890+0000 7f00957fa700 1 -- 172.21.15.107:0/3851915735 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f009001cae0 con 0x7f00980c92d0 2023-07-18T03:38:16.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.890+0000 7f00957fa700 1 -- 172.21.15.107:0/3851915735 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f009001cd90 con 0x7f00980c92d0 2023-07-18T03:38:16.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.891+0000 7f009f1ff700 1 -- 172.21.15.107:0/3851915735 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009806ba00 con 0x7f00980c92d0 2023-07-18T03:38:16.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.893+0000 7f00957fa700 1 -- 172.21.15.107:0/3851915735 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f0090043b20 con 0x7f00980c92d0 2023-07-18T03:38:16.937 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:16.941+0000 7f2738a92200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T03:38:16.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.977+0000 7f009f1ff700 1 -- 172.21.15.107:0/3851915735 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4389944f-ae37-4b2a-b9a8-c47911d7036c", "id": 2} v 0) v1 -- 0x7f0098158d90 con 0x7f00980c92d0 2023-07-18T03:38:16.976 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.981+0000 7f00957fa700 1 -- 172.21.15.107:0/3851915735 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4389944f-ae37-4b2a-b9a8-c47911d7036c", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0090032030 con 0x7f00980c92d0 2023-07-18T03:38:16.977 INFO:teuthology.orchestra.run.smithi107.stdout:2 2023-07-18T03:38:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.982+0000 7f009f1ff700 1 -- 172.21.15.107:0/3851915735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00980c92d0 msgr2=0x7f00980c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.982+0000 7f009f1ff700 1 --2- 172.21.15.107:0/3851915735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00980c92d0 0x7f00980c5d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0090015000 tx=0x7f0090004ad0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.982+0000 7f009f1ff700 1 -- 172.21.15.107:0/3851915735 shutdown_connections 2023-07-18T03:38:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.982+0000 7f009f1ff700 1 --2- 172.21.15.107:0/3851915735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00980c92d0 0x7f00980c5d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.979 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.982+0000 7f009f1ff700 1 --2- 172.21.15.107:0/3851915735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00980adea0 0x7f0098157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.979 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.982+0000 7f009f1ff700 1 --2- 172.21.15.107:0/3851915735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980b12f0 0x7f00980b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:16.979 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.982+0000 7f009f1ff700 1 -- 172.21.15.107:0/3851915735 >> 172.21.15.107:0/3851915735 conn(0x7f00980b05d0 msgr2=0x7f009806f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:16.979 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.983+0000 7f009f1ff700 1 -- 172.21.15.107:0/3851915735 shutdown_connections 2023-07-18T03:38:16.979 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:16.983+0000 7f009f1ff700 1 -- 172.21.15.107:0/3851915735 wait complete. 2023-07-18T03:38:17.002 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 2bc9c5af-bca8-4a1d-8bba-47c25a3e281c 3 2023-07-18T03:38:17.037 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:17.041+0000 7f2738a92200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T03:38:17.093 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:17.098+0000 7f465a047200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T03:38:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.208+0000 7f1d279ff700 1 Processor -- start 2023-07-18T03:38:17.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.208+0000 7f1d279ff700 1 -- start start 2023-07-18T03:38:17.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.209+0000 7f1d279ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d200d0480 0x7f1d200b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.209+0000 7f1d279ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2015cd60 con 0x7f1d200d0010 2023-07-18T03:38:17.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.209+0000 7f1d279ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d200cf340 con 0x7f1d200b4600 2023-07-18T03:38:17.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.209+0000 7f1d279ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d200d1550 con 0x7f1d200d0480 2023-07-18T03:38:17.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.209+0000 7f1d2579b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f1d200b4600 0x7f1d200cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:50392/0 (socket says 172.21.15.107:50392) 2023-07-18T03:38:17.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.209+0000 7f1d2579b700 1 -- 172.21.15.107:0/3935399284 learned_addr learned my addr 172.21.15.107:0/3935399284 (peer_addr_for_me v1:172.21.15.107:0/0) 2023-07-18T03:38:17.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.209+0000 7f1d25f9c700 1 --2- 172.21.15.107:0/3935399284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d200d0480 0x7f1d200b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.206 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:17.209+0000 7f2738a92200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T03:38:17.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.210+0000 7f1d17fff700 1 -- 172.21.15.107:0/3935399284 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059478625 0 0) 0x7f1d200cf340 con 0x7f1d200b4600 2023-07-18T03:38:17.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.210+0000 7f1d17fff700 1 -- 172.21.15.107:0/3935399284 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d0c0035b0 con 0x7f1d200b4600 2023-07-18T03:38:17.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.210+0000 7f1d25f9c700 1 -- 172.21.15.107:0/3935399284 >> v1:172.21.15.107:6789/0 conn(0x7f1d200b4600 legacy=0x7f1d200cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.210+0000 7f1d25f9c700 1 -- 172.21.15.107:0/3935399284 >> v1:172.21.15.49:6789/0 conn(0x7f1d200d0010 legacy=0x7f1d200b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.210+0000 7f1d25f9c700 1 -- 172.21.15.107:0/3935399284 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d200d0e90 con 0x7f1d200d0480 2023-07-18T03:38:17.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.210+0000 7f1d25f9c700 1 --2- 172.21.15.107:0/3935399284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d200d0480 0x7f1d200b6600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009dc0 tx=0x7f1d1800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94fa542aa9707824 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:17.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.210+0000 7f1d17fff700 1 -- 172.21.15.107:0/3935399284 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1800e070 con 0x7f1d200d0480 2023-07-18T03:38:17.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.210+0000 7f1d17fff700 1 -- 172.21.15.107:0/3935399284 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d18003a20 con 0x7f1d200d0480 2023-07-18T03:38:17.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.210+0000 7f1d17fff700 1 -- 172.21.15.107:0/3935399284 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18003dd0 con 0x7f1d200d0480 2023-07-18T03:38:17.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.211+0000 7f1d279ff700 1 -- 172.21.15.107:0/3935399284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d200d0480 msgr2=0x7f1d200b6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.211+0000 7f1d279ff700 1 --2- 172.21.15.107:0/3935399284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d200d0480 0x7f1d200b6600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009dc0 tx=0x7f1d1800b330 comp rx=0 tx=0).stop 2023-07-18T03:38:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.211+0000 7f1d279ff700 1 -- 172.21.15.107:0/3935399284 shutdown_connections 2023-07-18T03:38:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.211+0000 7f1d279ff700 1 --2- 172.21.15.107:0/3935399284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d200d0480 0x7f1d200b6600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.211+0000 7f1d279ff700 1 -- 172.21.15.107:0/3935399284 >> 172.21.15.107:0/3935399284 conn(0x7f1d2009c200 msgr2=0x7f1d200d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.211+0000 7f1d279ff700 1 -- 172.21.15.107:0/3935399284 shutdown_connections 2023-07-18T03:38:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.211+0000 7f1d279ff700 1 -- 172.21.15.107:0/3935399284 wait complete. 2023-07-18T03:38:17.210 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.214+0000 7f1d279ff700 1 Processor -- start 2023-07-18T03:38:17.210 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.214+0000 7f1d279ff700 1 -- start start 2023-07-18T03:38:17.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.215+0000 7f1d279ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d200d0010 0x7f1d2015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.215+0000 7f1d279ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d200d0480 0x7f1d200afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.215+0000 7f1d279ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d200b00a0 0x7f1d201f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.215+0000 7f1d279ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d201e56b0 con 0x7f1d200d0010 2023-07-18T03:38:17.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.215+0000 7f1d279ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d201f4920 con 0x7f1d200d0480 2023-07-18T03:38:17.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.215+0000 7f1d279ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d201e4ea0 con 0x7f1d200b00a0 2023-07-18T03:38:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.215+0000 7f1d2579b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d200d0010 0x7f1d2015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.215+0000 7f1d24f9a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d200d0480 0x7f1d200afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.215+0000 7f1d25f9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d200b00a0 0x7f1d201f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.215+0000 7f1d2579b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d200d0010 0x7f1d2015a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.107:60006/0 (socket says 172.21.15.107:60006) 2023-07-18T03:38:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.215+0000 7f1d2579b700 1 -- 172.21.15.107:0/247578576 learned_addr learned my addr 172.21.15.107:0/247578576 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.216+0000 7f1d24f9a700 1 -- 172.21.15.107:0/247578576 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d200b00a0 msgr2=0x7f1d201f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.216+0000 7f1d24f9a700 1 --2- 172.21.15.107:0/247578576 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d200b00a0 0x7f1d201f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.216+0000 7f1d24f9a700 1 -- 172.21.15.107:0/247578576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d200d0010 msgr2=0x7f1d2015a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.216+0000 7f1d24f9a700 1 --2- 172.21.15.107:0/247578576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d200d0010 0x7f1d2015a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.216+0000 7f1d24f9a700 1 -- 172.21.15.107:0/247578576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d18009a70 con 0x7f1d200d0480 2023-07-18T03:38:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.216+0000 7f1d2579b700 1 --2- 172.21.15.107:0/247578576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d200d0010 0x7f1d2015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.216+0000 7f1d24f9a700 1 --2- 172.21.15.107:0/247578576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d200d0480 0x7f1d200afb60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000c750 tx=0x7f1d1000ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.216+0000 7f1d15ffb700 1 -- 172.21.15.107:0/247578576 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d100054e0 con 0x7f1d200d0480 2023-07-18T03:38:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.216+0000 7f1d15ffb700 1 -- 172.21.15.107:0/247578576 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d10003680 con 0x7f1d200d0480 2023-07-18T03:38:17.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.216+0000 7f1d15ffb700 1 -- 172.21.15.107:0/247578576 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d100039f0 con 0x7f1d200d0480 2023-07-18T03:38:17.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.216+0000 7f1d279ff700 1 -- 172.21.15.107:0/247578576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d200c97a0 con 0x7f1d200d0480 2023-07-18T03:38:17.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.217+0000 7f1d279ff700 1 -- 172.21.15.107:0/247578576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d201e53d0 con 0x7f1d200d0480 2023-07-18T03:38:17.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.217+0000 7f1d15ffb700 1 -- 172.21.15.107:0/247578576 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1d10012400 con 0x7f1d200d0480 2023-07-18T03:38:17.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.217+0000 7f1d15ffb700 1 -- 172.21.15.107:0/247578576 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f1d1001b890 con 0x7f1d200d0480 2023-07-18T03:38:17.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.218+0000 7f1d279ff700 1 -- 172.21.15.107:0/247578576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d201f4a90 con 0x7f1d200d0480 2023-07-18T03:38:17.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.220+0000 7f1d15ffb700 1 -- 172.21.15.107:0/247578576 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f1d201f4a90 con 0x7f1d200d0480 2023-07-18T03:38:17.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.304+0000 7f1d279ff700 1 -- 172.21.15.107:0/247578576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2bc9c5af-bca8-4a1d-8bba-47c25a3e281c", "id": 3} v 0) v1 -- 0x7f1d20074350 con 0x7f1d200d0480 2023-07-18T03:38:17.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.312+0000 7f1d15ffb700 1 -- 172.21.15.107:0/247578576 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2bc9c5af-bca8-4a1d-8bba-47c25a3e281c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1d10021060 con 0x7f1d200d0480 2023-07-18T03:38:17.308 INFO:teuthology.orchestra.run.smithi107.stdout:3 2023-07-18T03:38:17.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.313+0000 7f1d279ff700 1 -- 172.21.15.107:0/247578576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d200d0480 msgr2=0x7f1d200afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.314+0000 7f1d279ff700 1 --2- 172.21.15.107:0/247578576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d200d0480 0x7f1d200afb60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000c750 tx=0x7f1d1000ca60 comp rx=0 tx=0).stop 2023-07-18T03:38:17.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.314+0000 7f1d279ff700 1 -- 172.21.15.107:0/247578576 shutdown_connections 2023-07-18T03:38:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.314+0000 7f1d279ff700 1 --2- 172.21.15.107:0/247578576 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d200b00a0 0x7f1d201f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.314+0000 7f1d279ff700 1 --2- 172.21.15.107:0/247578576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d200d0010 0x7f1d2015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.314+0000 7f1d279ff700 1 --2- 172.21.15.107:0/247578576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d200d0480 0x7f1d200afb60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.314+0000 7f1d279ff700 1 -- 172.21.15.107:0/247578576 >> 172.21.15.107:0/247578576 conn(0x7f1d2009c200 msgr2=0x7f1d200d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.314+0000 7f1d279ff700 1 -- 172.21.15.107:0/247578576 shutdown_connections 2023-07-18T03:38:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.314+0000 7f1d279ff700 1 -- 172.21.15.107:0/247578576 wait complete. 2023-07-18T03:38:17.311 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:17.315+0000 7f2738a92200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T03:38:17.328 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 03c48b18-d741-436e-a801-ae2aa956ff28 4 2023-07-18T03:38:17.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.519+0000 7faa1fdff700 1 Processor -- start 2023-07-18T03:38:17.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.520+0000 7faa1fdff700 1 -- start start 2023-07-18T03:38:17.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.521+0000 7faa1fdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7faa180b8a60 0x7faa180b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.521+0000 7faa1fdff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faa180b4530 0x7faa180d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.521+0000 7faa1fdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa180d1490 con 0x7faa180d1050 2023-07-18T03:38:17.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.521+0000 7faa1fdff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa180b8e30 con 0x7faa180b4530 2023-07-18T03:38:17.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.521+0000 7faa1fdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa180b1e30 con 0x7faa180b8a60 2023-07-18T03:38:17.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.521+0000 7faa1db9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7faa180b8a60 0x7faa180b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.519 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.521+0000 7faa1db9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7faa180b8a60 0x7faa180b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47538/0 (socket says 172.21.15.107:47538) 2023-07-18T03:38:17.519 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.521+0000 7faa1db9b700 1 -- 172.21.15.107:0/1597658000 learned_addr learned my addr 172.21.15.107:0/1597658000 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:17.519 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.521+0000 7faa1d39a700 1 --2- 172.21.15.107:0/1597658000 >> v2:172.21.15.49:3300/0 conn(0x7faa180b4530 0x7faa180d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.520 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.522+0000 7faa1db9b700 1 -- 172.21.15.107:0/1597658000 >> v2:172.21.15.49:3300/0 conn(0x7faa180b4530 msgr2=0x7faa180d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:17.520 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.522+0000 7faa1db9b700 1 --2- 172.21.15.107:0/1597658000 >> v2:172.21.15.49:3300/0 conn(0x7faa180b4530 0x7faa180d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.520 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.522+0000 7faa1db9b700 1 -- 172.21.15.107:0/1597658000 >> v1:172.21.15.107:6789/0 conn(0x7faa180d1050 legacy=0x7faa180b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.521 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.522+0000 7faa1db9b700 1 -- 172.21.15.107:0/1597658000 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa180afaa0 con 0x7faa180b8a60 2023-07-18T03:38:17.521 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.522+0000 7faa1db9b700 1 --2- 172.21.15.107:0/1597658000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa180b8a60 0x7faa180b3ff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faa180b5b00 tx=0x7faa14009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8ee133288a7d31a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:17.521 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.522+0000 7faa1cb99700 1 -- 172.21.15.107:0/1597658000 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14011830 con 0x7faa180b8a60 2023-07-18T03:38:17.521 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.522+0000 7faa1cb99700 1 -- 172.21.15.107:0/1597658000 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14011990 con 0x7faa180b8a60 2023-07-18T03:38:17.522 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.522+0000 7faa1cb99700 1 -- 172.21.15.107:0/1597658000 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14011d20 con 0x7faa180b8a60 2023-07-18T03:38:17.525 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.523+0000 7faa1fdff700 1 -- 172.21.15.107:0/1597658000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa180b8a60 msgr2=0x7faa180b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.525 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.523+0000 7faa1fdff700 1 --2- 172.21.15.107:0/1597658000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa180b8a60 0x7faa180b3ff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faa180b5b00 tx=0x7faa14009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.525 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.523+0000 7faa1fdff700 1 -- 172.21.15.107:0/1597658000 shutdown_connections 2023-07-18T03:38:17.525 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.523+0000 7faa1fdff700 1 --2- 172.21.15.107:0/1597658000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa180b8a60 0x7faa180b3ff0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.525 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.523+0000 7faa1fdff700 1 --2- 172.21.15.107:0/1597658000 >> v2:172.21.15.49:3300/0 conn(0x7faa180b4530 0x7faa180d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.523+0000 7faa1fdff700 1 -- 172.21.15.107:0/1597658000 >> 172.21.15.107:0/1597658000 conn(0x7faa1809c1b0 msgr2=0x7faa180d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:17.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.523+0000 7faa1fdff700 1 -- 172.21.15.107:0/1597658000 shutdown_connections 2023-07-18T03:38:17.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.523+0000 7faa1fdff700 1 -- 172.21.15.107:0/1597658000 wait complete. 2023-07-18T03:38:17.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.524+0000 7faa1fdff700 1 Processor -- start 2023-07-18T03:38:17.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.525+0000 7faa1fdff700 1 -- start start 2023-07-18T03:38:17.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.525+0000 7faa1fdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa180b4530 0x7faa180ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1fdff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa180b8a60 0x7faa180ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.527 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1fdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa180d1050 0x7faa180cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.527 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1fdff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1815db60 con 0x7faa180b8a60 2023-07-18T03:38:17.527 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1fdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1815de00 con 0x7faa180b4530 2023-07-18T03:38:17.527 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1fdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa180c9580 con 0x7faa180d1050 2023-07-18T03:38:17.527 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1db9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa180b4530 0x7faa180ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.527 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1db9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa180b4530 0x7faa180ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47544/0 (socket says 172.21.15.107:47544) 2023-07-18T03:38:17.527 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1db9b700 1 -- 172.21.15.107:0/698483786 learned_addr learned my addr 172.21.15.107:0/698483786 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:17.527 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1e39c700 1 --2- 172.21.15.107:0/698483786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa180d1050 0x7faa180cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1d39a700 1 --2- 172.21.15.107:0/698483786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa180b8a60 0x7faa180ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1db9b700 1 -- 172.21.15.107:0/698483786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa180d1050 msgr2=0x7faa180cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1db9b700 1 --2- 172.21.15.107:0/698483786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa180d1050 0x7faa180cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1db9b700 1 -- 172.21.15.107:0/698483786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa180b8a60 msgr2=0x7faa180ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1db9b700 1 --2- 172.21.15.107:0/698483786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa180b8a60 0x7faa180ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1db9b700 1 -- 172.21.15.107:0/698483786 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa14009550 con 0x7faa180b4530 2023-07-18T03:38:17.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.526+0000 7faa1db9b700 1 --2- 172.21.15.107:0/698483786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa180b4530 0x7faa180ca010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faa14000c00 tx=0x7faa14004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:17.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.527+0000 7faa0e7fc700 1 -- 172.21.15.107:0/698483786 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14004380 con 0x7faa180b4530 2023-07-18T03:38:17.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.527+0000 7faa0e7fc700 1 -- 172.21.15.107:0/698483786 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1401f070 con 0x7faa180b4530 2023-07-18T03:38:17.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.527+0000 7faa0e7fc700 1 -- 172.21.15.107:0/698483786 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140036a0 con 0x7faa180b4530 2023-07-18T03:38:17.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.527+0000 7faa1fdff700 1 -- 172.21.15.107:0/698483786 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa180cf7c0 con 0x7faa180b4530 2023-07-18T03:38:17.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.527+0000 7faa1fdff700 1 -- 172.21.15.107:0/698483786 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa180cfd80 con 0x7faa180b4530 2023-07-18T03:38:17.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.527+0000 7faa0e7fc700 1 -- 172.21.15.107:0/698483786 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faa14021030 con 0x7faa180b4530 2023-07-18T03:38:17.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.527+0000 7faa0e7fc700 1 -- 172.21.15.107:0/698483786 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7faa1402d440 con 0x7faa180b4530 2023-07-18T03:38:17.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.528+0000 7faa1fdff700 1 -- 172.21.15.107:0/698483786 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa180cff70 con 0x7faa180b4530 2023-07-18T03:38:17.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.530+0000 7faa0e7fc700 1 -- 172.21.15.107:0/698483786 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7faa14015020 con 0x7faa180b4530 2023-07-18T03:38:17.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.646+0000 7faa1fdff700 1 -- 172.21.15.107:0/698483786 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "03c48b18-d741-436e-a801-ae2aa956ff28", "id": 4} v 0) v1 -- 0x7faa18074350 con 0x7faa180b4530 2023-07-18T03:38:17.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.658+0000 7faa0e7fc700 1 -- 172.21.15.107:0/698483786 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "03c48b18-d741-436e-a801-ae2aa956ff28", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7faa1401a520 con 0x7faa180b4530 2023-07-18T03:38:17.654 INFO:teuthology.orchestra.run.smithi107.stdout:4 2023-07-18T03:38:17.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.659+0000 7faa1fdff700 1 -- 172.21.15.107:0/698483786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa180b4530 msgr2=0x7faa180ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.659+0000 7faa1fdff700 1 --2- 172.21.15.107:0/698483786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa180b4530 0x7faa180ca010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faa14000c00 tx=0x7faa14004750 comp rx=0 tx=0).stop 2023-07-18T03:38:17.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.659+0000 7faa1fdff700 1 -- 172.21.15.107:0/698483786 shutdown_connections 2023-07-18T03:38:17.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.659+0000 7faa1fdff700 1 --2- 172.21.15.107:0/698483786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa180d1050 0x7faa180cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.659+0000 7faa1fdff700 1 --2- 172.21.15.107:0/698483786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa180b4530 0x7faa180ca010 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.659+0000 7faa1fdff700 1 --2- 172.21.15.107:0/698483786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa180b8a60 0x7faa180ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.659+0000 7faa1fdff700 1 -- 172.21.15.107:0/698483786 >> 172.21.15.107:0/698483786 conn(0x7faa1809c1b0 msgr2=0x7faa1806b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:17.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.659+0000 7faa1fdff700 1 -- 172.21.15.107:0/698483786 shutdown_connections 2023-07-18T03:38:17.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.659+0000 7faa1fdff700 1 -- 172.21.15.107:0/698483786 wait complete. 2023-07-18T03:38:17.672 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 545a1dd0-15cd-4867-912c-525da6ea7752 5 2023-07-18T03:38:17.825 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:17.830+0000 7f465a047200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T03:38:17.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.868+0000 7f6236fff700 1 Processor -- start 2023-07-18T03:38:17.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.868+0000 7f6236fff700 1 -- start start 2023-07-18T03:38:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f6236fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62300b8660 0x7f62300cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f6236fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b4f70 0x7f62300d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f6236fff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62300d02a0 con 0x7f62300cd970 2023-07-18T03:38:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f6236fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62300cfce0 con 0x7f62300b8660 2023-07-18T03:38:17.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f6236fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62300cffc0 con 0x7f62300b4f70 2023-07-18T03:38:17.866 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:17.870+0000 7f2738a92200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T03:38:17.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f623559c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b4f70 0x7f62300d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f6234d9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62300b8660 0x7f62300cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f623559c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b4f70 0x7f62300d12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:33850/0 (socket says 172.21.15.107:33850) 2023-07-18T03:38:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f623559c700 1 -- 172.21.15.107:0/1254242303 learned_addr learned my addr 172.21.15.107:0/1254242303 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f623559c700 1 -- 172.21.15.107:0/1254242303 >> v2:172.21.15.49:3300/0 conn(0x7f62300b8660 msgr2=0x7f62300cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f623559c700 1 --2- 172.21.15.107:0/1254242303 >> v2:172.21.15.49:3300/0 conn(0x7f62300b8660 0x7f62300cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f623559c700 1 -- 172.21.15.107:0/1254242303 >> v1:172.21.15.49:6789/0 conn(0x7f62300cd970 legacy=0x7f62300b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.870+0000 7f623559c700 1 -- 172.21.15.107:0/1254242303 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62300d0ac0 con 0x7f62300b4f70 2023-07-18T03:38:17.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.871+0000 7f623559c700 1 --2- 172.21.15.107:0/1254242303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b4f70 0x7f62300d12c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6228009e50 tx=0x7f622800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89493f412a8ce427 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:17.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.871+0000 7f622f7fe700 1 -- 172.21.15.107:0/1254242303 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622800e070 con 0x7f62300b4f70 2023-07-18T03:38:17.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.871+0000 7f622f7fe700 1 -- 172.21.15.107:0/1254242303 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622800b910 con 0x7f62300b4f70 2023-07-18T03:38:17.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.871+0000 7f622f7fe700 1 -- 172.21.15.107:0/1254242303 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622800bcc0 con 0x7f62300b4f70 2023-07-18T03:38:17.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.872+0000 7f6236fff700 1 -- 172.21.15.107:0/1254242303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b4f70 msgr2=0x7f62300d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.872+0000 7f6236fff700 1 --2- 172.21.15.107:0/1254242303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b4f70 0x7f62300d12c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6228009e50 tx=0x7f622800b330 comp rx=0 tx=0).stop 2023-07-18T03:38:17.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.872+0000 7f6236fff700 1 -- 172.21.15.107:0/1254242303 shutdown_connections 2023-07-18T03:38:17.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.872+0000 7f6236fff700 1 --2- 172.21.15.107:0/1254242303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b4f70 0x7f62300d12c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.872+0000 7f6236fff700 1 --2- 172.21.15.107:0/1254242303 >> v2:172.21.15.49:3300/0 conn(0x7f62300b8660 0x7f62300cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.872+0000 7f6236fff700 1 -- 172.21.15.107:0/1254242303 >> 172.21.15.107:0/1254242303 conn(0x7f6230066860 msgr2=0x7f623006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:17.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.872+0000 7f6236fff700 1 -- 172.21.15.107:0/1254242303 shutdown_connections 2023-07-18T03:38:17.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.872+0000 7f6236fff700 1 -- 172.21.15.107:0/1254242303 wait complete. 2023-07-18T03:38:17.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.873+0000 7f6236fff700 1 Processor -- start 2023-07-18T03:38:17.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.874+0000 7f6236fff700 1 -- start start 2023-07-18T03:38:17.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.874+0000 7f6236fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62300b4f70 0x7f62300c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.874+0000 7f6236fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b8660 0x7f62300c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f6236fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62300cd970 0x7f62301f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:17.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f6236fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62300c81f0 con 0x7f62300b4f70 2023-07-18T03:38:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f6236fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62300c7f10 con 0x7f62300cd970 2023-07-18T03:38:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f6236fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62300ca460 con 0x7f62300b8660 2023-07-18T03:38:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f622ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b8660 0x7f62300c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f622ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b8660 0x7f62300c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:33858/0 (socket says 172.21.15.107:33858) 2023-07-18T03:38:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f622ffff700 1 -- 172.21.15.107:0/385941984 learned_addr learned my addr 172.21.15.107:0/385941984 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f6234d9b700 1 --2- 172.21.15.107:0/385941984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62300b4f70 0x7f62300c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f623559c700 1 --2- 172.21.15.107:0/385941984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62300cd970 0x7f62301f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f622ffff700 1 -- 172.21.15.107:0/385941984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62300cd970 msgr2=0x7f62301f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f622ffff700 1 --2- 172.21.15.107:0/385941984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62300cd970 0x7f62301f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f622ffff700 1 -- 172.21.15.107:0/385941984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62300b4f70 msgr2=0x7f62300c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f622ffff700 1 --2- 172.21.15.107:0/385941984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62300b4f70 0x7f62300c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f622ffff700 1 -- 172.21.15.107:0/385941984 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6228009b30 con 0x7f62300b8660 2023-07-18T03:38:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.875+0000 7f6234d9b700 1 --2- 172.21.15.107:0/385941984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62300b4f70 0x7f62300c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.876+0000 7f622ffff700 1 --2- 172.21.15.107:0/385941984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b8660 0x7f62300c8e80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c450 tx=0x7f621c00c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:17.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.876+0000 7f622d7fa700 1 -- 172.21.15.107:0/385941984 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0052d0 con 0x7f62300b8660 2023-07-18T03:38:17.875 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.876+0000 7f622d7fa700 1 -- 172.21.15.107:0/385941984 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c005430 con 0x7f62300b8660 2023-07-18T03:38:17.875 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.876+0000 7f6236fff700 1 -- 172.21.15.107:0/385941984 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62300ca740 con 0x7f62300b8660 2023-07-18T03:38:17.875 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.876+0000 7f622d7fa700 1 -- 172.21.15.107:0/385941984 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0038c0 con 0x7f62300b8660 2023-07-18T03:38:17.875 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.876+0000 7f6236fff700 1 -- 172.21.15.107:0/385941984 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62300c7c20 con 0x7f62300b8660 2023-07-18T03:38:17.875 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.876+0000 7f622d7fa700 1 -- 172.21.15.107:0/385941984 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f621c003d20 con 0x7f62300b8660 2023-07-18T03:38:17.875 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.876+0000 7f622d7fa700 1 -- 172.21.15.107:0/385941984 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f621c011cc0 con 0x7f62300b8660 2023-07-18T03:38:17.876 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.877+0000 7f6236fff700 1 -- 172.21.15.107:0/385941984 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62300cac60 con 0x7f62300b8660 2023-07-18T03:38:17.876 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.880+0000 7f622d7fa700 1 -- 172.21.15.107:0/385941984 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f62300cac60 con 0x7f62300b8660 2023-07-18T03:38:17.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.967+0000 7f6236fff700 1 -- 172.21.15.107:0/385941984 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "545a1dd0-15cd-4867-912c-525da6ea7752", "id": 5} v 0) v1 -- 0x7f6230074350 con 0x7f62300b8660 2023-07-18T03:38:17.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.976+0000 7f622d7fa700 1 -- 172.21.15.107:0/385941984 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "545a1dd0-15cd-4867-912c-525da6ea7752", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f621c017070 con 0x7f62300b8660 2023-07-18T03:38:17.971 INFO:teuthology.orchestra.run.smithi107.stdout:5 2023-07-18T03:38:17.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.977+0000 7f6236fff700 1 -- 172.21.15.107:0/385941984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b8660 msgr2=0x7f62300c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:17.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.977+0000 7f6236fff700 1 --2- 172.21.15.107:0/385941984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b8660 0x7f62300c8e80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c450 tx=0x7f621c00c760 comp rx=0 tx=0).stop 2023-07-18T03:38:17.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.977+0000 7f6236fff700 1 -- 172.21.15.107:0/385941984 shutdown_connections 2023-07-18T03:38:17.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.977+0000 7f6236fff700 1 --2- 172.21.15.107:0/385941984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62300cd970 0x7f62301f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.977+0000 7f6236fff700 1 --2- 172.21.15.107:0/385941984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62300b4f70 0x7f62300c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.977+0000 7f6236fff700 1 --2- 172.21.15.107:0/385941984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62300b8660 0x7f62300c8e80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:17.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.977+0000 7f6236fff700 1 -- 172.21.15.107:0/385941984 >> 172.21.15.107:0/385941984 conn(0x7f6230066860 msgr2=0x7f62300d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:17.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.977+0000 7f6236fff700 1 -- 172.21.15.107:0/385941984 shutdown_connections 2023-07-18T03:38:17.974 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:17.977+0000 7f6236fff700 1 -- 172.21.15.107:0/385941984 wait complete. 2023-07-18T03:38:17.996 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 19421f1b-8499-4bea-9a1b-3be1118862fd 6 2023-07-18T03:38:18.032 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:18.036+0000 7f465a047200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T03:38:18.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.197+0000 7f924d9ff700 1 Processor -- start 2023-07-18T03:38:18.193 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.197+0000 7f924d9ff700 1 -- start start 2023-07-18T03:38:18.194 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.198+0000 7f924d9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9248158bb0 0x7f92480cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:18.194 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.199+0000 7f924d9ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92480ce8c0 con 0x7f92480ce150 2023-07-18T03:38:18.194 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.199+0000 7f924d9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92480cc2b0 con 0x7f92480cbe40 2023-07-18T03:38:18.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.199+0000 7f924d9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924806b5b0 con 0x7f9248158bb0 2023-07-18T03:38:18.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.199+0000 7f92467fc700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f92480cbe40 0x7f92480cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:50398/0 (socket says 172.21.15.107:50398) 2023-07-18T03:38:18.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.199+0000 7f9246ffd700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9248158bb0 0x7f92480cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:18.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.199+0000 7f92467fc700 1 -- 172.21.15.107:0/4253994799 learned_addr learned my addr 172.21.15.107:0/4253994799 (peer_addr_for_me v1:172.21.15.107:0/0) 2023-07-18T03:38:18.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.199+0000 7f9245ffb700 1 -- 172.21.15.107:0/4253994799 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556783688 0 0) 0x7f92480cc2b0 con 0x7f92480cbe40 2023-07-18T03:38:18.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.199+0000 7f9245ffb700 1 -- 172.21.15.107:0/4253994799 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92300035b0 con 0x7f92480cbe40 2023-07-18T03:38:18.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.199+0000 7f9246ffd700 1 -- 172.21.15.107:0/4253994799 >> v1:172.21.15.107:6789/0 conn(0x7f92480cbe40 legacy=0x7f92480cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:18.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.199+0000 7f9246ffd700 1 -- 172.21.15.107:0/4253994799 >> v1:172.21.15.49:6789/0 conn(0x7f92480ce150 legacy=0x7f924806ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:18.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.199+0000 7f9245ffb700 1 -- 172.21.15.107:0/4253994799 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527226149 0 0) 0x7f92480ce8c0 con 0x7f92480ce150 2023-07-18T03:38:18.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.199+0000 7f9246ffd700 1 -- 172.21.15.107:0/4253994799 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92480cae30 con 0x7f9248158bb0 2023-07-18T03:38:18.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.200+0000 7f9246ffd700 1 --2- 172.21.15.107:0/4253994799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9248158bb0 0x7f92480cd0a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9238009870 tx=0x7f9238009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc24766e4b3af22 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:18.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.200+0000 7f9245ffb700 1 -- 172.21.15.107:0/4253994799 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238011830 con 0x7f9248158bb0 2023-07-18T03:38:18.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.200+0000 7f9245ffb700 1 -- 172.21.15.107:0/4253994799 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9238011990 con 0x7f9248158bb0 2023-07-18T03:38:18.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.200+0000 7f9245ffb700 1 -- 172.21.15.107:0/4253994799 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238011d20 con 0x7f9248158bb0 2023-07-18T03:38:18.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.200+0000 7f924d9ff700 1 -- 172.21.15.107:0/4253994799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9248158bb0 msgr2=0x7f92480cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:18.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.200+0000 7f924d9ff700 1 --2- 172.21.15.107:0/4253994799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9248158bb0 0x7f92480cd0a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9238009870 tx=0x7f9238009b80 comp rx=0 tx=0).stop 2023-07-18T03:38:18.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.201+0000 7f924d9ff700 1 -- 172.21.15.107:0/4253994799 shutdown_connections 2023-07-18T03:38:18.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.201+0000 7f924d9ff700 1 --2- 172.21.15.107:0/4253994799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9248158bb0 0x7f92480cd0a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.201+0000 7f924d9ff700 1 -- 172.21.15.107:0/4253994799 >> 172.21.15.107:0/4253994799 conn(0x7f92480b05d0 msgr2=0x7f92480b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:18.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.201+0000 7f924d9ff700 1 -- 172.21.15.107:0/4253994799 shutdown_connections 2023-07-18T03:38:18.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.201+0000 7f924d9ff700 1 -- 172.21.15.107:0/4253994799 wait complete. 2023-07-18T03:38:18.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.202+0000 7f924d9ff700 1 Processor -- start 2023-07-18T03:38:18.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.202+0000 7f924d9ff700 1 -- start start 2023-07-18T03:38:18.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.203+0000 7f924d9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92480cbe40 0x7f92481f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:18.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.203+0000 7f924d9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92480ce150 0x7f924814f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:18.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.203+0000 7f924d9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9248158bb0 0x7f924814f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:18.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.203+0000 7f924d9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92480af7b0 con 0x7f9248158bb0 2023-07-18T03:38:18.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.203+0000 7f924d9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92481f2530 con 0x7f92480ce150 2023-07-18T03:38:18.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.203+0000 7f924d9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92480aeba0 con 0x7f92480cbe40 2023-07-18T03:38:18.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.204+0000 7f9246ffd700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92480cbe40 0x7f92481f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:18.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.204+0000 7f92467fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92480ce150 0x7f924814f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:18.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.204+0000 7f9246ffd700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92480cbe40 0x7f92481f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:33872/0 (socket says 172.21.15.107:33872) 2023-07-18T03:38:18.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.204+0000 7f9246ffd700 1 -- 172.21.15.107:0/1365282562 learned_addr learned my addr 172.21.15.107:0/1365282562 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:18.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.204+0000 7f92477fe700 1 --2- 172.21.15.107:0/1365282562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9248158bb0 0x7f924814f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:18.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.204+0000 7f92467fc700 1 -- 172.21.15.107:0/1365282562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92480cbe40 msgr2=0x7f92481f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:18.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.204+0000 7f92467fc700 1 --2- 172.21.15.107:0/1365282562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92480cbe40 0x7f92481f1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.204+0000 7f92467fc700 1 -- 172.21.15.107:0/1365282562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9248158bb0 msgr2=0x7f924814f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:18.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.204+0000 7f92467fc700 1 --2- 172.21.15.107:0/1365282562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9248158bb0 0x7f924814f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.204+0000 7f92467fc700 1 -- 172.21.15.107:0/1365282562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92380094a0 con 0x7f92480ce150 2023-07-18T03:38:18.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.204+0000 7f9246ffd700 1 --2- 172.21.15.107:0/1365282562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92480cbe40 0x7f92481f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:18.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.204+0000 7f92467fc700 1 --2- 172.21.15.107:0/1365282562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92480ce150 0x7f924814f1a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f923c001120 tx=0x7f923c00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:18.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.205+0000 7f922ffff700 1 -- 172.21.15.107:0/1365282562 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c005d70 con 0x7f92480ce150 2023-07-18T03:38:18.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.205+0000 7f922ffff700 1 -- 172.21.15.107:0/1365282562 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c00ce10 con 0x7f92480ce150 2023-07-18T03:38:18.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.205+0000 7f922ffff700 1 -- 172.21.15.107:0/1365282562 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c007310 con 0x7f92480ce150 2023-07-18T03:38:18.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.205+0000 7f924d9ff700 1 -- 172.21.15.107:0/1365282562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92480aee80 con 0x7f92480ce150 2023-07-18T03:38:18.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.205+0000 7f924d9ff700 1 -- 172.21.15.107:0/1365282562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92480af440 con 0x7f92480ce150 2023-07-18T03:38:18.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.205+0000 7f922ffff700 1 -- 172.21.15.107:0/1365282562 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f923c01c440 con 0x7f92480ce150 2023-07-18T03:38:18.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.205+0000 7f922ffff700 1 -- 172.21.15.107:0/1365282562 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f923c007770 con 0x7f92480ce150 2023-07-18T03:38:18.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.206+0000 7f924d9ff700 1 -- 172.21.15.107:0/1365282562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9248074350 con 0x7f92480ce150 2023-07-18T03:38:18.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.210+0000 7f922ffff700 1 -- 172.21.15.107:0/1365282562 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f923c017070 con 0x7f92480ce150 2023-07-18T03:38:18.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.297+0000 7f924d9ff700 1 -- 172.21.15.107:0/1365282562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "19421f1b-8499-4bea-9a1b-3be1118862fd", "id": 6} v 0) v1 -- 0x7f924815cf80 con 0x7f92480ce150 2023-07-18T03:38:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.302+0000 7f922ffff700 1 -- 172.21.15.107:0/1365282562 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "19421f1b-8499-4bea-9a1b-3be1118862fd", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f923c01c820 con 0x7f92480ce150 2023-07-18T03:38:18.298 INFO:teuthology.orchestra.run.smithi107.stdout:6 2023-07-18T03:38:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.303+0000 7f924d9ff700 1 -- 172.21.15.107:0/1365282562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92480ce150 msgr2=0x7f924814f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.303+0000 7f924d9ff700 1 --2- 172.21.15.107:0/1365282562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92480ce150 0x7f924814f1a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f923c001120 tx=0x7f923c00c740 comp rx=0 tx=0).stop 2023-07-18T03:38:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.303+0000 7f924d9ff700 1 -- 172.21.15.107:0/1365282562 shutdown_connections 2023-07-18T03:38:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.303+0000 7f924d9ff700 1 --2- 172.21.15.107:0/1365282562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9248158bb0 0x7f924814f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.303+0000 7f924d9ff700 1 --2- 172.21.15.107:0/1365282562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92480cbe40 0x7f92481f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.303+0000 7f924d9ff700 1 --2- 172.21.15.107:0/1365282562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92480ce150 0x7f924814f1a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.303+0000 7f924d9ff700 1 -- 172.21.15.107:0/1365282562 >> 172.21.15.107:0/1365282562 conn(0x7f92480b05d0 msgr2=0x7f92480cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.303+0000 7f924d9ff700 1 -- 172.21.15.107:0/1365282562 shutdown_connections 2023-07-18T03:38:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.303+0000 7f924d9ff700 1 -- 172.21.15.107:0/1365282562 wait complete. 2023-07-18T03:38:18.315 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new c1a3e5ec-b97b-473d-8044-08c46c82698b 7 2023-07-18T03:38:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.550+0000 7f9c0f9ff700 1 Processor -- start 2023-07-18T03:38:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.550+0000 7f9c0f9ff700 1 -- start start 2023-07-18T03:38:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.551+0000 7f9c0f9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9c080cf890 0x7f9c080b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.551+0000 7f9c0f9ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9c080cf030 0x7f9c080cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.551+0000 7f9c0f9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c080cd8e0 con 0x7f9c081601b0 2023-07-18T03:38:18.550 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.551+0000 7f9c0f9ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c080d2760 con 0x7f9c080cf030 2023-07-18T03:38:18.550 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.551+0000 7f9c0f9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c080d01a0 con 0x7f9c080cf890 2023-07-18T03:38:18.550 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.551+0000 7f9c0d79b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9c080cf890 0x7f9c080b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:18.550 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.551+0000 7f9c0d79b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9c080cf890 0x7f9c080b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47574/0 (socket says 172.21.15.107:47574) 2023-07-18T03:38:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.551+0000 7f9c0d79b700 1 -- 172.21.15.107:0/3417795629 learned_addr learned my addr 172.21.15.107:0/3417795629 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.551+0000 7f9c0cf9a700 1 --2- 172.21.15.107:0/3417795629 >> v2:172.21.15.49:3300/0 conn(0x7f9c080cf030 0x7f9c080cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.552+0000 7f9c0d79b700 1 -- 172.21.15.107:0/3417795629 >> v2:172.21.15.49:3300/0 conn(0x7f9c080cf030 msgr2=0x7f9c080cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.552+0000 7f9c0d79b700 1 --2- 172.21.15.107:0/3417795629 >> v2:172.21.15.49:3300/0 conn(0x7f9c080cf030 0x7f9c080cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.552+0000 7f9c0d79b700 1 -- 172.21.15.107:0/3417795629 >> v1:172.21.15.107:6789/0 conn(0x7f9c081601b0 legacy=0x7f9c081605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:18.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.552+0000 7f9c0d79b700 1 -- 172.21.15.107:0/3417795629 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c080d1740 con 0x7f9c080cf890 2023-07-18T03:38:18.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.552+0000 7f9c0d79b700 1 --2- 172.21.15.107:0/3417795629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c080cf890 0x7f9c080b21f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c080cf860 tx=0x7f9c04009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e754605a0bb5a770 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:18.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.552+0000 7f9bfffff700 1 -- 172.21.15.107:0/3417795629 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04011830 con 0x7f9c080cf890 2023-07-18T03:38:18.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.552+0000 7f9bfffff700 1 -- 172.21.15.107:0/3417795629 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c04011990 con 0x7f9c080cf890 2023-07-18T03:38:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.552+0000 7f9bfffff700 1 -- 172.21.15.107:0/3417795629 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04011d20 con 0x7f9c080cf890 2023-07-18T03:38:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.553+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/3417795629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c080cf890 msgr2=0x7f9c080b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.553+0000 7f9c0f9ff700 1 --2- 172.21.15.107:0/3417795629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c080cf890 0x7f9c080b21f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c080cf860 tx=0x7f9c04009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.553+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/3417795629 shutdown_connections 2023-07-18T03:38:18.554 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.553+0000 7f9c0f9ff700 1 --2- 172.21.15.107:0/3417795629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c080cf890 0x7f9c080b21f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.554 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.553+0000 7f9c0f9ff700 1 --2- 172.21.15.107:0/3417795629 >> v2:172.21.15.49:3300/0 conn(0x7f9c080cf030 0x7f9c080cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.554 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.553+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/3417795629 >> 172.21.15.107:0/3417795629 conn(0x7f9c0806c170 msgr2=0x7f9c0806c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:18.554 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.553+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/3417795629 shutdown_connections 2023-07-18T03:38:18.555 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.553+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/3417795629 wait complete. 2023-07-18T03:38:18.558 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.555+0000 7f9c0f9ff700 1 Processor -- start 2023-07-18T03:38:18.558 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.555+0000 7f9c0f9ff700 1 -- start start 2023-07-18T03:38:18.558 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0f9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c080cf030 0x7f9c080b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:18.558 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0f9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c080cf890 0x7f9c0815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:18.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0f9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c081601b0 0x7f9c080b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:18.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0f9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c081f2640 con 0x7f9c081601b0 2023-07-18T03:38:18.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0f9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080ac150 con 0x7f9c080cf030 2023-07-18T03:38:18.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0f9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080a9800 con 0x7f9c080cf890 2023-07-18T03:38:18.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0d79b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c080cf030 0x7f9c080b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:18.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0cf9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c080cf890 0x7f9c0815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:18.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0d79b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c080cf030 0x7f9c080b1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47588/0 (socket says 172.21.15.107:47588) 2023-07-18T03:38:18.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0d79b700 1 -- 172.21.15.107:0/339482794 learned_addr learned my addr 172.21.15.107:0/339482794 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-18T03:38:18.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0df9c700 1 --2- 172.21.15.107:0/339482794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c081601b0 0x7f9c080b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:18.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0d79b700 1 -- 172.21.15.107:0/339482794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c080cf890 msgr2=0x7f9c0815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:18.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0d79b700 1 --2- 172.21.15.107:0/339482794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c080cf890 0x7f9c0815adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0d79b700 1 -- 172.21.15.107:0/339482794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c081601b0 msgr2=0x7f9c080b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:18.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0d79b700 1 --2- 172.21.15.107:0/339482794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c081601b0 0x7f9c080b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.557+0000 7f9c0d79b700 1 -- 172.21.15.107:0/339482794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c04009550 con 0x7f9c080cf030 2023-07-18T03:38:18.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.558+0000 7f9c0d79b700 1 --2- 172.21.15.107:0/339482794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c080cf030 0x7f9c080b1370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009c10 tx=0x7f9c04004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:18.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.558+0000 7f9bfdffb700 1 -- 172.21.15.107:0/339482794 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04004380 con 0x7f9c080cf030 2023-07-18T03:38:18.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.558+0000 7f9bfdffb700 1 -- 172.21.15.107:0/339482794 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0401f070 con 0x7f9c080cf030 2023-07-18T03:38:18.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.558+0000 7f9bfdffb700 1 -- 172.21.15.107:0/339482794 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c040036a0 con 0x7f9c080cf030 2023-07-18T03:38:18.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.558+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/339482794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c080a9aa0 con 0x7f9c080cf030 2023-07-18T03:38:18.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.558+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/339482794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c080abb90 con 0x7f9c080cf030 2023-07-18T03:38:18.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.558+0000 7f9bfdffb700 1 -- 172.21.15.107:0/339482794 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9c04021030 con 0x7f9c080cf030 2023-07-18T03:38:18.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.558+0000 7f9bfdffb700 1 -- 172.21.15.107:0/339482794 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f9c0401a450 con 0x7f9c080cf030 2023-07-18T03:38:18.563 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.559+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/339482794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c08067350 con 0x7f9c080cf030 2023-07-18T03:38:18.563 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.562+0000 7f9bfdffb700 1 -- 172.21.15.107:0/339482794 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f9c04044b20 con 0x7f9c080cf030 2023-07-18T03:38:18.611 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:18.615+0000 7f2738a92200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T03:38:18.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.648+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/339482794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c1a3e5ec-b97b-473d-8044-08c46c82698b", "id": 7} v 0) v1 -- 0x7f9c080b5230 con 0x7f9c080cf030 2023-07-18T03:38:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.653+0000 7f9bfdffb700 1 -- 172.21.15.107:0/339482794 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c1a3e5ec-b97b-473d-8044-08c46c82698b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9c04033070 con 0x7f9c080cf030 2023-07-18T03:38:18.650 INFO:teuthology.orchestra.run.smithi107.stdout:7 2023-07-18T03:38:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.654+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/339482794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c080cf030 msgr2=0x7f9c080b1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.654+0000 7f9c0f9ff700 1 --2- 172.21.15.107:0/339482794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c080cf030 0x7f9c080b1370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009c10 tx=0x7f9c04004750 comp rx=0 tx=0).stop 2023-07-18T03:38:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.654+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/339482794 shutdown_connections 2023-07-18T03:38:18.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.654+0000 7f9c0f9ff700 1 --2- 172.21.15.107:0/339482794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c081601b0 0x7f9c080b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.654+0000 7f9c0f9ff700 1 --2- 172.21.15.107:0/339482794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c080cf030 0x7f9c080b1370 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.654+0000 7f9c0f9ff700 1 --2- 172.21.15.107:0/339482794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c080cf890 0x7f9c0815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.654+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/339482794 >> 172.21.15.107:0/339482794 conn(0x7f9c0806c170 msgr2=0x7f9c080b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:18.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.654+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/339482794 shutdown_connections 2023-07-18T03:38:18.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-18T03:38:18.654+0000 7f9c0f9ff700 1 -- 172.21.15.107:0/339482794 wait complete. 2023-07-18T03:38:18.668 INFO:tasks.ceph.osd.0:Restarting daemon 2023-07-18T03:38:18.668 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-07-18T03:38:18.672 INFO:tasks.ceph.osd.0:Started 2023-07-18T03:38:18.673 INFO:tasks.ceph.osd.1:Restarting daemon 2023-07-18T03:38:18.673 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-07-18T03:38:18.678 INFO:tasks.ceph.osd.1:Started 2023-07-18T03:38:18.678 INFO:tasks.ceph.osd.2:Restarting daemon 2023-07-18T03:38:18.678 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-07-18T03:38:18.684 INFO:tasks.ceph.osd.2:Started 2023-07-18T03:38:18.684 INFO:tasks.ceph.osd.3:Restarting daemon 2023-07-18T03:38:18.684 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-07-18T03:38:18.690 INFO:tasks.ceph.osd.3:Started 2023-07-18T03:38:18.691 INFO:tasks.ceph.osd.4:Restarting daemon 2023-07-18T03:38:18.692 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-07-18T03:38:18.698 INFO:tasks.ceph.osd.4:Started 2023-07-18T03:38:18.699 INFO:tasks.ceph.osd.5:Restarting daemon 2023-07-18T03:38:18.699 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-07-18T03:38:18.705 INFO:tasks.ceph.osd.5:Started 2023-07-18T03:38:18.705 INFO:tasks.ceph.osd.6:Restarting daemon 2023-07-18T03:38:18.706 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-07-18T03:38:18.711 INFO:tasks.ceph.osd.6:Started 2023-07-18T03:38:18.712 INFO:tasks.ceph.osd.7:Restarting daemon 2023-07-18T03:38:18.712 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-07-18T03:38:18.716 INFO:tasks.ceph.osd.7:Started 2023-07-18T03:38:18.717 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:18.859 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:18.862+0000 7f2738a92200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T03:38:18.943 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:18.948+0000 7f465a047200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:38:18.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.994+0000 7fb3939ff700 1 Processor -- start 2023-07-18T03:38:18.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.994+0000 7fb3939ff700 1 -- start start 2023-07-18T03:38:18.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.995+0000 7fb3939ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb38c15c950 0x7fb38c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:18.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.995+0000 7fb3939ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb38c0b1e00 0x7fb38c0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:18.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.995+0000 7fb3939ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb38c0b2350 0x7fb38c0d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:18.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.995+0000 7fb3939ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38c0d2ce0 con 0x7fb38c15c950 2023-07-18T03:38:18.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.995+0000 7fb3939ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38c0b3350 con 0x7fb38c0b1e00 2023-07-18T03:38:18.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.995+0000 7fb3939ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0b3630 con 0x7fb38c0b2350 2023-07-18T03:38:18.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.996+0000 7fb39179b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb38c15c950 0x7fb38c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:18.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.996+0000 7fb39179b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb38c15c950 0x7fb38c0b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60176/0 (socket says 172.21.15.49:60176) 2023-07-18T03:38:18.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.996+0000 7fb39179b700 1 -- 172.21.15.49:0/3135918828 learned_addr learned my addr 172.21.15.49:0/3135918828 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:18.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.996+0000 7fb390f9a700 1 --2- 172.21.15.49:0/3135918828 >> v2:172.21.15.107:3300/0 conn(0x7fb38c0b1e00 0x7fb38c0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:18.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.996+0000 7fb391f9c700 1 --2- 172.21.15.49:0/3135918828 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb38c0b2350 0x7fb38c0d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:18.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.996+0000 7fb39179b700 1 -- 172.21.15.49:0/3135918828 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb38c0b2350 msgr2=0x7fb38c0d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:18.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.996+0000 7fb39179b700 1 --2- 172.21.15.49:0/3135918828 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb38c0b2350 0x7fb38c0d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.996+0000 7fb39179b700 1 -- 172.21.15.49:0/3135918828 >> v2:172.21.15.107:3300/0 conn(0x7fb38c0b1e00 msgr2=0x7fb38c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:18.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.996+0000 7fb39179b700 1 --2- 172.21.15.49:0/3135918828 >> v2:172.21.15.107:3300/0 conn(0x7fb38c0b1e00 0x7fb38c0d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.996+0000 7fb39179b700 1 -- 172.21.15.49:0/3135918828 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c0b3910 con 0x7fb38c15c950 2023-07-18T03:38:18.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.996+0000 7fb39179b700 1 --2- 172.21.15.49:0/3135918828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb38c15c950 0x7fb38c0b43f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb38c072140 tx=0x7fb388009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c2a58182f8ed3c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:18.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.997+0000 7fb383fff700 1 -- 172.21.15.49:0/3135918828 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388005e30 con 0x7fb38c15c950 2023-07-18T03:38:18.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.997+0000 7fb383fff700 1 -- 172.21.15.49:0/3135918828 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb388004030 con 0x7fb38c15c950 2023-07-18T03:38:18.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.997+0000 7fb383fff700 1 -- 172.21.15.49:0/3135918828 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3880043e0 con 0x7fb38c15c950 2023-07-18T03:38:18.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.997+0000 7fb3939ff700 1 -- 172.21.15.49:0/3135918828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb38c15c950 msgr2=0x7fb38c0b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:18.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.997+0000 7fb3939ff700 1 --2- 172.21.15.49:0/3135918828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb38c15c950 0x7fb38c0b43f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb38c072140 tx=0x7fb388009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.997+0000 7fb3939ff700 1 -- 172.21.15.49:0/3135918828 shutdown_connections 2023-07-18T03:38:18.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.997+0000 7fb3939ff700 1 --2- 172.21.15.49:0/3135918828 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb38c0b2350 0x7fb38c0d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.997+0000 7fb3939ff700 1 --2- 172.21.15.49:0/3135918828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb38c15c950 0x7fb38c0b43f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.997+0000 7fb3939ff700 1 --2- 172.21.15.49:0/3135918828 >> v2:172.21.15.107:3300/0 conn(0x7fb38c0b1e00 0x7fb38c0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:18.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.997+0000 7fb3939ff700 1 -- 172.21.15.49:0/3135918828 >> 172.21.15.49:0/3135918828 conn(0x7fb38c09c1d0 msgr2=0x7fb38c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:18.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.998+0000 7fb3939ff700 1 -- 172.21.15.49:0/3135918828 shutdown_connections 2023-07-18T03:38:18.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:18.998+0000 7fb3939ff700 1 -- 172.21.15.49:0/3135918828 wait complete. 2023-07-18T03:38:19.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.001+0000 7fb3939ff700 1 Processor -- start 2023-07-18T03:38:19.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.001+0000 7fb3939ff700 1 -- start start 2023-07-18T03:38:19.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb3939ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb38c0b1e00 0x7fb38c0b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:19.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb3939ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb38c0b2350 0x7fb38c0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:19.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb3939ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb38c15c950 0x7fb38c0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:19.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb3939ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0b5d70 con 0x7fb38c15c950 2023-07-18T03:38:19.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb3939ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0c83e0 con 0x7fb38c0b2350 2023-07-18T03:38:19.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb3939ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0c86c0 con 0x7fb38c0b1e00 2023-07-18T03:38:19.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb391f9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb38c15c950 0x7fb38c0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:19.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb391f9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb38c15c950 0x7fb38c0cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60190/0 (socket says 172.21.15.49:60190) 2023-07-18T03:38:19.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb391f9c700 1 -- 172.21.15.49:0/3977908008 learned_addr learned my addr 172.21.15.49:0/3977908008 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:19.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb39179b700 1 --2- 172.21.15.49:0/3977908008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb38c0b1e00 0x7fb38c0b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:19.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb390f9a700 1 --2- 172.21.15.49:0/3977908008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb38c0b2350 0x7fb38c0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:19.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb391f9c700 1 -- 172.21.15.49:0/3977908008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb38c0b1e00 msgr2=0x7fb38c0b7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:19.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb391f9c700 1 --2- 172.21.15.49:0/3977908008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb38c0b1e00 0x7fb38c0b7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:19.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb391f9c700 1 -- 172.21.15.49:0/3977908008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb38c0b2350 msgr2=0x7fb38c0c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:19.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb391f9c700 1 --2- 172.21.15.49:0/3977908008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb38c0b2350 0x7fb38c0c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:19.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.002+0000 7fb391f9c700 1 -- 172.21.15.49:0/3977908008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb388009500 con 0x7fb38c15c950 2023-07-18T03:38:19.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.003+0000 7fb391f9c700 1 --2- 172.21.15.49:0/3977908008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb38c15c950 0x7fb38c0cd570 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb38400efc0 tx=0x7fb38400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:19.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.003+0000 7fb381ffb700 1 -- 172.21.15.49:0/3977908008 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384009ea0 con 0x7fb38c15c950 2023-07-18T03:38:19.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.003+0000 7fb381ffb700 1 -- 172.21.15.49:0/3977908008 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38400f040 con 0x7fb38c15c950 2023-07-18T03:38:19.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.003+0000 7fb3939ff700 1 -- 172.21.15.49:0/3977908008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c0c89a0 con 0x7fb38c15c950 2023-07-18T03:38:19.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.003+0000 7fb381ffb700 1 -- 172.21.15.49:0/3977908008 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38400ced0 con 0x7fb38c15c950 2023-07-18T03:38:19.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.003+0000 7fb3939ff700 1 -- 172.21.15.49:0/3977908008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c0c7ff0 con 0x7fb38c15c950 2023-07-18T03:38:19.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.003+0000 7fb381ffb700 1 -- 172.21.15.49:0/3977908008 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb384014780 con 0x7fb38c15c950 2023-07-18T03:38:19.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.003+0000 7fb381ffb700 1 -- 172.21.15.49:0/3977908008 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb38401d400 con 0x7fb38c15c950 2023-07-18T03:38:19.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.004+0000 7fb3939ff700 1 -- 172.21.15.49:0/3977908008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c0b6200 con 0x7fb38c15c950 2023-07-18T03:38:19.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.007+0000 7fb381ffb700 1 -- 172.21.15.49:0/3977908008 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fb38403bb20 con 0x7fb38c15c950 2023-07-18T03:38:19.042 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:19.047+0000 7f465a047200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T03:38:19.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.081+0000 7fb3939ff700 1 -- 172.21.15.49:0/3977908008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb38c0c8180 con 0x7fb38c15c950 2023-07-18T03:38:19.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.081+0000 7fb381ffb700 1 -- 172.21.15.49:0/3977908008 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb38402a070 con 0x7fb38c15c950 2023-07-18T03:38:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":10,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:18.650495+0000","last_up_change":"0.000000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:19.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.082+0000 7fb3939ff700 1 -- 172.21.15.49:0/3977908008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb38c15c950 msgr2=0x7fb38c0cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:19.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.082+0000 7fb3939ff700 1 --2- 172.21.15.49:0/3977908008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb38c15c950 0x7fb38c0cd570 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb38400efc0 tx=0x7fb38400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:38:19.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.083+0000 7fb3939ff700 1 -- 172.21.15.49:0/3977908008 shutdown_connections 2023-07-18T03:38:19.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.083+0000 7fb3939ff700 1 --2- 172.21.15.49:0/3977908008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb38c15c950 0x7fb38c0cd570 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:19.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.083+0000 7fb3939ff700 1 --2- 172.21.15.49:0/3977908008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb38c0b1e00 0x7fb38c0b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:19.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.083+0000 7fb3939ff700 1 --2- 172.21.15.49:0/3977908008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb38c0b2350 0x7fb38c0c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:19.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.083+0000 7fb3939ff700 1 -- 172.21.15.49:0/3977908008 >> 172.21.15.49:0/3977908008 conn(0x7fb38c09c1d0 msgr2=0x7fb38c0b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:19.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.083+0000 7fb3939ff700 1 -- 172.21.15.49:0/3977908008 shutdown_connections 2023-07-18T03:38:19.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:19.083+0000 7fb3939ff700 1 -- 172.21.15.49:0/3977908008 wait complete. 2023-07-18T03:38:19.097 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-07-18T03:38:19.097 INFO:tasks.ceph:Waiting for OSDs to come up 2023-07-18T03:38:19.221 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:19.225+0000 7f465a047200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:38:19.389 INFO:tasks.ceph.osd.4.smithi107.stderr:2023-07-18T03:38:19.394+0000 7fc8dd597640 -1 Falling back to public interface 2023-07-18T03:38:19.403 INFO:tasks.ceph.osd.0.smithi049.stderr:2023-07-18T03:38:19.407+0000 7f9ac5080640 -1 Falling back to public interface 2023-07-18T03:38:19.407 INFO:tasks.ceph.osd.6.smithi107.stderr:2023-07-18T03:38:19.412+0000 7f675856c640 -1 Falling back to public interface 2023-07-18T03:38:19.416 INFO:tasks.ceph.osd.3.smithi049.stderr:2023-07-18T03:38:19.420+0000 7f2e1f497640 -1 Falling back to public interface 2023-07-18T03:38:19.420 INFO:tasks.ceph.osd.1.smithi049.stderr:2023-07-18T03:38:19.424+0000 7f871195d640 -1 Falling back to public interface 2023-07-18T03:38:19.440 INFO:tasks.ceph.osd.2.smithi049.stderr:2023-07-18T03:38:19.444+0000 7f5dfdec2640 -1 Falling back to public interface 2023-07-18T03:38:19.441 INFO:tasks.ceph.osd.7.smithi107.stderr:2023-07-18T03:38:19.446+0000 7fdf900a4640 -1 Falling back to public interface 2023-07-18T03:38:19.442 INFO:tasks.ceph.osd.5.smithi107.stderr:2023-07-18T03:38:19.447+0000 7f6adb2de640 -1 Falling back to public interface 2023-07-18T03:38:19.526 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:19.531+0000 7f465a047200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T03:38:19.622 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:19.627+0000 7f465a047200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T03:38:19.757 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:19.761+0000 7f2738a92200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:38:19.859 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:19.863+0000 7f2738a92200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T03:38:19.902 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:19.954 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:19.959+0000 7f465a047200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T03:38:20.051 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:20.055+0000 7f2738a92200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:38:20.082 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.086+0000 7f7c113ff700 1 Processor -- start 2023-07-18T03:38:20.082 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.086+0000 7f7c113ff700 1 -- start start 2023-07-18T03:38:20.083 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.087+0000 7f7c113ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7c0c15c710 0x7f7c0c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:20.083 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.087+0000 7f7c113ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c0c069f30 0x7f7c0c06a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:20.083 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.087+0000 7f7c113ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c0c06d420 0x7f7c0c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:20.084 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.087+0000 7f7c113ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0c06dde0 con 0x7f7c0c06d420 2023-07-18T03:38:20.084 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.087+0000 7f7c113ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0c0cd320 con 0x7f7c0c15c710 2023-07-18T03:38:20.084 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.087+0000 7f7c113ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c0ce750 con 0x7f7c0c069f30 2023-07-18T03:38:20.084 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.087+0000 7f7c0a7fc700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c0c069f30 0x7f7c0c06a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:20.084 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.087+0000 7f7c0affd700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7c0c15c710 0x7f7c0c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:20.084 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.087+0000 7f7c0b7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c0c06d420 0x7f7c0c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:20.085 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.088+0000 7f7c0affd700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7c0c15c710 0x7f7c0c0cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:56352/0 (socket says 172.21.15.49:56352) 2023-07-18T03:38:20.085 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.088+0000 7f7c0a7fc700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c0c069f30 0x7f7c0c06a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:44624/0 (socket says 172.21.15.49:44624) 2023-07-18T03:38:20.085 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.088+0000 7f7c0affd700 1 -- 172.21.15.49:0/3539118432 learned_addr learned my addr 172.21.15.49:0/3539118432 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:20.087 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.088+0000 7f7c0affd700 1 -- 172.21.15.49:0/3539118432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c0c069f30 msgr2=0x7f7c0c06a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:20.087 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.088+0000 7f7c0affd700 1 --2- 172.21.15.49:0/3539118432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c0c069f30 0x7f7c0c06a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:20.087 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.088+0000 7f7c0affd700 1 -- 172.21.15.49:0/3539118432 >> v2:172.21.15.49:3300/0 conn(0x7f7c0c06d420 msgr2=0x7f7c0c06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:20.088 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.088+0000 7f7c0affd700 1 --2- 172.21.15.49:0/3539118432 >> v2:172.21.15.49:3300/0 conn(0x7f7c0c06d420 0x7f7c0c06d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:20.088 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.088+0000 7f7c0affd700 1 -- 172.21.15.49:0/3539118432 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c0cea30 con 0x7f7c0c15c710 2023-07-18T03:38:20.088 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.088+0000 7f7c0affd700 1 --2- 172.21.15.49:0/3539118432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c0c15c710 0x7f7c0c0cf510 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c072690 tx=0x7f7bfc009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba4e5b0a733f7cd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:20.088 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.088+0000 7f7c09ffb700 1 -- 172.21.15.49:0/3539118432 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc011830 con 0x7f7c0c15c710 2023-07-18T03:38:20.088 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.089+0000 7f7c09ffb700 1 -- 172.21.15.49:0/3539118432 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bfc011990 con 0x7f7c0c15c710 2023-07-18T03:38:20.088 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.089+0000 7f7c09ffb700 1 -- 172.21.15.49:0/3539118432 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc011d20 con 0x7f7c0c15c710 2023-07-18T03:38:20.089 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.089+0000 7f7c113ff700 1 -- 172.21.15.49:0/3539118432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c0c15c710 msgr2=0x7f7c0c0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:20.089 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.089+0000 7f7c113ff700 1 --2- 172.21.15.49:0/3539118432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c0c15c710 0x7f7c0c0cf510 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c072690 tx=0x7f7bfc009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:20.089 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.089+0000 7f7c113ff700 1 -- 172.21.15.49:0/3539118432 shutdown_connections 2023-07-18T03:38:20.089 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.089+0000 7f7c113ff700 1 --2- 172.21.15.49:0/3539118432 >> v2:172.21.15.49:3300/0 conn(0x7f7c0c06d420 0x7f7c0c06d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:20.089 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.089+0000 7f7c113ff700 1 --2- 172.21.15.49:0/3539118432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c0c15c710 0x7f7c0c0cf510 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:20.089 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.089+0000 7f7c113ff700 1 --2- 172.21.15.49:0/3539118432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c0c069f30 0x7f7c0c06a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:20.089 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.089+0000 7f7c113ff700 1 -- 172.21.15.49:0/3539118432 >> 172.21.15.49:0/3539118432 conn(0x7f7c0c0b16c0 msgr2=0x7f7c0c0b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:20.089 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.089+0000 7f7c113ff700 1 -- 172.21.15.49:0/3539118432 shutdown_connections 2023-07-18T03:38:20.090 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.089+0000 7f7c113ff700 1 -- 172.21.15.49:0/3539118432 wait complete. 2023-07-18T03:38:20.090 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.091+0000 7f7c113ff700 1 Processor -- start 2023-07-18T03:38:20.091 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.091+0000 7f7c113ff700 1 -- start start 2023-07-18T03:38:20.092 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.092+0000 7f7c113ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c0c069f30 0x7f7c0c0b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:20.092 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.092+0000 7f7c113ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c0c06d420 0x7f7c0c15a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:20.092 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.092+0000 7f7c113ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c0c0b3b30 0x7f7c0c14edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:20.092 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.092+0000 7f7c113ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c0b5a50 con 0x7f7c0c069f30 2023-07-18T03:38:20.092 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.092+0000 7f7c113ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c15deb0 con 0x7f7c0c0b3b30 2023-07-18T03:38:20.092 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.092+0000 7f7c113ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c15e190 con 0x7f7c0c06d420 2023-07-18T03:38:20.093 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.093+0000 7f7c0affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c0c069f30 0x7f7c0c0b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:20.093 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.093+0000 7f7c0a7fc700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c0c06d420 0x7f7c0c15a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:20.093 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.093+0000 7f7c0affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c0c069f30 0x7f7c0c0b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60212/0 (socket says 172.21.15.49:60212) 2023-07-18T03:38:20.093 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.093+0000 7f7c0affd700 1 -- 172.21.15.49:0/3787433083 learned_addr learned my addr 172.21.15.49:0/3787433083 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:20.093 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.093+0000 7f7c0b7fe700 1 --2- 172.21.15.49:0/3787433083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c0c0b3b30 0x7f7c0c14edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:20.093 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.093+0000 7f7c0affd700 1 -- 172.21.15.49:0/3787433083 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c0c06d420 msgr2=0x7f7c0c15a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:20.094 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.093+0000 7f7c0affd700 1 --2- 172.21.15.49:0/3787433083 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c0c06d420 0x7f7c0c15a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:20.094 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.093+0000 7f7c0affd700 1 -- 172.21.15.49:0/3787433083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c0c0b3b30 msgr2=0x7f7c0c14edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:20.094 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.093+0000 7f7c0affd700 1 --2- 172.21.15.49:0/3787433083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c0c0b3b30 0x7f7c0c14edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:20.094 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.093+0000 7f7c0affd700 1 -- 172.21.15.49:0/3787433083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc009550 con 0x7f7c0c069f30 2023-07-18T03:38:20.094 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.093+0000 7f7c0affd700 1 --2- 172.21.15.49:0/3787433083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c0c069f30 0x7f7c0c0b5510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009c10 tx=0x7f7bfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:20.094 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.094+0000 7f7bf37fe700 1 -- 172.21.15.49:0/3787433083 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc004450 con 0x7f7c0c069f30 2023-07-18T03:38:20.094 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.094+0000 7f7bf37fe700 1 -- 172.21.15.49:0/3787433083 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bfc01e070 con 0x7f7c0c069f30 2023-07-18T03:38:20.095 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.094+0000 7f7bf37fe700 1 -- 172.21.15.49:0/3787433083 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc0038e0 con 0x7f7c0c069f30 2023-07-18T03:38:20.095 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.094+0000 7f7c113ff700 1 -- 172.21.15.49:0/3787433083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0c15e470 con 0x7f7c0c069f30 2023-07-18T03:38:20.095 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.094+0000 7f7c113ff700 1 -- 172.21.15.49:0/3787433083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0c0b3fc0 con 0x7f7c0c069f30 2023-07-18T03:38:20.095 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.094+0000 7f7bf37fe700 1 -- 172.21.15.49:0/3787433083 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7bfc003d40 con 0x7f7c0c069f30 2023-07-18T03:38:20.095 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.094+0000 7f7bf37fe700 1 -- 172.21.15.49:0/3787433083 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f7bfc01aaf0 con 0x7f7c0c069f30 2023-07-18T03:38:20.095 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.095+0000 7f7c113ff700 1 -- 172.21.15.49:0/3787433083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0c15e990 con 0x7f7c0c069f30 2023-07-18T03:38:20.096 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.098+0000 7f7bf37fe700 1 -- 172.21.15.49:0/3787433083 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f7bfc020070 con 0x7f7c0c069f30 2023-07-18T03:38:20.168 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.172+0000 7f7c113ff700 1 -- 172.21.15.49:0/3787433083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c0c04e280 con 0x7f7c0c069f30 2023-07-18T03:38:20.169 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.173+0000 7f7bf37fe700 1 -- 172.21.15.49:0/3787433083 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f7bfc015020 con 0x7f7c0c069f30 2023-07-18T03:38:20.169 INFO:teuthology.misc.health.smithi049.stdout: 2023-07-18T03:38:20.169 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":10,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:18.650495+0000","last_up_change":"0.000000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:20.170 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.174+0000 7f7c113ff700 1 -- 172.21.15.49:0/3787433083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c0c069f30 msgr2=0x7f7c0c0b5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:20.170 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.174+0000 7f7c113ff700 1 --2- 172.21.15.49:0/3787433083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c0c069f30 0x7f7c0c0b5510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009c10 tx=0x7f7bfc004750 comp rx=0 tx=0).stop 2023-07-18T03:38:20.171 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.174+0000 7f7c113ff700 1 -- 172.21.15.49:0/3787433083 shutdown_connections 2023-07-18T03:38:20.171 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.174+0000 7f7c113ff700 1 --2- 172.21.15.49:0/3787433083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c0c0b3b30 0x7f7c0c14edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:20.171 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.174+0000 7f7c113ff700 1 --2- 172.21.15.49:0/3787433083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c0c069f30 0x7f7c0c0b5510 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:20.171 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.174+0000 7f7c113ff700 1 --2- 172.21.15.49:0/3787433083 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c0c06d420 0x7f7c0c15a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:20.171 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.174+0000 7f7c113ff700 1 -- 172.21.15.49:0/3787433083 >> 172.21.15.49:0/3787433083 conn(0x7f7c0c0b16c0 msgr2=0x7f7c0c0b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:20.172 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.174+0000 7f7c113ff700 1 -- 172.21.15.49:0/3787433083 shutdown_connections 2023-07-18T03:38:20.172 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:20.174+0000 7f7c113ff700 1 -- 172.21.15.49:0/3787433083 wait complete. 2023-07-18T03:38:20.184 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-18T03:38:20.233 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:20.237+0000 7f465a047200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T03:38:20.367 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:20.371+0000 7f2738a92200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T03:38:20.434 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:20.439+0000 7f465a047200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T03:38:20.468 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:20.472+0000 7f2738a92200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T03:38:20.582 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:20.586+0000 7f465a047200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T03:38:20.768 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:20.772+0000 7f465a047200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T03:38:20.815 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:20.819+0000 7f2738a92200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T03:38:21.100 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:21.104+0000 7f465a047200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T03:38:21.104 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:21.108+0000 7f2738a92200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T03:38:21.313 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:21.318+0000 7f2738a92200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T03:38:21.330 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:21.335+0000 7f465a047200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T03:38:21.468 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:21.472+0000 7f2738a92200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T03:38:21.663 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:21.667+0000 7f2738a92200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T03:38:22.016 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:22.020+0000 7f2738a92200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T03:38:22.076 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:22.080+0000 7f465a047200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T03:38:22.283 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:22.287+0000 7f2738a92200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T03:38:22.953 INFO:tasks.ceph.osd.4.smithi107.stderr:2023-07-18T03:38:22.958+0000 7fc8dd597640 -1 osd.4 0 log_to_monitors true 2023-07-18T03:38:22.995 INFO:tasks.ceph.osd.6.smithi107.stderr:2023-07-18T03:38:23.000+0000 7f675856c640 -1 osd.6 0 log_to_monitors true 2023-07-18T03:38:23.006 INFO:tasks.ceph.osd.7.smithi107.stderr:2023-07-18T03:38:23.010+0000 7fdf900a4640 -1 osd.7 0 log_to_monitors true 2023-07-18T03:38:23.015 INFO:tasks.ceph.osd.5.smithi107.stderr:2023-07-18T03:38:23.020+0000 7f6adb2de640 -1 osd.5 0 log_to_monitors true 2023-07-18T03:38:23.027 INFO:tasks.ceph.osd.3.smithi049.stderr:2023-07-18T03:38:23.031+0000 7f2e1f497640 -1 osd.3 0 log_to_monitors true 2023-07-18T03:38:23.043 INFO:tasks.ceph.osd.1.smithi049.stderr:2023-07-18T03:38:23.047+0000 7f871195d640 -1 osd.1 0 log_to_monitors true 2023-07-18T03:38:23.093 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:23.098+0000 7f460dbfa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-18T03:38:23.094 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:38:23.098+0000 7f460dbfa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-18T03:38:23.111 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-07-18T03:38:23.115+0000 7f2738a92200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T03:38:23.232 INFO:tasks.ceph.osd.0.smithi049.stderr:2023-07-18T03:38:23.236+0000 7f9ac5080640 -1 osd.0 0 log_to_monitors true 2023-07-18T03:38:23.294 INFO:tasks.ceph.osd.2.smithi049.stderr:2023-07-18T03:38:23.299+0000 7f5dfdec2640 -1 osd.2 0 log_to_monitors true 2023-07-18T03:38:25.386 INFO:tasks.ceph.osd.1.smithi049.stderr:2023-07-18T03:38:25.389+0000 7f86ffff6700 -1 osd.1 0 waiting for initial osdmap 2023-07-18T03:38:25.395 INFO:tasks.ceph.osd.3.smithi049.stderr:2023-07-18T03:38:25.399+0000 7f2e0dbf6700 -1 osd.3 0 waiting for initial osdmap 2023-07-18T03:38:25.408 INFO:tasks.ceph.osd.1.smithi049.stderr:2023-07-18T03:38:25.412+0000 7f86f7ffb700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:38:25.433 INFO:tasks.ceph.osd.3.smithi049.stderr:2023-07-18T03:38:25.437+0000 7f2e05bfb700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:38:26.106 INFO:tasks.ceph.osd.5.smithi107.stderr:2023-07-18T03:38:26.110+0000 7f6ac99f6700 -1 osd.5 0 waiting for initial osdmap 2023-07-18T03:38:26.130 INFO:tasks.ceph.osd.5.smithi107.stderr:2023-07-18T03:38:26.134+0000 7f6ac19fb700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:38:26.208 INFO:tasks.ceph.osd.4.smithi107.stderr:2023-07-18T03:38:26.212+0000 7fc8cbbf6700 -1 osd.4 0 waiting for initial osdmap 2023-07-18T03:38:26.226 INFO:tasks.ceph.osd.4.smithi107.stderr:2023-07-18T03:38:26.231+0000 7fc8c3bfb700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:38:26.290 INFO:tasks.ceph.osd.7.smithi107.stderr:2023-07-18T03:38:26.295+0000 7fdf7e7f6700 -1 osd.7 0 waiting for initial osdmap 2023-07-18T03:38:26.311 INFO:tasks.ceph.osd.6.smithi107.stderr:2023-07-18T03:38:26.315+0000 7f6746bf6700 -1 osd.6 0 waiting for initial osdmap 2023-07-18T03:38:26.314 INFO:tasks.ceph.osd.7.smithi107.stderr:2023-07-18T03:38:26.319+0000 7fdf767fb700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:38:26.332 INFO:tasks.ceph.osd.6.smithi107.stderr:2023-07-18T03:38:26.337+0000 7f673ebfb700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:38:26.406 INFO:tasks.ceph.osd.2.smithi049.stderr:2023-07-18T03:38:26.410+0000 7f5dec5f6700 -1 osd.2 0 waiting for initial osdmap 2023-07-18T03:38:26.414 INFO:tasks.ceph.osd.0.smithi049.stderr:2023-07-18T03:38:26.418+0000 7f9ab37f6700 -1 osd.0 0 waiting for initial osdmap 2023-07-18T03:38:26.423 INFO:tasks.ceph.osd.2.smithi049.stderr:2023-07-18T03:38:26.427+0000 7f5de45fb700 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:38:26.431 INFO:tasks.ceph.osd.0.smithi049.stderr:2023-07-18T03:38:26.435+0000 7f9aab7fb700 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:38:26.990 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:27.180 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.184+0000 7f1b043ff700 1 Processor -- start 2023-07-18T03:38:27.180 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.184+0000 7f1b043ff700 1 -- start start 2023-07-18T03:38:27.183 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.185+0000 7f1b043ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1afc15c950 0x7f1afc0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:27.183 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.185+0000 7f1b043ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1afc0b1e00 0x7f1afc0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:27.183 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.185+0000 7f1b043ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1afc0d2ce0 con 0x7f1afc0b2350 2023-07-18T03:38:27.184 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.185+0000 7f1b043ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1afc0b3350 con 0x7f1afc15c950 2023-07-18T03:38:27.184 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.185+0000 7f1b043ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1afc0b3630 con 0x7f1afc0b1e00 2023-07-18T03:38:27.184 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.185+0000 7f1b0219b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1afc15c950 0x7f1afc0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:27.184 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.185+0000 7f1b0219b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1afc15c950 0x7f1afc0b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60270/0 (socket says 172.21.15.49:60270) 2023-07-18T03:38:27.184 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.185+0000 7f1b0219b700 1 -- 172.21.15.49:0/2270513092 learned_addr learned my addr 172.21.15.49:0/2270513092 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:27.184 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.185+0000 7f1b0199a700 1 --2- 172.21.15.49:0/2270513092 >> v2:172.21.15.107:3300/0 conn(0x7f1afc0b1e00 0x7f1afc0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:27.185 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.186+0000 7f1b0219b700 1 -- 172.21.15.49:0/2270513092 >> v2:172.21.15.107:3300/0 conn(0x7f1afc0b1e00 msgr2=0x7f1afc0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:27.185 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.186+0000 7f1b0219b700 1 --2- 172.21.15.49:0/2270513092 >> v2:172.21.15.107:3300/0 conn(0x7f1afc0b1e00 0x7f1afc0d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.185 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.186+0000 7f1b0219b700 1 -- 172.21.15.49:0/2270513092 >> v1:172.21.15.107:6789/0 conn(0x7f1afc0b2350 legacy=0x7f1afc0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:27.185 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.186+0000 7f1b0219b700 1 -- 172.21.15.49:0/2270513092 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1afc0b3910 con 0x7f1afc15c950 2023-07-18T03:38:27.185 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.186+0000 7f1b0199a700 1 --2- 172.21.15.49:0/2270513092 >> v2:172.21.15.107:3300/0 conn(0x7f1afc0b1e00 0x7f1afc0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:27.186 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.186+0000 7f1b0219b700 1 --2- 172.21.15.49:0/2270513092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1afc15c950 0x7f1afc0b43f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1afc072140 tx=0x7f1af8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2e617a9f4c3b52 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:27.186 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.186+0000 7f1b01199700 1 -- 172.21.15.49:0/2270513092 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8005e30 con 0x7f1afc15c950 2023-07-18T03:38:27.186 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.186+0000 7f1b01199700 1 -- 172.21.15.49:0/2270513092 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af8004030 con 0x7f1afc15c950 2023-07-18T03:38:27.186 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.186+0000 7f1b01199700 1 -- 172.21.15.49:0/2270513092 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af80043e0 con 0x7f1afc15c950 2023-07-18T03:38:27.186 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.187+0000 7f1b043ff700 1 -- 172.21.15.49:0/2270513092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1afc15c950 msgr2=0x7f1afc0b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:27.186 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.187+0000 7f1b043ff700 1 --2- 172.21.15.49:0/2270513092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1afc15c950 0x7f1afc0b43f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1afc072140 tx=0x7f1af8009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.186 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.187+0000 7f1b043ff700 1 -- 172.21.15.49:0/2270513092 shutdown_connections 2023-07-18T03:38:27.187 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.187+0000 7f1b043ff700 1 --2- 172.21.15.49:0/2270513092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1afc15c950 0x7f1afc0b43f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.187 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.187+0000 7f1b043ff700 1 --2- 172.21.15.49:0/2270513092 >> v2:172.21.15.107:3300/0 conn(0x7f1afc0b1e00 0x7f1afc0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.187 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.187+0000 7f1b043ff700 1 -- 172.21.15.49:0/2270513092 >> 172.21.15.49:0/2270513092 conn(0x7f1afc09c1d0 msgr2=0x7f1afc0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:27.187 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.187+0000 7f1b043ff700 1 -- 172.21.15.49:0/2270513092 shutdown_connections 2023-07-18T03:38:27.187 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.187+0000 7f1b043ff700 1 -- 172.21.15.49:0/2270513092 wait complete. 2023-07-18T03:38:27.189 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.189+0000 7f1b043ff700 1 Processor -- start 2023-07-18T03:38:27.189 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.189+0000 7f1b043ff700 1 -- start start 2023-07-18T03:38:27.189 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.190+0000 7f1b043ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1afc0b1e00 0x7f1afc151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:27.189 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.190+0000 7f1b043ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1afc0b2350 0x7f1afc0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:27.190 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.190+0000 7f1b043ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1afc15c950 0x7f1afc0b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:27.190 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.190+0000 7f1b043ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc1f2e70 con 0x7f1afc15c950 2023-07-18T03:38:27.190 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.190+0000 7f1b043ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc151e50 con 0x7f1afc0b1e00 2023-07-18T03:38:27.190 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.190+0000 7f1b043ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc1f21f0 con 0x7f1afc0b2350 2023-07-18T03:38:27.191 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.190+0000 7f1b0299c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1afc15c950 0x7f1afc0b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:27.191 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.190+0000 7f1b0299c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1afc15c950 0x7f1afc0b0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60286/0 (socket says 172.21.15.49:60286) 2023-07-18T03:38:27.191 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.190+0000 7f1b0299c700 1 -- 172.21.15.49:0/347645721 learned_addr learned my addr 172.21.15.49:0/347645721 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:27.192 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.190+0000 7f1b0219b700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1afc0b1e00 0x7f1afc151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:27.192 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.190+0000 7f1b0199a700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1afc0b2350 0x7f1afc0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:27.192 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.191+0000 7f1b0299c700 1 -- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1afc0b2350 msgr2=0x7f1afc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:27.192 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.191+0000 7f1b0299c700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1afc0b2350 0x7f1afc0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.193 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.191+0000 7f1b0299c700 1 -- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1afc0b1e00 msgr2=0x7f1afc151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:27.193 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.191+0000 7f1b0299c700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1afc0b1e00 0x7f1afc151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.193 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.191+0000 7f1b0299c700 1 -- 172.21.15.49:0/347645721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af8009500 con 0x7f1afc15c950 2023-07-18T03:38:27.194 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.191+0000 7f1b0299c700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1afc15c950 0x7f1afc0b0c40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1af400cc50 tx=0x7f1af400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:27.194 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.191+0000 7f1af2ffd700 1 -- 172.21.15.49:0/347645721 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4003c00 con 0x7f1afc15c950 2023-07-18T03:38:27.194 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.191+0000 7f1af2ffd700 1 -- 172.21.15.49:0/347645721 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af4003d60 con 0x7f1afc15c950 2023-07-18T03:38:27.194 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.191+0000 7f1af2ffd700 1 -- 172.21.15.49:0/347645721 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af400aab0 con 0x7f1afc15c950 2023-07-18T03:38:27.195 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.191+0000 7f1b043ff700 1 -- 172.21.15.49:0/347645721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1afc1f24d0 con 0x7f1afc15c950 2023-07-18T03:38:27.195 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.191+0000 7f1b043ff700 1 -- 172.21.15.49:0/347645721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1afc1f2a90 con 0x7f1afc15c950 2023-07-18T03:38:27.195 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.192+0000 7f1b043ff700 1 -- 172.21.15.49:0/347645721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1afc1f1fb0 con 0x7f1afc15c950 2023-07-18T03:38:27.197 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.194+0000 7f1af2ffd700 1 -- 172.21.15.49:0/347645721 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f1af400ac10 con 0x7f1afc15c950 2023-07-18T03:38:27.197 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.194+0000 7f1af2ffd700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1ae006ac50 0x7f1ae006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:27.197 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.195+0000 7f1af2ffd700 1 -- 172.21.15.49:0/347645721 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3351+0+0 (secure 0 0 0) 0x7f1af4088880 con 0x7f1afc15c950 2023-07-18T03:38:27.197 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.195+0000 7f1b0219b700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1ae006ac50 0x7f1ae006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:27.198 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.196+0000 7f1b0219b700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1ae006ac50 0x7f1ae006d100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1afc072140 tx=0x7f1af8008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:27.198 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.200+0000 7f1af2ffd700 1 -- 172.21.15.49:0/347645721 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1afc1f1fb0 con 0x7f1afc15c950 2023-07-18T03:38:27.411 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.415+0000 7f1b043ff700 1 -- 172.21.15.49:0/347645721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1afc1f1fb0 con 0x7f1afc15c950 2023-07-18T03:38:27.412 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.416+0000 7f1af2ffd700 1 -- 172.21.15.49:0/347645721 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10964 (secure 0 0 0) 0x7f1afc1f1fb0 con 0x7f1afc15c950 2023-07-18T03:38:27.412 INFO:teuthology.misc.health.smithi049.stdout: 2023-07-18T03:38:27.413 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":15,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:27.318708+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:27.414 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.418+0000 7f1b043ff700 1 -- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1ae006ac50 msgr2=0x7f1ae006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:27.415 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.418+0000 7f1b043ff700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1ae006ac50 0x7f1ae006d100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1afc072140 tx=0x7f1af8008070 comp rx=0 tx=0).stop 2023-07-18T03:38:27.415 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.418+0000 7f1b043ff700 1 -- 172.21.15.49:0/347645721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1afc15c950 msgr2=0x7f1afc0b0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:27.415 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.418+0000 7f1b043ff700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1afc15c950 0x7f1afc0b0c40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1af400cc50 tx=0x7f1af400cf60 comp rx=0 tx=0).stop 2023-07-18T03:38:27.415 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.419+0000 7f1b043ff700 1 -- 172.21.15.49:0/347645721 shutdown_connections 2023-07-18T03:38:27.415 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.419+0000 7f1b043ff700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1ae006ac50 0x7f1ae006d100 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.415 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.419+0000 7f1b043ff700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1afc15c950 0x7f1afc0b0c40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.416 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.419+0000 7f1b043ff700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1afc0b1e00 0x7f1afc151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.416 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.419+0000 7f1b043ff700 1 --2- 172.21.15.49:0/347645721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1afc0b2350 0x7f1afc0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.416 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.419+0000 7f1b043ff700 1 -- 172.21.15.49:0/347645721 >> 172.21.15.49:0/347645721 conn(0x7f1afc09c1d0 msgr2=0x7f1afc0b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:27.416 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.419+0000 7f1b043ff700 1 -- 172.21.15.49:0/347645721 shutdown_connections 2023-07-18T03:38:27.416 INFO:teuthology.misc.health.smithi049.stderr:2023-07-18T03:38:27.419+0000 7f1b043ff700 1 -- 172.21.15.49:0/347645721 wait complete. 2023-07-18T03:38:27.430 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-07-18T03:38:27.430 INFO:tasks.ceph:Creating RBD pool 2023-07-18T03:38:27.431 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd pool create rbd 8 2023-07-18T03:38:27.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.633+0000 7fb9467ff700 1 Processor -- start 2023-07-18T03:38:27.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.633+0000 7fb9467ff700 1 -- start start 2023-07-18T03:38:27.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.634+0000 7fb9467ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb9400b8a60 0x7fb9400b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:27.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.634+0000 7fb9467ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b4530 0x7fb9400d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:27.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.634+0000 7fb9467ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9400b8e30 con 0x7fb9400d1050 2023-07-18T03:38:27.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.634+0000 7fb9467ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9400d14d0 con 0x7fb9400b8a60 2023-07-18T03:38:27.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.634+0000 7fb9467ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400b1e30 con 0x7fb9400b4530 2023-07-18T03:38:27.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93ffff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb9400b8a60 0x7fb9400b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:27.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93f7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b4530 0x7fb9400d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:27.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93f7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b4530 0x7fb9400d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:44684/0 (socket says 172.21.15.49:44684) 2023-07-18T03:38:27.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93ffff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb9400b8a60 0x7fb9400b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:56430/0 (socket says 172.21.15.49:56430) 2023-07-18T03:38:27.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb944d9c700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fb9400d1050 0x7fb9400b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:41378/0 (socket says 172.21.15.49:41378) 2023-07-18T03:38:27.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93f7fe700 1 -- 172.21.15.49:0/903106636 learned_addr learned my addr 172.21.15.49:0/903106636 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:27.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93effd700 1 -- 172.21.15.49:0/903106636 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402412925 0 0) 0x7fb9400b8e30 con 0x7fb9400d1050 2023-07-18T03:38:27.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93effd700 1 -- 172.21.15.49:0/903106636 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9280035b0 con 0x7fb9400d1050 2023-07-18T03:38:27.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93f7fe700 1 -- 172.21.15.49:0/903106636 >> v2:172.21.15.107:3300/0 conn(0x7fb9400b8a60 msgr2=0x7fb9400b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:27.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93f7fe700 1 --2- 172.21.15.49:0/903106636 >> v2:172.21.15.107:3300/0 conn(0x7fb9400b8a60 0x7fb9400b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93f7fe700 1 -- 172.21.15.49:0/903106636 >> v1:172.21.15.107:6789/0 conn(0x7fb9400d1050 legacy=0x7fb9400b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:27.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93f7fe700 1 -- 172.21.15.49:0/903106636 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9400afaa0 con 0x7fb9400b4530 2023-07-18T03:38:27.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93ffff700 1 --2- 172.21.15.49:0/903106636 >> v2:172.21.15.107:3300/0 conn(0x7fb9400b8a60 0x7fb9400b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:27.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.635+0000 7fb93f7fe700 1 --2- 172.21.15.49:0/903106636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b4530 0x7fb9400d0b10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb9340098f0 tx=0x7fb934009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48280e7edb003cad server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:27.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.636+0000 7fb93effd700 1 -- 172.21.15.49:0/903106636 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934011830 con 0x7fb9400b4530 2023-07-18T03:38:27.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.636+0000 7fb93effd700 1 -- 172.21.15.49:0/903106636 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb934011990 con 0x7fb9400b4530 2023-07-18T03:38:27.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.636+0000 7fb93effd700 1 -- 172.21.15.49:0/903106636 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934011d00 con 0x7fb9400b4530 2023-07-18T03:38:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.636+0000 7fb9467ff700 1 -- 172.21.15.49:0/903106636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b4530 msgr2=0x7fb9400d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.636+0000 7fb9467ff700 1 --2- 172.21.15.49:0/903106636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b4530 0x7fb9400d0b10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb9340098f0 tx=0x7fb934009c00 comp rx=0 tx=0).stop 2023-07-18T03:38:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.636+0000 7fb9467ff700 1 -- 172.21.15.49:0/903106636 shutdown_connections 2023-07-18T03:38:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.636+0000 7fb9467ff700 1 --2- 172.21.15.49:0/903106636 >> v2:172.21.15.107:3300/0 conn(0x7fb9400b8a60 0x7fb9400b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.636+0000 7fb9467ff700 1 --2- 172.21.15.49:0/903106636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b4530 0x7fb9400d0b10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.636+0000 7fb9467ff700 1 -- 172.21.15.49:0/903106636 >> 172.21.15.49:0/903106636 conn(0x7fb94009c1b0 msgr2=0x7fb9400d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:27.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.636+0000 7fb9467ff700 1 -- 172.21.15.49:0/903106636 shutdown_connections 2023-07-18T03:38:27.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.636+0000 7fb9467ff700 1 -- 172.21.15.49:0/903106636 wait complete. 2023-07-18T03:38:27.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.638+0000 7fb9467ff700 1 Processor -- start 2023-07-18T03:38:27.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.638+0000 7fb9467ff700 1 -- start start 2023-07-18T03:38:27.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.639+0000 7fb9467ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9400b4530 0x7fb94015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:27.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.639+0000 7fb9467ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b8a60 0x7fb940157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:27.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.639+0000 7fb9467ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9400d1050 0x7fb94015f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:27.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.639+0000 7fb9467ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb940158720 con 0x7fb9400b4530 2023-07-18T03:38:27.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.639+0000 7fb9467ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94015cee0 con 0x7fb9400d1050 2023-07-18T03:38:27.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.639+0000 7fb9467ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94015d1c0 con 0x7fb9400b8a60 2023-07-18T03:38:27.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb93ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9400b4530 0x7fb94015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:27.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb93ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9400b4530 0x7fb94015dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60302/0 (socket says 172.21.15.49:60302) 2023-07-18T03:38:27.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb93ffff700 1 -- 172.21.15.49:0/1918188193 learned_addr learned my addr 172.21.15.49:0/1918188193 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:27.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb944d9c700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9400d1050 0x7fb94015f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:27.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb93f7fe700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b8a60 0x7fb940157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb93ffff700 1 -- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b8a60 msgr2=0x7fb940157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb93ffff700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b8a60 0x7fb940157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb93ffff700 1 -- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9400d1050 msgr2=0x7fb94015f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb93ffff700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9400d1050 0x7fb94015f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb93ffff700 1 -- 172.21.15.49:0/1918188193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9300094a0 con 0x7fb9400b4530 2023-07-18T03:38:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb93f7fe700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b8a60 0x7fb940157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb944d9c700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9400d1050 0x7fb94015f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.640+0000 7fb93ffff700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9400b4530 0x7fb94015dca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb93000ebe0 tx=0x7fb930009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.641+0000 7fb93cff9700 1 -- 172.21.15.49:0/1918188193 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93000c910 con 0x7fb9400b4530 2023-07-18T03:38:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.641+0000 7fb93cff9700 1 -- 172.21.15.49:0/1918188193 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93000ca70 con 0x7fb9400b4530 2023-07-18T03:38:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.641+0000 7fb9467ff700 1 -- 172.21.15.49:0/1918188193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb934009550 con 0x7fb9400b4530 2023-07-18T03:38:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.641+0000 7fb93cff9700 1 -- 172.21.15.49:0/1918188193 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93000cde0 con 0x7fb9400b4530 2023-07-18T03:38:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.641+0000 7fb9467ff700 1 -- 172.21.15.49:0/1918188193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94015d780 con 0x7fb9400b4530 2023-07-18T03:38:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.642+0000 7fb9467ff700 1 -- 172.21.15.49:0/1918188193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94015daf0 con 0x7fb9400b4530 2023-07-18T03:38:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.643+0000 7fb93cff9700 1 -- 172.21.15.49:0/1918188193 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fb930013470 con 0x7fb9400b4530 2023-07-18T03:38:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.644+0000 7fb93cff9700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb92c06ac50 0x7fb92c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.644+0000 7fb93cff9700 1 -- 172.21.15.49:0/1918188193 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fb930088ff0 con 0x7fb9400b4530 2023-07-18T03:38:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.644+0000 7fb93f7fe700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb92c06ac50 0x7fb92c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.645+0000 7fb93f7fe700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb92c06ac50 0x7fb92c06d100 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb934009c40 tx=0x7fb934008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.648+0000 7fb93cff9700 1 -- 172.21.15.49:0/1918188193 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb9300547a0 con 0x7fb9400b4530 2023-07-18T03:38:27.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:27.861+0000 7fb9467ff700 1 -- 172.21.15.49:0/1918188193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fb9400d23e0 con 0x7fb9400b4530 2023-07-18T03:38:28.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.327+0000 7fb93cff9700 1 -- 172.21.15.49:0/1918188193 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7fb930057dc0 con 0x7fb9400b4530 2023-07-18T03:38:28.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.330+0000 7fb9467ff700 1 -- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb92c06ac50 msgr2=0x7fb92c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.330+0000 7fb9467ff700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb92c06ac50 0x7fb92c06d100 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb934009c40 tx=0x7fb934008030 comp rx=0 tx=0).stop 2023-07-18T03:38:28.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.330+0000 7fb9467ff700 1 -- 172.21.15.49:0/1918188193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9400b4530 msgr2=0x7fb94015dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.330+0000 7fb9467ff700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9400b4530 0x7fb94015dca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb93000ebe0 tx=0x7fb930009ec0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.330+0000 7fb9467ff700 1 -- 172.21.15.49:0/1918188193 shutdown_connections 2023-07-18T03:38:28.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.330+0000 7fb9467ff700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9400d1050 0x7fb94015f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.330+0000 7fb9467ff700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb92c06ac50 0x7fb92c06d100 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.330+0000 7fb9467ff700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9400b4530 0x7fb94015dca0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.330+0000 7fb9467ff700 1 --2- 172.21.15.49:0/1918188193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9400b8a60 0x7fb940157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.330+0000 7fb9467ff700 1 -- 172.21.15.49:0/1918188193 >> 172.21.15.49:0/1918188193 conn(0x7fb94009c1b0 msgr2=0x7fb94015b150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:28.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.331+0000 7fb9467ff700 1 -- 172.21.15.49:0/1918188193 shutdown_connections 2023-07-18T03:38:28.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.331+0000 7fb9467ff700 1 -- 172.21.15.49:0/1918188193 wait complete. 2023-07-18T03:38:28.328 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'rbd' created 2023-07-18T03:38:28.346 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-07-18T03:38:28.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.551+0000 7f874a9ff700 1 Processor -- start 2023-07-18T03:38:28.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.551+0000 7f874a9ff700 1 -- start start 2023-07-18T03:38:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.552+0000 7f874a9ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87440ca1e0 0x7f87440ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.552+0000 7f874a9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f87440b0480 0x7f87440b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.552+0000 7f874a9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874415b530 con 0x7f87440c8fc0 2023-07-18T03:38:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.552+0000 7f874a9ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874415aa10 con 0x7f87440ca1e0 2023-07-18T03:38:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.552+0000 7f874a9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874415eef0 con 0x7f87440b0480 2023-07-18T03:38:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.552+0000 7f8743fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87440ca1e0 0x7f87440ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.552+0000 7f8743fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87440ca1e0 0x7f87440ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60306/0 (socket says 172.21.15.49:60306) 2023-07-18T03:38:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.552+0000 7f8743fff700 1 -- 172.21.15.49:0/3005229588 learned_addr learned my addr 172.21.15.49:0/3005229588 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.552+0000 7f87437fe700 1 --2- 172.21.15.49:0/3005229588 >> v2:172.21.15.107:3300/0 conn(0x7f87440b0480 0x7f87440b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.553+0000 7f8743fff700 1 -- 172.21.15.49:0/3005229588 >> v2:172.21.15.107:3300/0 conn(0x7f87440b0480 msgr2=0x7f87440b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.553+0000 7f8743fff700 1 --2- 172.21.15.49:0/3005229588 >> v2:172.21.15.107:3300/0 conn(0x7f87440b0480 0x7f87440b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.553+0000 7f8743fff700 1 -- 172.21.15.49:0/3005229588 >> v1:172.21.15.107:6789/0 conn(0x7f87440c8fc0 legacy=0x7f87440c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.553+0000 7f8743fff700 1 -- 172.21.15.49:0/3005229588 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87440caaf0 con 0x7f87440ca1e0 2023-07-18T03:38:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.553+0000 7f8743fff700 1 --2- 172.21.15.49:0/3005229588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87440ca1e0 0x7f87440ca5b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f87440726a0 tx=0x7f8734009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8ab32c77c8e88d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.553+0000 7f8742ffd700 1 -- 172.21.15.49:0/3005229588 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734011830 con 0x7f87440ca1e0 2023-07-18T03:38:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.553+0000 7f8742ffd700 1 -- 172.21.15.49:0/3005229588 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8734011990 con 0x7f87440ca1e0 2023-07-18T03:38:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.553+0000 7f8742ffd700 1 -- 172.21.15.49:0/3005229588 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734011d20 con 0x7f87440ca1e0 2023-07-18T03:38:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.554+0000 7f874a9ff700 1 -- 172.21.15.49:0/3005229588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87440ca1e0 msgr2=0x7f87440ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.554+0000 7f874a9ff700 1 --2- 172.21.15.49:0/3005229588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87440ca1e0 0x7f87440ca5b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f87440726a0 tx=0x7f8734009c00 comp rx=0 tx=0).stop 2023-07-18T03:38:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.554+0000 7f874a9ff700 1 -- 172.21.15.49:0/3005229588 shutdown_connections 2023-07-18T03:38:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.554+0000 7f874a9ff700 1 --2- 172.21.15.49:0/3005229588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87440ca1e0 0x7f87440ca5b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.554+0000 7f874a9ff700 1 --2- 172.21.15.49:0/3005229588 >> v2:172.21.15.107:3300/0 conn(0x7f87440b0480 0x7f87440b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.554+0000 7f874a9ff700 1 -- 172.21.15.49:0/3005229588 >> 172.21.15.49:0/3005229588 conn(0x7f87440cea00 msgr2=0x7f87440cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:28.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.554+0000 7f874a9ff700 1 -- 172.21.15.49:0/3005229588 shutdown_connections 2023-07-18T03:38:28.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.554+0000 7f874a9ff700 1 -- 172.21.15.49:0/3005229588 wait complete. 2023-07-18T03:38:28.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.556+0000 7f874a9ff700 1 Processor -- start 2023-07-18T03:38:28.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.556+0000 7f874a9ff700 1 -- start start 2023-07-18T03:38:28.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f874a9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87440b0480 0x7f87440b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f874a9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87440c8fc0 0x7f87440aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f874a9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87440ca1e0 0x7f87440af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f874a9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87441f6ba0 con 0x7f87440b0480 2023-07-18T03:38:28.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f874a9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87440b2990 con 0x7f87440c8fc0 2023-07-18T03:38:28.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f874a9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87441f5f90 con 0x7f87440ca1e0 2023-07-18T03:38:28.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f8743fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87440b0480 0x7f87440b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f8743fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87440b0480 0x7f87440b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60316/0 (socket says 172.21.15.49:60316) 2023-07-18T03:38:28.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f8743fff700 1 -- 172.21.15.49:0/1812413916 learned_addr learned my addr 172.21.15.49:0/1812413916 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:28.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f87437fe700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87440c8fc0 0x7f87440aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f8748f9c700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87440ca1e0 0x7f87440af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f8743fff700 1 -- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87440ca1e0 msgr2=0x7f87440af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f8743fff700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87440ca1e0 0x7f87440af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f8743fff700 1 -- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87440c8fc0 msgr2=0x7f87440aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f8743fff700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87440c8fc0 0x7f87440aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.557+0000 7f8743fff700 1 -- 172.21.15.49:0/1812413916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8734009550 con 0x7f87440b0480 2023-07-18T03:38:28.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.558+0000 7f8743fff700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87440b0480 0x7f87440b2450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8734014040 tx=0x7f8734004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:28.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.558+0000 7f8740ff9700 1 -- 172.21.15.49:0/1812413916 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734004590 con 0x7f87440b0480 2023-07-18T03:38:28.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.558+0000 7f8740ff9700 1 -- 172.21.15.49:0/1812413916 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873401e070 con 0x7f87440b0480 2023-07-18T03:38:28.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.558+0000 7f874a9ff700 1 -- 172.21.15.49:0/1812413916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87441f6270 con 0x7f87440b0480 2023-07-18T03:38:28.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.558+0000 7f8740ff9700 1 -- 172.21.15.49:0/1812413916 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87340038c0 con 0x7f87440b0480 2023-07-18T03:38:28.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.558+0000 7f874a9ff700 1 -- 172.21.15.49:0/1812413916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87441f6830 con 0x7f87440b0480 2023-07-18T03:38:28.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.559+0000 7f874a9ff700 1 -- 172.21.15.49:0/1812413916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8744074350 con 0x7f87440b0480 2023-07-18T03:38:28.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.560+0000 7f8740ff9700 1 -- 172.21.15.49:0/1812413916 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f8734003a20 con 0x7f87440b0480 2023-07-18T03:38:28.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.561+0000 7f8740ff9700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f873006ac50 0x7f873006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.561+0000 7f8740ff9700 1 -- 172.21.15.49:0/1812413916 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5188+0+0 (secure 0 0 0) 0x7f8734089e70 con 0x7f87440b0480 2023-07-18T03:38:28.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.562+0000 7f87437fe700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f873006ac50 0x7f873006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.563+0000 7f87437fe700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f873006ac50 0x7f873006d100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87441f0460 tx=0x7f8738009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:28.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.566+0000 7f8740ff9700 1 -- 172.21.15.49:0/1812413916 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8734055350 con 0x7f87440b0480 2023-07-18T03:38:28.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:28.774+0000 7f874a9ff700 1 -- 172.21.15.49:0/1812413916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8744154830 con 0x7f87440b0480 2023-07-18T03:38:29.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.328+0000 7f8740ff9700 1 -- 172.21.15.49:0/1812413916 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8734020020 con 0x7f87440b0480 2023-07-18T03:38:29.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.331+0000 7f874a9ff700 1 -- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f873006ac50 msgr2=0x7f873006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:29.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.331+0000 7f874a9ff700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f873006ac50 0x7f873006d100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87441f0460 tx=0x7f8738009210 comp rx=0 tx=0).stop 2023-07-18T03:38:29.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.331+0000 7f874a9ff700 1 -- 172.21.15.49:0/1812413916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87440b0480 msgr2=0x7f87440b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:29.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.331+0000 7f874a9ff700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87440b0480 0x7f87440b2450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8734014040 tx=0x7f8734004750 comp rx=0 tx=0).stop 2023-07-18T03:38:29.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.331+0000 7f874a9ff700 1 -- 172.21.15.49:0/1812413916 shutdown_connections 2023-07-18T03:38:29.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.331+0000 7f874a9ff700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87440ca1e0 0x7f87440af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.331+0000 7f874a9ff700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f873006ac50 0x7f873006d100 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.331+0000 7f874a9ff700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87440b0480 0x7f87440b2450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.331+0000 7f874a9ff700 1 --2- 172.21.15.49:0/1812413916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87440c8fc0 0x7f87440aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.331+0000 7f874a9ff700 1 -- 172.21.15.49:0/1812413916 >> 172.21.15.49:0/1812413916 conn(0x7f87440cea00 msgr2=0x7f874406c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:29.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.331+0000 7f874a9ff700 1 -- 172.21.15.49:0/1812413916 shutdown_connections 2023-07-18T03:38:29.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.331+0000 7f874a9ff700 1 -- 172.21.15.49:0/1812413916 wait complete. 2023-07-18T03:38:29.330 INFO:teuthology.orchestra.run.smithi049.stderr:enabled application 'rbd' on pool 'rbd' 2023-07-18T03:38:29.348 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-07-18T03:38:29.349 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-18T03:38:29.350 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-18T03:38:29.355 INFO:tasks.ceph.mds.a:Started 2023-07-18T03:38:29.355 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-18T03:38:29.355 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-18T03:38:29.360 INFO:tasks.ceph.mds.c:Started 2023-07-18T03:38:29.360 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-18T03:38:29.360 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-18T03:38:29.365 INFO:tasks.ceph.mds.b:Started 2023-07-18T03:38:29.366 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-18T03:38:29.366 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-18T03:38:29.371 INFO:tasks.ceph.mds.d:Started 2023-07-18T03:38:29.372 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-07-18T03:38:29.373 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:29.524 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2023-07-18T03:38:29.526 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2023-07-18T03:38:29.527 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-07-18T03:38:29.529 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2023-07-18T03:38:29.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.608+0000 7fae601ff700 1 Processor -- start 2023-07-18T03:38:29.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.608+0000 7fae601ff700 1 -- start start 2023-07-18T03:38:29.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.610+0000 7fae601ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fae5815c950 0x7fae580b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:29.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.610+0000 7fae601ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fae580d1220 0x7fae580afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:29.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.610+0000 7fae601ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae580d2ce0 con 0x7fae580b1e00 2023-07-18T03:38:29.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.610+0000 7fae601ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae580b3350 con 0x7fae5815c950 2023-07-18T03:38:29.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.610+0000 7fae601ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae580b3630 con 0x7fae580d1220 2023-07-18T03:38:29.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.610+0000 7fae5d79a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fae580b1e00 0x7fae580d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45836/0 (socket says 172.21.15.49:45836) 2023-07-18T03:38:29.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.610+0000 7fae5d79a700 1 -- 172.21.15.49:0/1088778620 learned_addr learned my addr 172.21.15.49:0/1088778620 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:29.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.610+0000 7fae5e79c700 1 --2- 172.21.15.49:0/1088778620 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fae580d1220 0x7fae580afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:29.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.610+0000 7fae5df9b700 1 --2- 172.21.15.49:0/1088778620 >> v2:172.21.15.107:3300/0 conn(0x7fae5815c950 0x7fae580b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:29.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.611+0000 7fae5cf99700 1 -- 172.21.15.49:0/1088778620 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224451834 0 0) 0x7fae580d2ce0 con 0x7fae580b1e00 2023-07-18T03:38:29.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.611+0000 7fae5cf99700 1 -- 172.21.15.49:0/1088778620 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae440035b0 con 0x7fae580b1e00 2023-07-18T03:38:29.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.611+0000 7fae5e79c700 1 -- 172.21.15.49:0/1088778620 >> v2:172.21.15.107:3300/0 conn(0x7fae5815c950 msgr2=0x7fae580b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:29.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.611+0000 7fae5e79c700 1 --2- 172.21.15.49:0/1088778620 >> v2:172.21.15.107:3300/0 conn(0x7fae5815c950 0x7fae580b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.611+0000 7fae5e79c700 1 -- 172.21.15.49:0/1088778620 >> v1:172.21.15.49:6789/0 conn(0x7fae580b1e00 legacy=0x7fae580d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.611+0000 7fae5e79c700 1 -- 172.21.15.49:0/1088778620 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580b3910 con 0x7fae580d1220 2023-07-18T03:38:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.611+0000 7fae5e79c700 1 --2- 172.21.15.49:0/1088778620 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fae580d1220 0x7fae580afac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fae50009d90 tx=0x7fae5000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3068715b7035178 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.611+0000 7fae5cf99700 1 -- 172.21.15.49:0/1088778620 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5000e070 con 0x7fae580d1220 2023-07-18T03:38:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.612+0000 7fae5cf99700 1 -- 172.21.15.49:0/1088778620 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5000bb20 con 0x7fae580d1220 2023-07-18T03:38:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.612+0000 7fae5cf99700 1 -- 172.21.15.49:0/1088778620 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50012430 con 0x7fae580d1220 2023-07-18T03:38:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.612+0000 7fae601ff700 1 -- 172.21.15.49:0/1088778620 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fae580d1220 msgr2=0x7fae580afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.612+0000 7fae601ff700 1 --2- 172.21.15.49:0/1088778620 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fae580d1220 0x7fae580afac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fae50009d90 tx=0x7fae5000b330 comp rx=0 tx=0).stop 2023-07-18T03:38:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.612+0000 7fae601ff700 1 -- 172.21.15.49:0/1088778620 shutdown_connections 2023-07-18T03:38:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.612+0000 7fae601ff700 1 --2- 172.21.15.49:0/1088778620 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fae580d1220 0x7fae580afac0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.612+0000 7fae601ff700 1 --2- 172.21.15.49:0/1088778620 >> v2:172.21.15.107:3300/0 conn(0x7fae5815c950 0x7fae580b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.612+0000 7fae601ff700 1 -- 172.21.15.49:0/1088778620 >> 172.21.15.49:0/1088778620 conn(0x7fae5809c1d0 msgr2=0x7fae580d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.612+0000 7fae601ff700 1 -- 172.21.15.49:0/1088778620 shutdown_connections 2023-07-18T03:38:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.612+0000 7fae601ff700 1 -- 172.21.15.49:0/1088778620 wait complete. 2023-07-18T03:38:29.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.614+0000 7fae601ff700 1 Processor -- start 2023-07-18T03:38:29.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.614+0000 7fae601ff700 1 -- start start 2023-07-18T03:38:29.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.615+0000 7fae601ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae580b1e00 0x7fae580c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:29.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.615+0000 7fae601ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fae580d1220 0x7fae581f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.615+0000 7fae601ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae5815c950 0x7fae580b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.615+0000 7fae601ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae58146000 con 0x7fae5815c950 2023-07-18T03:38:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.615+0000 7fae601ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae581462a0 con 0x7fae580b1e00 2023-07-18T03:38:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.615+0000 7fae601ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae58146580 con 0x7fae580d1220 2023-07-18T03:38:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.615+0000 7fae5df9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae580b1e00 0x7fae580c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.615+0000 7fae5d79a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fae580d1220 0x7fae581f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.615+0000 7fae5e79c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae5815c950 0x7fae580b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.615+0000 7fae5df9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae580b1e00 0x7fae580c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:54818/0 (socket says 172.21.15.49:54818) 2023-07-18T03:38:29.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.615+0000 7fae5df9b700 1 -- 172.21.15.49:0/828261128 learned_addr learned my addr 172.21.15.49:0/828261128 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:29.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.616+0000 7fae5df9b700 1 -- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fae580d1220 msgr2=0x7fae581f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:29.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.616+0000 7fae5df9b700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fae580d1220 0x7fae581f14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.616+0000 7fae5df9b700 1 -- 172.21.15.49:0/828261128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae5815c950 msgr2=0x7fae580b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:29.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.616+0000 7fae5df9b700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae5815c950 0x7fae580b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.616+0000 7fae5df9b700 1 -- 172.21.15.49:0/828261128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae50009a70 con 0x7fae580b1e00 2023-07-18T03:38:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.616+0000 7fae5d79a700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fae580d1220 0x7fae581f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.616+0000 7fae5df9b700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae580b1e00 0x7fae580c9460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e900 tx=0x7fae5400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.616+0000 7fae4e7fc700 1 -- 172.21.15.49:0/828261128 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54013680 con 0x7fae580b1e00 2023-07-18T03:38:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.616+0000 7fae4e7fc700 1 -- 172.21.15.49:0/828261128 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae540137e0 con 0x7fae580b1e00 2023-07-18T03:38:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.616+0000 7fae4e7fc700 1 -- 172.21.15.49:0/828261128 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54013b50 con 0x7fae580b1e00 2023-07-18T03:38:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.617+0000 7fae601ff700 1 -- 172.21.15.49:0/828261128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae58146860 con 0x7fae580b1e00 2023-07-18T03:38:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.617+0000 7fae601ff700 1 -- 172.21.15.49:0/828261128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae58146e20 con 0x7fae580b1e00 2023-07-18T03:38:29.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.618+0000 7fae601ff700 1 -- 172.21.15.49:0/828261128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae580afb30 con 0x7fae580b1e00 2023-07-18T03:38:29.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.618+0000 7fae4e7fc700 1 -- 172.21.15.49:0/828261128 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fae54013cb0 con 0x7fae580b1e00 2023-07-18T03:38:29.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.619+0000 7fae4e7fc700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fae3c06ac50 0x7fae3c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:29.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.619+0000 7fae4e7fc700 1 -- 172.21.15.49:0/828261128 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fae54089430 con 0x7fae580b1e00 2023-07-18T03:38:29.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.620+0000 7fae5d79a700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fae3c06ac50 0x7fae3c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:29.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.620+0000 7fae5d79a700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fae3c06ac50 0x7fae3c06d100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fae580d3e30 tx=0x7fae48007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:29.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.625+0000 7fae4e7fc700 1 -- 172.21.15.49:0/828261128 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fae540547e0 con 0x7fae580b1e00 2023-07-18T03:38:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.847+0000 7fae601ff700 1 -- 172.21.15.49:0/828261128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae5804e280 con 0x7fae580b1e00 2023-07-18T03:38:29.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.848+0000 7fae4e7fc700 1 -- 172.21.15.49:0/828261128 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+14569 (secure 0 0 0) 0x7fae54021020 con 0x7fae580b1e00 2023-07-18T03:38:29.844 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:29.845 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:29.325892+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:29.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.850+0000 7fae601ff700 1 -- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fae3c06ac50 msgr2=0x7fae3c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:29.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.850+0000 7fae601ff700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fae3c06ac50 0x7fae3c06d100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fae580d3e30 tx=0x7fae48007dc0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.850+0000 7fae601ff700 1 -- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae580b1e00 msgr2=0x7fae580c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:29.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.850+0000 7fae601ff700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae580b1e00 0x7fae580c9460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e900 tx=0x7fae5400ec10 comp rx=0 tx=0).stop 2023-07-18T03:38:29.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.851+0000 7fae601ff700 1 -- 172.21.15.49:0/828261128 shutdown_connections 2023-07-18T03:38:29.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.851+0000 7fae601ff700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fae3c06ac50 0x7fae3c06d100 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.851+0000 7fae601ff700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae5815c950 0x7fae580b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.851+0000 7fae601ff700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae580b1e00 0x7fae580c9460 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.851+0000 7fae601ff700 1 --2- 172.21.15.49:0/828261128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fae580d1220 0x7fae581f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:29.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.851+0000 7fae601ff700 1 -- 172.21.15.49:0/828261128 >> 172.21.15.49:0/828261128 conn(0x7fae5809c1d0 msgr2=0x7fae580d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:29.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.851+0000 7fae601ff700 1 -- 172.21.15.49:0/828261128 shutdown_connections 2023-07-18T03:38:29.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:29.851+0000 7fae601ff700 1 -- 172.21.15.49:0/828261128 wait complete. 2023-07-18T03:38:29.866 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:38:29.867 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:38:30.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.082+0000 7f00687ff700 1 Processor -- start 2023-07-18T03:38:30.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.082+0000 7f00687ff700 1 -- start start 2023-07-18T03:38:30.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.083+0000 7f00687ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f00600d0010 0x7f00600b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:30.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.083+0000 7f00687ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006015cd60 con 0x7f00600d0480 2023-07-18T03:38:30.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.083+0000 7f00687ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00600cf340 con 0x7f00600b4600 2023-07-18T03:38:30.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.083+0000 7f00687ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00600d1550 con 0x7f00600d0010 2023-07-18T03:38:30.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.083+0000 7f0065d9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f00600d0010 0x7f00600b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:30.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.084+0000 7f0065d9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f00600d0010 0x7f00600b5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45730/0 (socket says 172.21.15.49:45730) 2023-07-18T03:38:30.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.084+0000 7f0065d9a700 1 -- 172.21.15.49:0/3395546423 learned_addr learned my addr 172.21.15.49:0/3395546423 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:30.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.084+0000 7f006659b700 1 --1- 172.21.15.49:0/3395546423 >> v1:172.21.15.107:6789/0 conn(0x7f00600b4600 0x7f00600cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:57386/0 (socket says 172.21.15.49:57386) 2023-07-18T03:38:30.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.084+0000 7f0065d9a700 1 -- 172.21.15.49:0/3395546423 >> v1:172.21.15.107:6789/0 conn(0x7f00600b4600 legacy=0x7f00600cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.084+0000 7f0065d9a700 1 -- 172.21.15.49:0/3395546423 >> v1:172.21.15.49:6789/0 conn(0x7f00600d0480 legacy=0x7f00600b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.084+0000 7f0065d9a700 1 -- 172.21.15.49:0/3395546423 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00600d0e90 con 0x7f00600d0010 2023-07-18T03:38:30.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.084+0000 7f0065599700 1 -- 172.21.15.49:0/3395546423 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051529263 0 0) 0x7f00600cf340 con 0x7f00600b4600 2023-07-18T03:38:30.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.084+0000 7f0065d9a700 1 --2- 172.21.15.49:0/3395546423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00600d0010 0x7f00600b5ef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f00500098f0 tx=0x7f0050009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce2106a27c0b492 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:30.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.085+0000 7f0065599700 1 -- 172.21.15.49:0/3395546423 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050004230 con 0x7f00600d0010 2023-07-18T03:38:30.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.085+0000 7f0065599700 1 -- 172.21.15.49:0/3395546423 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0050004390 con 0x7f00600d0010 2023-07-18T03:38:30.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.085+0000 7f0065599700 1 -- 172.21.15.49:0/3395546423 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00500036a0 con 0x7f00600d0010 2023-07-18T03:38:30.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.085+0000 7f00687ff700 1 -- 172.21.15.49:0/3395546423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00600d0010 msgr2=0x7f00600b5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.085+0000 7f00687ff700 1 --2- 172.21.15.49:0/3395546423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00600d0010 0x7f00600b5ef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f00500098f0 tx=0x7f0050009c00 comp rx=0 tx=0).stop 2023-07-18T03:38:30.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.085+0000 7f00687ff700 1 -- 172.21.15.49:0/3395546423 shutdown_connections 2023-07-18T03:38:30.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.085+0000 7f00687ff700 1 --2- 172.21.15.49:0/3395546423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00600d0010 0x7f00600b5ef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.085+0000 7f00687ff700 1 -- 172.21.15.49:0/3395546423 >> 172.21.15.49:0/3395546423 conn(0x7f006009c200 msgr2=0x7f00600d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:30.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.085+0000 7f00687ff700 1 -- 172.21.15.49:0/3395546423 shutdown_connections 2023-07-18T03:38:30.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.085+0000 7f00687ff700 1 -- 172.21.15.49:0/3395546423 wait complete. 2023-07-18T03:38:30.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.087+0000 7f00687ff700 1 Processor -- start 2023-07-18T03:38:30.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.087+0000 7f00687ff700 1 -- start start 2023-07-18T03:38:30.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.088+0000 7f00687ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00600d0010 0x7f006015f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:30.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.088+0000 7f00687ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00600d0480 0x7f0060153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:30.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.088+0000 7f00687ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00601582d0 0x7f0060159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:30.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.088+0000 7f00687ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00601572c0 con 0x7f00600d0010 2023-07-18T03:38:30.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.088+0000 7f00687ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0060159d90 con 0x7f00601582d0 2023-07-18T03:38:30.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.088+0000 7f00687ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0060157510 con 0x7f00600d0480 2023-07-18T03:38:30.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.089+0000 7f006659b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00600d0010 0x7f006015f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:30.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.089+0000 7f006659b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00600d0010 0x7f006015f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45740/0 (socket says 172.21.15.49:45740) 2023-07-18T03:38:30.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.089+0000 7f006659b700 1 -- 172.21.15.49:0/1481485261 learned_addr learned my addr 172.21.15.49:0/1481485261 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:30.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.089+0000 7f0065d9a700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00600d0480 0x7f0060153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:30.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.089+0000 7f0066d9c700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00601582d0 0x7f0060159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:30.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.089+0000 7f006659b700 1 -- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00600d0480 msgr2=0x7f0060153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.089+0000 7f006659b700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00600d0480 0x7f0060153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.089+0000 7f006659b700 1 -- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00601582d0 msgr2=0x7f0060159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.089+0000 7f006659b700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00601582d0 0x7f0060159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.089+0000 7f006659b700 1 -- 172.21.15.49:0/1481485261 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0050009550 con 0x7f00600d0010 2023-07-18T03:38:30.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.089+0000 7f0066d9c700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00601582d0 0x7f0060159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:30.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.089+0000 7f006659b700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00600d0010 0x7f006015f9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f005c00c520 tx=0x7f005c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:30.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.090+0000 7f0056ffd700 1 -- 172.21.15.49:0/1481485261 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c004c80 con 0x7f00600d0010 2023-07-18T03:38:30.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.090+0000 7f0056ffd700 1 -- 172.21.15.49:0/1481485261 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c004e00 con 0x7f00600d0010 2023-07-18T03:38:30.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.090+0000 7f0056ffd700 1 -- 172.21.15.49:0/1481485261 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c007310 con 0x7f00600d0010 2023-07-18T03:38:30.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.090+0000 7f00687ff700 1 -- 172.21.15.49:0/1481485261 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00601577f0 con 0x7f00600d0010 2023-07-18T03:38:30.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.090+0000 7f00687ff700 1 -- 172.21.15.49:0/1481485261 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0060157db0 con 0x7f00600d0010 2023-07-18T03:38:30.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.091+0000 7f00687ff700 1 -- 172.21.15.49:0/1481485261 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0060074350 con 0x7f00600d0010 2023-07-18T03:38:30.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.092+0000 7f0056ffd700 1 -- 172.21.15.49:0/1481485261 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f005c007470 con 0x7f00600d0010 2023-07-18T03:38:30.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.093+0000 7f0056ffd700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f004406ac50 0x7f004406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:30.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.094+0000 7f0056ffd700 1 -- 172.21.15.49:0/1481485261 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f005c088d60 con 0x7f00600d0010 2023-07-18T03:38:30.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.094+0000 7f0065d9a700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f004406ac50 0x7f004406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:30.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.094+0000 7f0065d9a700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f004406ac50 0x7f004406d100 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0050009130 tx=0x7f0050008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:30.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.099+0000 7f0056ffd700 1 -- 172.21.15.49:0/1481485261 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f005c08e020 con 0x7f00600d0010 2023-07-18T03:38:30.170 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:38:30.177 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:38:30.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.312+0000 7f00687ff700 1 -- 172.21.15.49:0/1481485261 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f00600d1830 con 0x7f00600d0010 2023-07-18T03:38:30.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.313+0000 7f0056ffd700 1 -- 172.21.15.49:0/1481485261 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f005c054230 con 0x7f00600d0010 2023-07-18T03:38:30.309 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:38:30.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.315+0000 7f00687ff700 1 -- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f004406ac50 msgr2=0x7f004406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.315+0000 7f00687ff700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f004406ac50 0x7f004406d100 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0050009130 tx=0x7f0050008030 comp rx=0 tx=0).stop 2023-07-18T03:38:30.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.315+0000 7f00687ff700 1 -- 172.21.15.49:0/1481485261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00600d0010 msgr2=0x7f006015f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.315+0000 7f00687ff700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00600d0010 0x7f006015f9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f005c00c520 tx=0x7f005c00c8e0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.315+0000 7f00687ff700 1 -- 172.21.15.49:0/1481485261 shutdown_connections 2023-07-18T03:38:30.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.315+0000 7f00687ff700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00601582d0 0x7f0060159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.315+0000 7f00687ff700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f004406ac50 0x7f004406d100 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.315+0000 7f00687ff700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00600d0010 0x7f006015f9d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.315+0000 7f00687ff700 1 --2- 172.21.15.49:0/1481485261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00600d0480 0x7f0060153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.315+0000 7f00687ff700 1 -- 172.21.15.49:0/1481485261 >> 172.21.15.49:0/1481485261 conn(0x7f006009c200 msgr2=0x7f00600d3650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:30.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.315+0000 7f00687ff700 1 -- 172.21.15.49:0/1481485261 shutdown_connections 2023-07-18T03:38:30.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.315+0000 7f00687ff700 1 -- 172.21.15.49:0/1481485261 wait complete. 2023-07-18T03:38:30.330 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:38:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.588+0000 7f19f2bff700 1 Processor -- start 2023-07-18T03:38:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.588+0000 7f19f2bff700 1 -- start start 2023-07-18T03:38:30.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.589+0000 7f19f2bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19ec0b8660 0x7f19ec0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:30.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.589+0000 7f19f2bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f19ec0cd970 0x7f19ec0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:30.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.589+0000 7f19f2bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec0d02a0 con 0x7f19ec0b4da0 2023-07-18T03:38:30.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.589+0000 7f19f2bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec0b51b0 con 0x7f19ec0b8660 2023-07-18T03:38:30.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.589+0000 7f19f2bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec0cfce0 con 0x7f19ec0cd970 2023-07-18T03:38:30.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.589+0000 7f19f099b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19ec0b8660 0x7f19ec0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:30.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.589+0000 7f19f119c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f19ec0b4da0 0x7f19ec1601f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45860/0 (socket says 172.21.15.49:45860) 2023-07-18T03:38:30.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.589+0000 7f19f119c700 1 -- 172.21.15.49:0/2140621847 learned_addr learned my addr 172.21.15.49:0/2140621847 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:30.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.589+0000 7f19f099b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19ec0b8660 0x7f19ec0cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45752/0 (socket says 172.21.15.49:45752) 2023-07-18T03:38:30.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.590+0000 7f19ebfff700 1 --2- 172.21.15.49:0/2140621847 >> v2:172.21.15.107:3300/0 conn(0x7f19ec0cd970 0x7f19ec0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:30.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.590+0000 7f19f099b700 1 -- 172.21.15.49:0/2140621847 >> v2:172.21.15.107:3300/0 conn(0x7f19ec0cd970 msgr2=0x7f19ec0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:30.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.590+0000 7f19f099b700 1 --2- 172.21.15.49:0/2140621847 >> v2:172.21.15.107:3300/0 conn(0x7f19ec0cd970 0x7f19ec0b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.590+0000 7f19f099b700 1 -- 172.21.15.49:0/2140621847 >> v1:172.21.15.49:6789/0 conn(0x7f19ec0b4da0 legacy=0x7f19ec1601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.590+0000 7f19f099b700 1 -- 172.21.15.49:0/2140621847 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0cffc0 con 0x7f19ec0b8660 2023-07-18T03:38:30.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.590+0000 7f19f099b700 1 --2- 172.21.15.49:0/2140621847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec0b8660 0x7f19ec0cd430 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0d0600 tx=0x7f19e0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b855a2391ad8cd3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:30.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.590+0000 7f19eb7fe700 1 -- 172.21.15.49:0/2140621847 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0011830 con 0x7f19ec0b8660 2023-07-18T03:38:30.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.590+0000 7f19eb7fe700 1 -- 172.21.15.49:0/2140621847 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e0011990 con 0x7f19ec0b8660 2023-07-18T03:38:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.590+0000 7f19eb7fe700 1 -- 172.21.15.49:0/2140621847 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0011d20 con 0x7f19ec0b8660 2023-07-18T03:38:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.591+0000 7f19f2bff700 1 -- 172.21.15.49:0/2140621847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec0b8660 msgr2=0x7f19ec0cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.591+0000 7f19f2bff700 1 --2- 172.21.15.49:0/2140621847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec0b8660 0x7f19ec0cd430 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0d0600 tx=0x7f19e0009b20 comp rx=0 tx=0).stop 2023-07-18T03:38:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.591+0000 7f19f2bff700 1 -- 172.21.15.49:0/2140621847 shutdown_connections 2023-07-18T03:38:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.591+0000 7f19f2bff700 1 --2- 172.21.15.49:0/2140621847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec0b8660 0x7f19ec0cd430 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.591+0000 7f19f2bff700 1 --2- 172.21.15.49:0/2140621847 >> v2:172.21.15.107:3300/0 conn(0x7f19ec0cd970 0x7f19ec0b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.591+0000 7f19f2bff700 1 -- 172.21.15.49:0/2140621847 >> 172.21.15.49:0/2140621847 conn(0x7f19ec066860 msgr2=0x7f19ec06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.591+0000 7f19f2bff700 1 -- 172.21.15.49:0/2140621847 shutdown_connections 2023-07-18T03:38:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.591+0000 7f19f2bff700 1 -- 172.21.15.49:0/2140621847 wait complete. 2023-07-18T03:38:30.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.593+0000 7f19f2bff700 1 Processor -- start 2023-07-18T03:38:30.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.593+0000 7f19f2bff700 1 -- start start 2023-07-18T03:38:30.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.594+0000 7f19f2bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19ec0b4da0 0x7f19ec1f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:30.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.594+0000 7f19f2bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19ec0b8660 0x7f19ec0d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:30.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.594+0000 7f19f2bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec0cd970 0x7f19ec0d2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:30.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.594+0000 7f19f2bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0cbd10 con 0x7f19ec0cd970 2023-07-18T03:38:30.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.594+0000 7f19f2bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec15e2a0 con 0x7f19ec0b4da0 2023-07-18T03:38:30.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.594+0000 7f19f2bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0cb650 con 0x7f19ec0b8660 2023-07-18T03:38:30.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.594+0000 7f19f099b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19ec0b4da0 0x7f19ec1f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:30.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.594+0000 7f19f099b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19ec0b4da0 0x7f19ec1f3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:54844/0 (socket says 172.21.15.49:54844) 2023-07-18T03:38:30.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.594+0000 7f19f099b700 1 -- 172.21.15.49:0/2433842344 learned_addr learned my addr 172.21.15.49:0/2433842344 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:30.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.594+0000 7f19f119c700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec0cd970 0x7f19ec0d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:30.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.594+0000 7f19ebfff700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19ec0b8660 0x7f19ec0d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:30.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.595+0000 7f19f099b700 1 -- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19ec0b8660 msgr2=0x7f19ec0d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.595+0000 7f19f099b700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19ec0b8660 0x7f19ec0d1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.595+0000 7f19f099b700 1 -- 172.21.15.49:0/2433842344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec0cd970 msgr2=0x7f19ec0d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.595+0000 7f19f099b700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec0cd970 0x7f19ec0d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.595+0000 7f19f099b700 1 -- 172.21.15.49:0/2433842344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e00094a0 con 0x7f19ec0b4da0 2023-07-18T03:38:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.595+0000 7f19ebfff700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19ec0b8660 0x7f19ec0d1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.595+0000 7f19f099b700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19ec0b4da0 0x7f19ec1f3b10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009b60 tx=0x7f19e0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.595+0000 7f19e97fa700 1 -- 172.21.15.49:0/2433842344 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e00036a0 con 0x7f19ec0b4da0 2023-07-18T03:38:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.595+0000 7f19e97fa700 1 -- 172.21.15.49:0/2433842344 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e001f070 con 0x7f19ec0b4da0 2023-07-18T03:38:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.595+0000 7f19f2bff700 1 -- 172.21.15.49:0/2433842344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec0cb930 con 0x7f19ec0b4da0 2023-07-18T03:38:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.595+0000 7f19e97fa700 1 -- 172.21.15.49:0/2433842344 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0003a60 con 0x7f19ec0b4da0 2023-07-18T03:38:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.596+0000 7f19f2bff700 1 -- 172.21.15.49:0/2433842344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec1f5a00 con 0x7f19ec0b4da0 2023-07-18T03:38:30.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.597+0000 7f19f2bff700 1 -- 172.21.15.49:0/2433842344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec0b5570 con 0x7f19ec0b4da0 2023-07-18T03:38:30.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.597+0000 7f19e97fa700 1 -- 172.21.15.49:0/2433842344 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f19e0003bc0 con 0x7f19ec0b4da0 2023-07-18T03:38:30.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.599+0000 7f19e97fa700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f19d006ac50 0x7f19d006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:30.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.599+0000 7f19e97fa700 1 -- 172.21.15.49:0/2433842344 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f19e0021070 con 0x7f19ec0b4da0 2023-07-18T03:38:30.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.599+0000 7f19ebfff700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f19d006ac50 0x7f19d006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:30.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.600+0000 7f19ebfff700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f19d006ac50 0x7f19d006d100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009f30 tx=0x7f19d8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:30.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.603+0000 7f19e97fa700 1 -- 172.21.15.49:0/2433842344 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f19e0055d00 con 0x7f19ec0b4da0 2023-07-18T03:38:30.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.810+0000 7f19f2bff700 1 -- 172.21.15.49:0/2433842344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19ec074350 con 0x7f19ec0b4da0 2023-07-18T03:38:30.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.811+0000 7f19e97fa700 1 -- 172.21.15.49:0/2433842344 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19e00150a0 con 0x7f19ec0b4da0 2023-07-18T03:38:30.807 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:38:30.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.813+0000 7f19f2bff700 1 -- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f19d006ac50 msgr2=0x7f19d006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.813+0000 7f19f2bff700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f19d006ac50 0x7f19d006d100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009f30 tx=0x7f19d8009210 comp rx=0 tx=0).stop 2023-07-18T03:38:30.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.813+0000 7f19f2bff700 1 -- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19ec0b4da0 msgr2=0x7f19ec1f3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:30.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.813+0000 7f19f2bff700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19ec0b4da0 0x7f19ec1f3b10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009b60 tx=0x7f19e0004750 comp rx=0 tx=0).stop 2023-07-18T03:38:30.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.813+0000 7f19f2bff700 1 -- 172.21.15.49:0/2433842344 shutdown_connections 2023-07-18T03:38:30.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.813+0000 7f19f2bff700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f19d006ac50 0x7f19d006d100 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.813+0000 7f19f2bff700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19ec0cd970 0x7f19ec0d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.813+0000 7f19f2bff700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19ec0b4da0 0x7f19ec1f3b10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.813+0000 7f19f2bff700 1 --2- 172.21.15.49:0/2433842344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19ec0b8660 0x7f19ec0d1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:30.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.813+0000 7f19f2bff700 1 -- 172.21.15.49:0/2433842344 >> 172.21.15.49:0/2433842344 conn(0x7f19ec066860 msgr2=0x7f19ec0d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:30.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.813+0000 7f19f2bff700 1 -- 172.21.15.49:0/2433842344 shutdown_connections 2023-07-18T03:38:30.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:30.813+0000 7f19f2bff700 1 -- 172.21.15.49:0/2433842344 wait complete. 2023-07-18T03:38:30.826 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:38:31.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.041+0000 7f155d1ff700 1 Processor -- start 2023-07-18T03:38:31.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.041+0000 7f155d1ff700 1 -- start start 2023-07-18T03:38:31.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.042+0000 7f155d1ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f15580d1220 0x7f15580b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:31.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.043+0000 7f155d1ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15580d2ce0 con 0x7f15580b1e00 2023-07-18T03:38:31.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.043+0000 7f155d1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15580b3350 con 0x7f155815c950 2023-07-18T03:38:31.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.043+0000 7f155d1ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15580b3630 con 0x7f15580d1220 2023-07-18T03:38:31.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.043+0000 7f155659c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f15580d1220 0x7f15580b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:31.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.043+0000 7f1556d9d700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f155815c950 0x7f15580d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:57390/0 (socket says 172.21.15.49:57390) 2023-07-18T03:38:31.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.043+0000 7f1556d9d700 1 -- 172.21.15.49:0/1499607051 learned_addr learned my addr 172.21.15.49:0/1499607051 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:31.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.044+0000 7f1555d9b700 1 -- 172.21.15.49:0/1499607051 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402810139 0 0) 0x7f15580b3350 con 0x7f155815c950 2023-07-18T03:38:31.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.044+0000 7f1555d9b700 1 -- 172.21.15.49:0/1499607051 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15400035b0 con 0x7f155815c950 2023-07-18T03:38:31.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.044+0000 7f155659c700 1 -- 172.21.15.49:0/1499607051 >> v1:172.21.15.107:6789/0 conn(0x7f155815c950 legacy=0x7f15580d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.044+0000 7f155659c700 1 -- 172.21.15.49:0/1499607051 >> v1:172.21.15.49:6789/0 conn(0x7f15580b1e00 legacy=0x7f15580afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.044+0000 7f155659c700 1 -- 172.21.15.49:0/1499607051 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15580b3910 con 0x7f15580d1220 2023-07-18T03:38:31.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.044+0000 7f155659c700 1 --2- 172.21.15.49:0/1499607051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15580d1220 0x7f15580b43f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f154c009840 tx=0x7f154c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa559ce6ac0d62b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:31.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.187+0000 7f1555d9b700 1 -- 172.21.15.49:0/1499607051 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c011830 con 0x7f15580d1220 2023-07-18T03:38:31.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.188+0000 7f1555d9b700 1 -- 172.21.15.49:0/1499607051 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c011990 con 0x7f15580d1220 2023-07-18T03:38:31.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.188+0000 7f1555d9b700 1 -- 172.21.15.49:0/1499607051 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c011d00 con 0x7f15580d1220 2023-07-18T03:38:31.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.188+0000 7f155d1ff700 1 -- 172.21.15.49:0/1499607051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15580d1220 msgr2=0x7f15580b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.188+0000 7f155d1ff700 1 --2- 172.21.15.49:0/1499607051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15580d1220 0x7f15580b43f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f154c009840 tx=0x7f154c009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:31.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.188+0000 7f155d1ff700 1 -- 172.21.15.49:0/1499607051 shutdown_connections 2023-07-18T03:38:31.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.188+0000 7f155d1ff700 1 --2- 172.21.15.49:0/1499607051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15580d1220 0x7f15580b43f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.188+0000 7f155d1ff700 1 -- 172.21.15.49:0/1499607051 >> 172.21.15.49:0/1499607051 conn(0x7f155809c1d0 msgr2=0x7f15580d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:31.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.188+0000 7f155d1ff700 1 -- 172.21.15.49:0/1499607051 shutdown_connections 2023-07-18T03:38:31.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.189+0000 7f155d1ff700 1 -- 172.21.15.49:0/1499607051 wait complete. 2023-07-18T03:38:31.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.190+0000 7f155d1ff700 1 Processor -- start 2023-07-18T03:38:31.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.191+0000 7f155d1ff700 1 -- start start 2023-07-18T03:38:31.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.191+0000 7f155d1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15580b1e00 0x7f15580cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:31.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f155d1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15580d1220 0x7f15581e3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:31.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f155d1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f155815c950 0x7f15581e4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:31.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f155d1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15581e53e0 con 0x7f15580b1e00 2023-07-18T03:38:31.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f155d1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15581e56c0 con 0x7f155815c950 2023-07-18T03:38:31.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f155d1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15581e59a0 con 0x7f15580d1220 2023-07-18T03:38:31.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f1556d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15580b1e00 0x7f15580cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:31.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f1556d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15580b1e00 0x7f15580cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45790/0 (socket says 172.21.15.49:45790) 2023-07-18T03:38:31.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f1556d9d700 1 -- 172.21.15.49:0/3013277531 learned_addr learned my addr 172.21.15.49:0/3013277531 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:31.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f155659c700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15580d1220 0x7f15581e3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:31.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f1556d9d700 1 -- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15580d1220 msgr2=0x7f15581e3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f1556d9d700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15580d1220 0x7f15581e3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f1556d9d700 1 -- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f155815c950 msgr2=0x7f15581e4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f1556d9d700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f155815c950 0x7f15581e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f1556d9d700 1 -- 172.21.15.49:0/3013277531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0094a0 con 0x7f15580b1e00 2023-07-18T03:38:31.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.192+0000 7f155659c700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15580d1220 0x7f15581e3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:31.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.193+0000 7f1556d9d700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15580b1e00 0x7f15580cd550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1548001130 tx=0x7f154800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:31.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.193+0000 7f153f7fe700 1 -- 172.21.15.49:0/3013277531 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548005dc0 con 0x7f15580b1e00 2023-07-18T03:38:31.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.193+0000 7f153f7fe700 1 -- 172.21.15.49:0/3013277531 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15480070d0 con 0x7f15580b1e00 2023-07-18T03:38:31.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.193+0000 7f153f7fe700 1 -- 172.21.15.49:0/3013277531 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548007460 con 0x7f15580b1e00 2023-07-18T03:38:31.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.193+0000 7f155d1ff700 1 -- 172.21.15.49:0/3013277531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15581e5c80 con 0x7f15580b1e00 2023-07-18T03:38:31.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.193+0000 7f155d1ff700 1 -- 172.21.15.49:0/3013277531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15580c6a70 con 0x7f15580b1e00 2023-07-18T03:38:31.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.194+0000 7f155d1ff700 1 -- 172.21.15.49:0/3013277531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15580c6c00 con 0x7f15580b1e00 2023-07-18T03:38:31.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.195+0000 7f153f7fe700 1 -- 172.21.15.49:0/3013277531 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f15480075c0 con 0x7f15580b1e00 2023-07-18T03:38:31.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.196+0000 7f153f7fe700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f154406ace0 0x7f154406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:31.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.196+0000 7f153f7fe700 1 -- 172.21.15.49:0/3013277531 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1548088900 con 0x7f15580b1e00 2023-07-18T03:38:31.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.197+0000 7f155659c700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f154406ace0 0x7f154406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:31.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.197+0000 7f155659c700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f154406ace0 0x7f154406d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15580d1610 tx=0x7f154c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:31.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.200+0000 7f153f7fe700 1 -- 172.21.15.49:0/3013277531 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1548053ce0 con 0x7f15580b1e00 2023-07-18T03:38:31.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.455+0000 7f155d1ff700 1 -- 172.21.15.49:0/3013277531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15580c6c00 con 0x7f15580b1e00 2023-07-18T03:38:31.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.455+0000 7f153f7fe700 1 -- 172.21.15.49:0/3013277531 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f15580c6c00 con 0x7f15580b1e00 2023-07-18T03:38:31.450 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:31.450 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1477016053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1477016053},{"type":"v1","addr":"172.21.15.107:6835","nonce":1477016053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2998451500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2998451500},{"type":"v1","addr":"172.21.15.49:6835","nonce":2998451500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4247281109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4247281109},{"type":"v1","addr":"172.21.15.49:6833","nonce":4247281109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/1127901907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1127901907},{"type":"v1","addr":"172.21.15.107:6837","nonce":1127901907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-18T03:38:31.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.457+0000 7f155d1ff700 1 -- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f154406ace0 msgr2=0x7f154406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.457+0000 7f155d1ff700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f154406ace0 0x7f154406d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15580d1610 tx=0x7f154c008030 comp rx=0 tx=0).stop 2023-07-18T03:38:31.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.457+0000 7f155d1ff700 1 -- 172.21.15.49:0/3013277531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15580b1e00 msgr2=0x7f15580cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.457+0000 7f155d1ff700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15580b1e00 0x7f15580cd550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1548001130 tx=0x7f154800c7b0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.458+0000 7f155d1ff700 1 -- 172.21.15.49:0/3013277531 shutdown_connections 2023-07-18T03:38:31.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.458+0000 7f155d1ff700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f155815c950 0x7f15581e4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.458+0000 7f155d1ff700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f154406ace0 0x7f154406d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.458+0000 7f155d1ff700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15580b1e00 0x7f15580cd550 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.458+0000 7f155d1ff700 1 --2- 172.21.15.49:0/3013277531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15580d1220 0x7f15581e3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.458+0000 7f155d1ff700 1 -- 172.21.15.49:0/3013277531 >> 172.21.15.49:0/3013277531 conn(0x7f155809c1d0 msgr2=0x7f15580b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:31.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.458+0000 7f155d1ff700 1 -- 172.21.15.49:0/3013277531 shutdown_connections 2023-07-18T03:38:31.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.458+0000 7f155d1ff700 1 -- 172.21.15.49:0/3013277531 wait complete. 2023-07-18T03:38:31.454 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 3 2023-07-18T03:38:31.471 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:31.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.683+0000 7f163a7ff700 1 Processor -- start 2023-07-18T03:38:31.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.684+0000 7f163a7ff700 1 -- start start 2023-07-18T03:38:31.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.685+0000 7f163a7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16340d1220 0x7f16340b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:31.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.685+0000 7f163a7ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16340d2ce0 con 0x7f16340b1e00 2023-07-18T03:38:31.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.685+0000 7f163a7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16340b3350 con 0x7f163415c950 2023-07-18T03:38:31.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.685+0000 7f163a7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340b3630 con 0x7f16340d1220 2023-07-18T03:38:31.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.685+0000 7f1638d9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f16340b1e00 0x7f16340afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45886/0 (socket says 172.21.15.49:45886) 2023-07-18T03:38:31.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.685+0000 7f1638d9c700 1 -- 172.21.15.49:0/1660809973 learned_addr learned my addr 172.21.15.49:0/1660809973 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:31.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.685+0000 7f16337fe700 1 --2- 172.21.15.49:0/1660809973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16340d1220 0x7f16340b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:31.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.685+0000 7f1632ffd700 1 -- 172.21.15.49:0/1660809973 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056323346 0 0) 0x7f16340d2ce0 con 0x7f16340b1e00 2023-07-18T03:38:31.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.685+0000 7f1632ffd700 1 -- 172.21.15.49:0/1660809973 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f161c0035b0 con 0x7f16340b1e00 2023-07-18T03:38:31.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.686+0000 7f1632ffd700 1 -- 172.21.15.49:0/1660809973 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824233390 0 0) 0x7f16340b3350 con 0x7f163415c950 2023-07-18T03:38:31.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.686+0000 7f1632ffd700 1 -- 172.21.15.49:0/1660809973 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16340d2ce0 con 0x7f163415c950 2023-07-18T03:38:31.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.686+0000 7f16337fe700 1 -- 172.21.15.49:0/1660809973 >> v1:172.21.15.107:6789/0 conn(0x7f163415c950 legacy=0x7f16340d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.686+0000 7f16337fe700 1 -- 172.21.15.49:0/1660809973 >> v1:172.21.15.49:6789/0 conn(0x7f16340b1e00 legacy=0x7f16340afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.686+0000 7f1632ffd700 1 -- 172.21.15.49:0/1660809973 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3334322909 0 0) 0x7f161c0035b0 con 0x7f16340b1e00 2023-07-18T03:38:31.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.686+0000 7f16337fe700 1 -- 172.21.15.49:0/1660809973 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340b3910 con 0x7f16340d1220 2023-07-18T03:38:31.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.686+0000 7f16337fe700 1 --2- 172.21.15.49:0/1660809973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16340d1220 0x7f16340b43f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1628009870 tx=0x7f1628009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72a903a121b82d65 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.686+0000 7f1632ffd700 1 -- 172.21.15.49:0/1660809973 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c0019f0 con 0x7f16340d1220 2023-07-18T03:38:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.686+0000 7f1632ffd700 1 -- 172.21.15.49:0/1660809973 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1628004030 con 0x7f16340d1220 2023-07-18T03:38:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.686+0000 7f1632ffd700 1 -- 172.21.15.49:0/1660809973 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16280043c0 con 0x7f16340d1220 2023-07-18T03:38:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.687+0000 7f163a7ff700 1 -- 172.21.15.49:0/1660809973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16340d1220 msgr2=0x7f16340b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.687+0000 7f163a7ff700 1 --2- 172.21.15.49:0/1660809973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16340d1220 0x7f16340b43f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1628009870 tx=0x7f1628009b80 comp rx=0 tx=0).stop 2023-07-18T03:38:31.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.687+0000 7f163a7ff700 1 -- 172.21.15.49:0/1660809973 shutdown_connections 2023-07-18T03:38:31.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.687+0000 7f163a7ff700 1 --2- 172.21.15.49:0/1660809973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16340d1220 0x7f16340b43f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.687+0000 7f163a7ff700 1 -- 172.21.15.49:0/1660809973 >> 172.21.15.49:0/1660809973 conn(0x7f163409c1d0 msgr2=0x7f16340d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:31.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.687+0000 7f163a7ff700 1 -- 172.21.15.49:0/1660809973 shutdown_connections 2023-07-18T03:38:31.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.687+0000 7f163a7ff700 1 -- 172.21.15.49:0/1660809973 wait complete. 2023-07-18T03:38:31.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.689+0000 7f163a7ff700 1 Processor -- start 2023-07-18T03:38:31.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.689+0000 7f163a7ff700 1 -- start start 2023-07-18T03:38:31.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.690+0000 7f163a7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16340b1e00 0x7f1634152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:31.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.690+0000 7f163a7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16340d1220 0x7f16341e9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:31.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.690+0000 7f163a7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163415c950 0x7f16341e9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:31.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.690+0000 7f163a7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16340cd710 con 0x7f16340d1220 2023-07-18T03:38:31.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.690+0000 7f163a7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16340cd9f0 con 0x7f163415c950 2023-07-18T03:38:31.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.690+0000 7f163a7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340cdcd0 con 0x7f16340b1e00 2023-07-18T03:38:31.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.691+0000 7f16337fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16340d1220 0x7f16341e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:31.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.691+0000 7f16337fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16340d1220 0x7f16341e9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45804/0 (socket says 172.21.15.49:45804) 2023-07-18T03:38:31.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.691+0000 7f16337fe700 1 -- 172.21.15.49:0/471925643 learned_addr learned my addr 172.21.15.49:0/471925643 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:31.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.691+0000 7f1633fff700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16340b1e00 0x7f1634152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:31.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.691+0000 7f1638d9c700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163415c950 0x7f16341e9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:31.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.691+0000 7f16337fe700 1 -- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16340b1e00 msgr2=0x7f1634152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.691+0000 7f16337fe700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16340b1e00 0x7f1634152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.691+0000 7f16337fe700 1 -- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163415c950 msgr2=0x7f16341e9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.691+0000 7f16337fe700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163415c950 0x7f16341e9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.691+0000 7f16337fe700 1 -- 172.21.15.49:0/471925643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16280094a0 con 0x7f16340d1220 2023-07-18T03:38:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.691+0000 7f16337fe700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16340d1220 0x7f16341e9660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f16280090b0 tx=0x7f1628004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.692+0000 7f1630ff9700 1 -- 172.21.15.49:0/471925643 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628003e60 con 0x7f16340d1220 2023-07-18T03:38:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.692+0000 7f1630ff9700 1 -- 172.21.15.49:0/471925643 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162801f070 con 0x7f16340d1220 2023-07-18T03:38:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.692+0000 7f1630ff9700 1 -- 172.21.15.49:0/471925643 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628011850 con 0x7f16340d1220 2023-07-18T03:38:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.692+0000 7f163a7ff700 1 -- 172.21.15.49:0/471925643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16340cdfb0 con 0x7f16340d1220 2023-07-18T03:38:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.692+0000 7f163a7ff700 1 -- 172.21.15.49:0/471925643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16340c8510 con 0x7f16340d1220 2023-07-18T03:38:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.693+0000 7f163a7ff700 1 -- 172.21.15.49:0/471925643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16340ce4d0 con 0x7f16340d1220 2023-07-18T03:38:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.694+0000 7f1630ff9700 1 -- 172.21.15.49:0/471925643 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f16280119b0 con 0x7f16340d1220 2023-07-18T03:38:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.695+0000 7f1630ff9700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f162006ace0 0x7f162006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.695+0000 7f1630ff9700 1 -- 172.21.15.49:0/471925643 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1628021070 con 0x7f16340d1220 2023-07-18T03:38:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.695+0000 7f1633fff700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f162006ace0 0x7f162006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.696+0000 7f1633fff700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f162006ace0 0x7f162006d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1624001270 tx=0x7f1624009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.699+0000 7f1630ff9700 1 -- 172.21.15.49:0/471925643 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f16340ce4d0 con 0x7f16340d1220 2023-07-18T03:38:31.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.912+0000 7f163a7ff700 1 -- 172.21.15.49:0/471925643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16340ce4d0 con 0x7f16340d1220 2023-07-18T03:38:31.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.913+0000 7f1630ff9700 1 -- 172.21.15.49:0/471925643 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14543 (secure 0 0 0) 0x7f16340ce4d0 con 0x7f16340d1220 2023-07-18T03:38:31.909 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:31.909 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":18,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:30.335723+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:31.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.916+0000 7f163a7ff700 1 -- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f162006ace0 msgr2=0x7f162006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.916+0000 7f163a7ff700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f162006ace0 0x7f162006d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1624001270 tx=0x7f1624009040 comp rx=0 tx=0).stop 2023-07-18T03:38:31.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.916+0000 7f163a7ff700 1 -- 172.21.15.49:0/471925643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16340d1220 msgr2=0x7f16341e9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:31.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.916+0000 7f163a7ff700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16340d1220 0x7f16341e9660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f16280090b0 tx=0x7f1628004750 comp rx=0 tx=0).stop 2023-07-18T03:38:31.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.916+0000 7f163a7ff700 1 -- 172.21.15.49:0/471925643 shutdown_connections 2023-07-18T03:38:31.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.916+0000 7f163a7ff700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f163415c950 0x7f16341e9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.916+0000 7f163a7ff700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16340b1e00 0x7f1634152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.916+0000 7f163a7ff700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f162006ace0 0x7f162006d190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.916+0000 7f163a7ff700 1 --2- 172.21.15.49:0/471925643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16340d1220 0x7f16341e9660 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:31.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.916+0000 7f163a7ff700 1 -- 172.21.15.49:0/471925643 >> 172.21.15.49:0/471925643 conn(0x7f163409c1d0 msgr2=0x7f16340d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:31.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.916+0000 7f163a7ff700 1 -- 172.21.15.49:0/471925643 shutdown_connections 2023-07-18T03:38:31.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:31.916+0000 7f163a7ff700 1 -- 172.21.15.49:0/471925643 wait complete. 2023-07-18T03:38:31.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:38:31.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:38:32.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.144+0000 7f6e7f9ff700 1 Processor -- start 2023-07-18T03:38:32.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.144+0000 7f6e7f9ff700 1 -- start start 2023-07-18T03:38:32.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7f9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6e780b8a60 0x7f6e780b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:32.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7f9ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6e780b4530 0x7f6e780d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:32.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7f9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e780d1490 con 0x7f6e780d1050 2023-07-18T03:38:32.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7f9ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e780b8e30 con 0x7f6e780b4530 2023-07-18T03:38:32.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7f9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e780b1e30 con 0x7f6e780b8a60 2023-07-18T03:38:32.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7cf9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6e780b4530 0x7f6e780d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:32.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7cf9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6e780b4530 0x7f6e780d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45820/0 (socket says 172.21.15.49:45820) 2023-07-18T03:38:32.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7cf9a700 1 -- 172.21.15.49:0/189006308 learned_addr learned my addr 172.21.15.49:0/189006308 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:32.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7d79b700 1 --2- 172.21.15.49:0/189006308 >> v2:172.21.15.107:3300/0 conn(0x7f6e780b8a60 0x7f6e780b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:32.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7cf9a700 1 -- 172.21.15.49:0/189006308 >> v2:172.21.15.107:3300/0 conn(0x7f6e780b8a60 msgr2=0x7f6e780b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:32.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7cf9a700 1 --2- 172.21.15.49:0/189006308 >> v2:172.21.15.107:3300/0 conn(0x7f6e780b8a60 0x7f6e780b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7cf9a700 1 -- 172.21.15.49:0/189006308 >> v1:172.21.15.107:6789/0 conn(0x7f6e780d1050 legacy=0x7f6e780b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:32.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7cf9a700 1 -- 172.21.15.49:0/189006308 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e780afaa0 con 0x7f6e780b4530 2023-07-18T03:38:32.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.146+0000 7f6e7d79b700 1 --2- 172.21.15.49:0/189006308 >> v2:172.21.15.107:3300/0 conn(0x7f6e780b8a60 0x7f6e780b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:32.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.147+0000 7f6e7cf9a700 1 --2- 172.21.15.49:0/189006308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e780b4530 0x7f6e780d0b10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6e780b5b00 tx=0x7f6e68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164fa2f1336706d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:32.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.147+0000 7f6e6ffff700 1 -- 172.21.15.49:0/189006308 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800b870 con 0x7f6e780b4530 2023-07-18T03:38:32.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.147+0000 7f6e6ffff700 1 -- 172.21.15.49:0/189006308 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6800b9d0 con 0x7f6e780b4530 2023-07-18T03:38:32.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.147+0000 7f6e6ffff700 1 -- 172.21.15.49:0/189006308 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800bd60 con 0x7f6e780b4530 2023-07-18T03:38:32.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.147+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/189006308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e780b4530 msgr2=0x7f6e780d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:32.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.147+0000 7f6e7f9ff700 1 --2- 172.21.15.49:0/189006308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e780b4530 0x7f6e780d0b10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6e780b5b00 tx=0x7f6e68009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.148+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/189006308 shutdown_connections 2023-07-18T03:38:32.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.148+0000 7f6e7f9ff700 1 --2- 172.21.15.49:0/189006308 >> v2:172.21.15.107:3300/0 conn(0x7f6e780b8a60 0x7f6e780b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.148+0000 7f6e7f9ff700 1 --2- 172.21.15.49:0/189006308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e780b4530 0x7f6e780d0b10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.148+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/189006308 >> 172.21.15.49:0/189006308 conn(0x7f6e7809c1b0 msgr2=0x7f6e780d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:32.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.148+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/189006308 shutdown_connections 2023-07-18T03:38:32.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.148+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/189006308 wait complete. 2023-07-18T03:38:32.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.149+0000 7f6e7f9ff700 1 Processor -- start 2023-07-18T03:38:32.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.150+0000 7f6e7f9ff700 1 -- start start 2023-07-18T03:38:32.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.150+0000 7f6e7f9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e780b4530 0x7f6e780d0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:32.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.150+0000 7f6e7f9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e780b8a60 0x7f6e780cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:32.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7f9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e780d1050 0x7f6e780ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:32.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7f9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780ca550 con 0x7f6e780b8a60 2023-07-18T03:38:32.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7f9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780cf090 con 0x7f6e780d1050 2023-07-18T03:38:32.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7f9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780cf370 con 0x7f6e780b4530 2023-07-18T03:38:32.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7cf9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e780b8a60 0x7f6e780cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:32.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7cf9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e780b8a60 0x7f6e780cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45822/0 (socket says 172.21.15.49:45822) 2023-07-18T03:38:32.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7cf9a700 1 -- 172.21.15.49:0/3071684378 learned_addr learned my addr 172.21.15.49:0/3071684378 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:32.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7d79b700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e780b4530 0x7f6e780d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:32.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7df9c700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e780d1050 0x7f6e780ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:32.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7cf9a700 1 -- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e780b4530 msgr2=0x7f6e780d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:32.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7cf9a700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e780b4530 0x7f6e780d0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7cf9a700 1 -- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e780d1050 msgr2=0x7f6e780ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:32.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7cf9a700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e780d1050 0x7f6e780ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7cf9a700 1 -- 172.21.15.49:0/3071684378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68009550 con 0x7f6e780b8a60 2023-07-18T03:38:32.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7d79b700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e780b4530 0x7f6e780d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:32.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.151+0000 7f6e7cf9a700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e780b8a60 0x7f6e780cea00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6e68000c00 tx=0x7f6e68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:32.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.152+0000 7f6e6dffb700 1 -- 172.21.15.49:0/3071684378 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68004540 con 0x7f6e780b8a60 2023-07-18T03:38:32.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.152+0000 7f6e6dffb700 1 -- 172.21.15.49:0/3071684378 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e680036a0 con 0x7f6e780b8a60 2023-07-18T03:38:32.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.152+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/3071684378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e780cf650 con 0x7f6e780b8a60 2023-07-18T03:38:32.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.152+0000 7f6e6dffb700 1 -- 172.21.15.49:0/3071684378 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68003a30 con 0x7f6e780b8a60 2023-07-18T03:38:32.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.152+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/3071684378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e780cfc10 con 0x7f6e780b8a60 2023-07-18T03:38:32.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.153+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/3071684378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e780cfda0 con 0x7f6e780b8a60 2023-07-18T03:38:32.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.154+0000 7f6e6dffb700 1 -- 172.21.15.49:0/3071684378 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6e68021070 con 0x7f6e780b8a60 2023-07-18T03:38:32.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.155+0000 7f6e6dffb700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6e5c06ace0 0x7f6e5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:32.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.155+0000 7f6e6dffb700 1 -- 172.21.15.49:0/3071684378 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6e6808a6e0 con 0x7f6e780b8a60 2023-07-18T03:38:32.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.155+0000 7f6e7d79b700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6e5c06ace0 0x7f6e5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:32.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.156+0000 7f6e7d79b700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6e5c06ace0 0x7f6e5c06d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009a50 tx=0x7f6e74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:32.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.159+0000 7f6e6dffb700 1 -- 172.21.15.49:0/3071684378 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6e68055a60 con 0x7f6e780b8a60 2023-07-18T03:38:32.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.367+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/3071684378 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6e78074350 con 0x7f6e780b8a60 2023-07-18T03:38:32.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.368+0000 7f6e6dffb700 1 -- 172.21.15.49:0/3071684378 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6e68015020 con 0x7f6e780b8a60 2023-07-18T03:38:32.363 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:38:32.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.370+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6e5c06ace0 msgr2=0x7f6e5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:32.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.370+0000 7f6e7f9ff700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6e5c06ace0 0x7f6e5c06d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009a50 tx=0x7f6e74008040 comp rx=0 tx=0).stop 2023-07-18T03:38:32.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.370+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/3071684378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e780b8a60 msgr2=0x7f6e780cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:32.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.370+0000 7f6e7f9ff700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e780b8a60 0x7f6e780cea00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6e68000c00 tx=0x7f6e68004750 comp rx=0 tx=0).stop 2023-07-18T03:38:32.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.370+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/3071684378 shutdown_connections 2023-07-18T03:38:32.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.370+0000 7f6e7f9ff700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e780d1050 0x7f6e780ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.370+0000 7f6e7f9ff700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e780b4530 0x7f6e780d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.370+0000 7f6e7f9ff700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6e5c06ace0 0x7f6e5c06d190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.370+0000 7f6e7f9ff700 1 --2- 172.21.15.49:0/3071684378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e780b8a60 0x7f6e780cea00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.370+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/3071684378 >> 172.21.15.49:0/3071684378 conn(0x7f6e7809c1b0 msgr2=0x7f6e7806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:32.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.370+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/3071684378 shutdown_connections 2023-07-18T03:38:32.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.370+0000 7f6e7f9ff700 1 -- 172.21.15.49:0/3071684378 wait complete. 2023-07-18T03:38:32.383 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:38:32.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.597+0000 7fa34a7ff700 1 Processor -- start 2023-07-18T03:38:32.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.597+0000 7fa34a7ff700 1 -- start start 2023-07-18T03:38:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.598+0000 7fa34a7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3440d0b10 0x7fa3440d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.598+0000 7fa34a7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa3440b0e20 0x7fa3440b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.598+0000 7fa34a7ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3440d14a0 con 0x7fa3440b8a60 2023-07-18T03:38:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.599+0000 7fa34a7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3440b8e30 con 0x7fa3440b0e20 2023-07-18T03:38:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.599+0000 7fa34a7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3440afaa0 con 0x7fa3440d0b10 2023-07-18T03:38:32.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.599+0000 7fa343fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa3440b8a60 0x7fa3440b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45900/0 (socket says 172.21.15.49:45900) 2023-07-18T03:38:32.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.599+0000 7fa343fff700 1 -- 172.21.15.49:0/1910358979 learned_addr learned my addr 172.21.15.49:0/1910358979 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:32.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.599+0000 7fa3437fe700 1 --2- 172.21.15.49:0/1910358979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3440d0b10 0x7fa3440d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:32.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.599+0000 7fa348d9c700 1 --2- 172.21.15.49:0/1910358979 >> v2:172.21.15.107:3300/0 conn(0x7fa3440b0e20 0x7fa3440b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:32.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.599+0000 7fa342ffd700 1 -- 172.21.15.49:0/1910358979 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487597516 0 0) 0x7fa334001110 con 0x7fa3440b8a60 2023-07-18T03:38:32.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.599+0000 7fa342ffd700 1 -- 172.21.15.49:0/1910358979 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa32c0035b0 con 0x7fa3440b8a60 2023-07-18T03:38:32.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.599+0000 7fa3437fe700 1 -- 172.21.15.49:0/1910358979 >> v2:172.21.15.107:3300/0 conn(0x7fa3440b0e20 msgr2=0x7fa3440b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:32.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.599+0000 7fa3437fe700 1 --2- 172.21.15.49:0/1910358979 >> v2:172.21.15.107:3300/0 conn(0x7fa3440b0e20 0x7fa3440b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.599+0000 7fa3437fe700 1 -- 172.21.15.49:0/1910358979 >> v1:172.21.15.49:6789/0 conn(0x7fa3440b8a60 legacy=0x7fa3440b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:32.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.599+0000 7fa3437fe700 1 -- 172.21.15.49:0/1910358979 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3440afd80 con 0x7fa3440d0b10 2023-07-18T03:38:32.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.600+0000 7fa3437fe700 1 --2- 172.21.15.49:0/1910358979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3440d0b10 0x7fa3440d0f60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa338009840 tx=0x7fa338009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb0a8721d435390f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:32.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.600+0000 7fa342ffd700 1 -- 172.21.15.49:0/1910358979 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338011830 con 0x7fa3440d0b10 2023-07-18T03:38:32.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.600+0000 7fa342ffd700 1 -- 172.21.15.49:0/1910358979 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa338011990 con 0x7fa3440d0b10 2023-07-18T03:38:32.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.600+0000 7fa342ffd700 1 -- 172.21.15.49:0/1910358979 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338011d00 con 0x7fa3440d0b10 2023-07-18T03:38:32.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.601+0000 7fa34a7ff700 1 -- 172.21.15.49:0/1910358979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3440d0b10 msgr2=0x7fa3440d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:32.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.601+0000 7fa34a7ff700 1 --2- 172.21.15.49:0/1910358979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3440d0b10 0x7fa3440d0f60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa338009840 tx=0x7fa338009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:32.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.601+0000 7fa34a7ff700 1 -- 172.21.15.49:0/1910358979 shutdown_connections 2023-07-18T03:38:32.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.601+0000 7fa34a7ff700 1 --2- 172.21.15.49:0/1910358979 >> v2:172.21.15.107:3300/0 conn(0x7fa3440b0e20 0x7fa3440b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.601+0000 7fa34a7ff700 1 --2- 172.21.15.49:0/1910358979 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3440d0b10 0x7fa3440d0f60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.601+0000 7fa34a7ff700 1 -- 172.21.15.49:0/1910358979 >> 172.21.15.49:0/1910358979 conn(0x7fa34409c1b0 msgr2=0x7fa3440d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:32.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.601+0000 7fa34a7ff700 1 -- 172.21.15.49:0/1910358979 shutdown_connections 2023-07-18T03:38:32.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.601+0000 7fa34a7ff700 1 -- 172.21.15.49:0/1910358979 wait complete. 2023-07-18T03:38:32.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.603+0000 7fa34a7ff700 1 Processor -- start 2023-07-18T03:38:32.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.603+0000 7fa34a7ff700 1 -- start start 2023-07-18T03:38:32.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa34a7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3440b0e20 0x7fa3440cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:32.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa34a7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3440b8a60 0x7fa3440cf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:32.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa34a7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3440d0b10 0x7fa3440cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:32.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa34a7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3440cfb60 con 0x7fa3440b0e20 2023-07-18T03:38:32.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa34a7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa344151f80 con 0x7fa3440d0b10 2023-07-18T03:38:32.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa34a7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3440cfe10 con 0x7fa3440b8a60 2023-07-18T03:38:32.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa343fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3440b0e20 0x7fa3440cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:32.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa343fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3440b0e20 0x7fa3440cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45834/0 (socket says 172.21.15.49:45834) 2023-07-18T03:38:32.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa343fff700 1 -- 172.21.15.49:0/2747526011 learned_addr learned my addr 172.21.15.49:0/2747526011 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:32.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa3437fe700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3440b8a60 0x7fa3440cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:32.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa348d9c700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3440d0b10 0x7fa3440cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:32.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa343fff700 1 -- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3440b8a60 msgr2=0x7fa3440cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:32.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa343fff700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3440b8a60 0x7fa3440cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa343fff700 1 -- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3440d0b10 msgr2=0x7fa3440cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:32.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa343fff700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3440d0b10 0x7fa3440cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa343fff700 1 -- 172.21.15.49:0/2747526011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3380094a0 con 0x7fa3440b0e20 2023-07-18T03:38:32.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa3437fe700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3440b8a60 0x7fa3440cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:32.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.604+0000 7fa348d9c700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3440d0b10 0x7fa3440cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:32.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.605+0000 7fa343fff700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3440b0e20 0x7fa3440cf0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa33400c4d0 tx=0x7fa33400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:32.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.605+0000 7fa340ff9700 1 -- 172.21.15.49:0/2747526011 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334005d70 con 0x7fa3440b0e20 2023-07-18T03:38:32.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.605+0000 7fa340ff9700 1 -- 172.21.15.49:0/2747526011 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3340070d0 con 0x7fa3440b0e20 2023-07-18T03:38:32.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.605+0000 7fa340ff9700 1 -- 172.21.15.49:0/2747526011 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334007480 con 0x7fa3440b0e20 2023-07-18T03:38:32.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.605+0000 7fa34a7ff700 1 -- 172.21.15.49:0/2747526011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3440d00f0 con 0x7fa3440b0e20 2023-07-18T03:38:32.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.605+0000 7fa34a7ff700 1 -- 172.21.15.49:0/2747526011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34415e4e0 con 0x7fa3440b0e20 2023-07-18T03:38:32.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.606+0000 7fa34a7ff700 1 -- 172.21.15.49:0/2747526011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa344074350 con 0x7fa3440b0e20 2023-07-18T03:38:32.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.607+0000 7fa340ff9700 1 -- 172.21.15.49:0/2747526011 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3340075e0 con 0x7fa3440b0e20 2023-07-18T03:38:32.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.608+0000 7fa340ff9700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa33006ace0 0x7fa33006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:32.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.608+0000 7fa340ff9700 1 -- 172.21.15.49:0/2747526011 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa334088520 con 0x7fa3440b0e20 2023-07-18T03:38:32.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.608+0000 7fa3437fe700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa33006ace0 0x7fa33006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:32.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.609+0000 7fa3437fe700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa33006ace0 0x7fa33006d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa338009fd0 tx=0x7fa338008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:32.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.612+0000 7fa340ff9700 1 -- 172.21.15.49:0/2747526011 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa334053920 con 0x7fa3440b0e20 2023-07-18T03:38:32.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.818+0000 7fa34a7ff700 1 -- 172.21.15.49:0/2747526011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3440d14a0 con 0x7fa3440b0e20 2023-07-18T03:38:32.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.818+0000 7fa340ff9700 1 -- 172.21.15.49:0/2747526011 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa334017020 con 0x7fa3440b0e20 2023-07-18T03:38:32.813 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:38:32.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.820+0000 7fa34a7ff700 1 -- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa33006ace0 msgr2=0x7fa33006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:32.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.820+0000 7fa34a7ff700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa33006ace0 0x7fa33006d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa338009fd0 tx=0x7fa338008030 comp rx=0 tx=0).stop 2023-07-18T03:38:32.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.820+0000 7fa34a7ff700 1 -- 172.21.15.49:0/2747526011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3440b0e20 msgr2=0x7fa3440cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:32.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.820+0000 7fa34a7ff700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3440b0e20 0x7fa3440cf0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa33400c4d0 tx=0x7fa33400c890 comp rx=0 tx=0).stop 2023-07-18T03:38:32.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.820+0000 7fa34a7ff700 1 -- 172.21.15.49:0/2747526011 shutdown_connections 2023-07-18T03:38:32.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.820+0000 7fa34a7ff700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3440d0b10 0x7fa3440cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.820+0000 7fa34a7ff700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa33006ace0 0x7fa33006d190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.820+0000 7fa34a7ff700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3440b0e20 0x7fa3440cf0e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.820+0000 7fa34a7ff700 1 --2- 172.21.15.49:0/2747526011 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3440b8a60 0x7fa3440cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:32.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.820+0000 7fa34a7ff700 1 -- 172.21.15.49:0/2747526011 >> 172.21.15.49:0/2747526011 conn(0x7fa34409c1b0 msgr2=0x7fa3440b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:32.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.820+0000 7fa34a7ff700 1 -- 172.21.15.49:0/2747526011 shutdown_connections 2023-07-18T03:38:32.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:32.820+0000 7fa34a7ff700 1 -- 172.21.15.49:0/2747526011 wait complete. 2023-07-18T03:38:32.834 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-18T03:38:33.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.043+0000 7fea26fff700 1 Processor -- start 2023-07-18T03:38:33.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.044+0000 7fea26fff700 1 -- start start 2023-07-18T03:38:33.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea26fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fea200cfc60 0x7fea20160860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea26fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fea2006ae40 0x7fea200d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea26fff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea200cd4e0 con 0x7fea200b87d0 2023-07-18T03:38:33.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea26fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea200cdcf0 con 0x7fea200cfc60 2023-07-18T03:38:33.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea26fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea200d00d0 con 0x7fea2006ae40 2023-07-18T03:38:33.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea1ffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fea200cfc60 0x7fea20160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea1ffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fea200cfc60 0x7fea20160860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45850/0 (socket says 172.21.15.49:45850) 2023-07-18T03:38:33.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea1ffff700 1 -- 172.21.15.49:0/3887761503 learned_addr learned my addr 172.21.15.49:0/3887761503 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:33.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea24d9b700 1 --1- 172.21.15.49:0/3887761503 >> v1:172.21.15.49:6789/0 conn(0x7fea200b87d0 0x7fea20160150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45910/0 (socket says 172.21.15.49:45910) 2023-07-18T03:38:33.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea2559c700 1 --2- 172.21.15.49:0/3887761503 >> v2:172.21.15.107:3300/0 conn(0x7fea2006ae40 0x7fea200d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea1ffff700 1 -- 172.21.15.49:0/3887761503 >> v2:172.21.15.107:3300/0 conn(0x7fea2006ae40 msgr2=0x7fea200d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:33.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea1ffff700 1 --2- 172.21.15.49:0/3887761503 >> v2:172.21.15.107:3300/0 conn(0x7fea2006ae40 0x7fea200d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea1ffff700 1 -- 172.21.15.49:0/3887761503 >> v1:172.21.15.49:6789/0 conn(0x7fea200b87d0 legacy=0x7fea20160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:33.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea1ffff700 1 -- 172.21.15.49:0/3887761503 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea200cd730 con 0x7fea200cfc60 2023-07-18T03:38:33.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.045+0000 7fea1f7fe700 1 -- 172.21.15.49:0/3887761503 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356386053 0 0) 0x7fea200cd4e0 con 0x7fea200b87d0 2023-07-18T03:38:33.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.046+0000 7fea1ffff700 1 --2- 172.21.15.49:0/3887761503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea200cfc60 0x7fea20160860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fea2015cbc0 tx=0x7fea0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e945463be3313d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:33.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.046+0000 7fea1f7fe700 1 -- 172.21.15.49:0/3887761503 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c00b870 con 0x7fea200cfc60 2023-07-18T03:38:33.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.046+0000 7fea1f7fe700 1 -- 172.21.15.49:0/3887761503 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c00b9d0 con 0x7fea200cfc60 2023-07-18T03:38:33.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.046+0000 7fea1f7fe700 1 -- 172.21.15.49:0/3887761503 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c00bd60 con 0x7fea200cfc60 2023-07-18T03:38:33.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.046+0000 7fea26fff700 1 -- 172.21.15.49:0/3887761503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea200cfc60 msgr2=0x7fea20160860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:33.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.046+0000 7fea26fff700 1 --2- 172.21.15.49:0/3887761503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea200cfc60 0x7fea20160860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fea2015cbc0 tx=0x7fea0c009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.047+0000 7fea26fff700 1 -- 172.21.15.49:0/3887761503 shutdown_connections 2023-07-18T03:38:33.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.047+0000 7fea26fff700 1 --2- 172.21.15.49:0/3887761503 >> v2:172.21.15.107:3300/0 conn(0x7fea2006ae40 0x7fea200d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.047+0000 7fea26fff700 1 --2- 172.21.15.49:0/3887761503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea200cfc60 0x7fea20160860 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.047+0000 7fea26fff700 1 -- 172.21.15.49:0/3887761503 >> 172.21.15.49:0/3887761503 conn(0x7fea2006c140 msgr2=0x7fea2006c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:33.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.047+0000 7fea26fff700 1 -- 172.21.15.49:0/3887761503 shutdown_connections 2023-07-18T03:38:33.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.047+0000 7fea26fff700 1 -- 172.21.15.49:0/3887761503 wait complete. 2023-07-18T03:38:33.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.048+0000 7fea26fff700 1 Processor -- start 2023-07-18T03:38:33.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.049+0000 7fea26fff700 1 -- start start 2023-07-18T03:38:33.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.049+0000 7fea26fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea200b87d0 0x7fea201f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.049+0000 7fea26fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea200cfc60 0x7fea200b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.049+0000 7fea26fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea200ca970 0x7fea201f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea26fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea201f2230 con 0x7fea200b87d0 2023-07-18T03:38:33.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea26fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea200cc130 con 0x7fea200ca970 2023-07-18T03:38:33.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea26fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea200c65a0 con 0x7fea200cfc60 2023-07-18T03:38:33.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea24d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea200b87d0 0x7fea201f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea24d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea200b87d0 0x7fea201f1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45858/0 (socket says 172.21.15.49:45858) 2023-07-18T03:38:33.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea24d9b700 1 -- 172.21.15.49:0/1723407998 learned_addr learned my addr 172.21.15.49:0/1723407998 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:33.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea1ffff700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea200cfc60 0x7fea200b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea24d9b700 1 -- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea200cfc60 msgr2=0x7fea200b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:33.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea2559c700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea200ca970 0x7fea201f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea24d9b700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea200cfc60 0x7fea200b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea24d9b700 1 -- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea200ca970 msgr2=0x7fea201f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:33.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea24d9b700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea200ca970 0x7fea201f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea24d9b700 1 -- 172.21.15.49:0/1723407998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c009550 con 0x7fea200b87d0 2023-07-18T03:38:33.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea1ffff700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea200cfc60 0x7fea200b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:33.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea2559c700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea200ca970 0x7fea201f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:33.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.050+0000 7fea24d9b700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea200b87d0 0x7fea201f1cf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea140011e0 tx=0x7fea1400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:33.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.051+0000 7fea1d7fa700 1 -- 172.21.15.49:0/1723407998 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14004c80 con 0x7fea200b87d0 2023-07-18T03:38:33.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.051+0000 7fea1d7fa700 1 -- 172.21.15.49:0/1723407998 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea14004e00 con 0x7fea200b87d0 2023-07-18T03:38:33.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.051+0000 7fea26fff700 1 -- 172.21.15.49:0/1723407998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea200c6880 con 0x7fea200b87d0 2023-07-18T03:38:33.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.051+0000 7fea1d7fa700 1 -- 172.21.15.49:0/1723407998 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14007310 con 0x7fea200b87d0 2023-07-18T03:38:33.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.051+0000 7fea26fff700 1 -- 172.21.15.49:0/1723407998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea200c6e40 con 0x7fea200b87d0 2023-07-18T03:38:33.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.052+0000 7fea26fff700 1 -- 172.21.15.49:0/1723407998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2004e280 con 0x7fea200b87d0 2023-07-18T03:38:33.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.053+0000 7fea1d7fa700 1 -- 172.21.15.49:0/1723407998 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fea14007470 con 0x7fea200b87d0 2023-07-18T03:38:33.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.054+0000 7fea1d7fa700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fea0406ace0 0x7fea0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.054+0000 7fea1d7fa700 1 -- 172.21.15.49:0/1723407998 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fea14088f70 con 0x7fea200b87d0 2023-07-18T03:38:33.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.055+0000 7fea1ffff700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fea0406ace0 0x7fea0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.055+0000 7fea1ffff700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fea0406ace0 0x7fea0406d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea200cbb90 tx=0x7fea0c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:33.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.058+0000 7fea1d7fa700 1 -- 172.21.15.49:0/1723407998 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fea140542d0 con 0x7fea200b87d0 2023-07-18T03:38:33.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.266+0000 7fea26fff700 1 -- 172.21.15.49:0/1723407998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fea2015d6a0 con 0x7fea200b87d0 2023-07-18T03:38:33.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.267+0000 7fea1d7fa700 1 -- 172.21.15.49:0/1723407998 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+126 (secure 0 0 0) 0x7fea140578f0 con 0x7fea200b87d0 2023-07-18T03:38:33.262 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:33.263 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2023-07-18T03:38:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:38:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2023-07-18T03:38:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": "rbd" 2023-07-18T03:38:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:38:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:38:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 2, 2023-07-18T03:38:33.264 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2023-07-18T03:38:33.264 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:38:33.264 INFO:teuthology.orchestra.run.smithi049.stdout:] 2023-07-18T03:38:33.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.269+0000 7fea26fff700 1 -- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fea0406ace0 msgr2=0x7fea0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:33.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.269+0000 7fea26fff700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fea0406ace0 0x7fea0406d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea200cbb90 tx=0x7fea0c008030 comp rx=0 tx=0).stop 2023-07-18T03:38:33.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.269+0000 7fea26fff700 1 -- 172.21.15.49:0/1723407998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea200b87d0 msgr2=0x7fea201f1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:33.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.269+0000 7fea26fff700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea200b87d0 0x7fea201f1cf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea140011e0 tx=0x7fea1400cae0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.270+0000 7fea26fff700 1 -- 172.21.15.49:0/1723407998 shutdown_connections 2023-07-18T03:38:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.270+0000 7fea26fff700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea200ca970 0x7fea201f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.270+0000 7fea26fff700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fea0406ace0 0x7fea0406d190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.270+0000 7fea26fff700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea200b87d0 0x7fea201f1cf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.270+0000 7fea26fff700 1 --2- 172.21.15.49:0/1723407998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea200cfc60 0x7fea200b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.270+0000 7fea26fff700 1 -- 172.21.15.49:0/1723407998 >> 172.21.15.49:0/1723407998 conn(0x7fea2006c140 msgr2=0x7fea200b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.270+0000 7fea26fff700 1 -- 172.21.15.49:0/1723407998 shutdown_connections 2023-07-18T03:38:33.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.270+0000 7fea26fff700 1 -- 172.21.15.49:0/1723407998 wait complete. 2023-07-18T03:38:33.282 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-18T03:38:33.283 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-18T03:38:33.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.496+0000 7f28d27ff700 1 Processor -- start 2023-07-18T03:38:33.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.496+0000 7f28d27ff700 1 -- start start 2023-07-18T03:38:33.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.497+0000 7f28d27ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f28cc15c7b0 0x7f28cc0d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.497+0000 7f28d27ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b1e50 0x7f28cc0d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.497+0000 7f28d27ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28cc0d0190 con 0x7f28cc0d0f70 2023-07-18T03:38:33.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.497+0000 7f28d27ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28cc0d3de0 con 0x7f28cc15c7b0 2023-07-18T03:38:33.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.497+0000 7f28d27ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0d13f0 con 0x7f28cc0b1e50 2023-07-18T03:38:33.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28d0d9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f28cc0d0f70 0x7f28cc160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45920/0 (socket says 172.21.15.49:45920) 2023-07-18T03:38:33.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28d0d9c700 1 -- 172.21.15.49:0/464865736 learned_addr learned my addr 172.21.15.49:0/464865736 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:33.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28cbfff700 1 --2- 172.21.15.49:0/464865736 >> v2:172.21.15.107:3300/0 conn(0x7f28cc15c7b0 0x7f28cc0d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28cb7fe700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b1e50 0x7f28cc0d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1255052749 0 0) 0x7f28cc0d0190 con 0x7f28cc0d0f70 2023-07-18T03:38:33.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b40035b0 con 0x7f28cc0d0f70 2023-07-18T03:38:33.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1357663270 0 0) 0x7f28b40035b0 con 0x7f28cc0d0f70 2023-07-18T03:38:33.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b1e50 msgr2=0x7f28cc0d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:33.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28caffd700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b1e50 0x7f28cc0d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 >> v2:172.21.15.107:3300/0 conn(0x7f28cc15c7b0 msgr2=0x7f28cc0d28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:33.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28caffd700 1 --2- 172.21.15.49:0/464865736 >> v2:172.21.15.107:3300/0 conn(0x7f28cc15c7b0 0x7f28cc0d28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc0d0470 con 0x7f28cc0d0f70 2023-07-18T03:38:33.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f28c40019f0 con 0x7f28cc0d0f70 2023-07-18T03:38:33.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.498+0000 7f28cbfff700 1 --2- 172.21.15.49:0/464865736 >> v2:172.21.15.107:3300/0 conn(0x7f28cc15c7b0 0x7f28cc0d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:33.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 >> v1:172.21.15.49:6789/0 conn(0x7f28cc0d0f70 legacy=0x7f28cc160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:33.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28caffd700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b1e50 0x7f28b4003ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28caffd700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b4004040 0x7f28b4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28caffd700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b4004d90 0x7f28b4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0d3de0 con 0x7f28b4004040 2023-07-18T03:38:33.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0d13f0 con 0x7f28b4004d90 2023-07-18T03:38:33.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b400ba60 con 0x7f28cc0b1e50 2023-07-18T03:38:33.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28cb7fe700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b4004040 0x7f28b4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28cbfff700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b1e50 0x7f28b4003ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28d0d9c700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b4004d90 0x7f28b4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28cb7fe700 1 -- 172.21.15.49:0/464865736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b1e50 msgr2=0x7f28b4003ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:33.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28cb7fe700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0b1e50 0x7f28b4003ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28cb7fe700 1 -- 172.21.15.49:0/464865736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b4004d90 msgr2=0x7f28b4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:33.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28cb7fe700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b4004d90 0x7f28b4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28cb7fe700 1 -- 172.21.15.49:0/464865736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b40035b0 con 0x7f28b4004040 2023-07-18T03:38:33.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.499+0000 7f28d0d9c700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b4004d90 0x7f28b4005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:33.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.500+0000 7f28cb7fe700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b4004040 0x7f28b4004850 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009b90 tx=0x7f28b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d8867d056995c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:33.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.500+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c000e6c0 con 0x7f28b4004040 2023-07-18T03:38:33.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.500+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c000e820 con 0x7f28b4004040 2023-07-18T03:38:33.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.500+0000 7f28caffd700 1 -- 172.21.15.49:0/464865736 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c000eb90 con 0x7f28b4004040 2023-07-18T03:38:33.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.500+0000 7f28d27ff700 1 -- 172.21.15.49:0/464865736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b4004040 msgr2=0x7f28b4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:33.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.500+0000 7f28d27ff700 1 --2- 172.21.15.49:0/464865736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28b4004040 0x7f28b4004850 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009b90 tx=0x7f28b4005880 comp rx=0 tx=0).stop 2023-07-18T03:38:33.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.500+0000 7f28d0d9c700 1 -- 172.21.15.49:0/464865736 reap_dead start 2023-07-18T03:38:33.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.500+0000 7f28d27ff700 1 -- 172.21.15.49:0/464865736 shutdown_connections 2023-07-18T03:38:33.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.500+0000 7f28d27ff700 1 -- 172.21.15.49:0/464865736 >> 172.21.15.49:0/464865736 conn(0x7f28cc09c1b0 msgr2=0x7f28cc0d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:33.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.500+0000 7f28d27ff700 1 -- 172.21.15.49:0/464865736 shutdown_connections 2023-07-18T03:38:33.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.501+0000 7f28d27ff700 1 -- 172.21.15.49:0/464865736 wait complete. 2023-07-18T03:38:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.502+0000 7f28d27ff700 1 Processor -- start 2023-07-18T03:38:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.502+0000 7f28d27ff700 1 -- start start 2023-07-18T03:38:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.503+0000 7f28d27ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b4004040 0x7f28cc15b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.503+0000 7f28d27ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0c93e0 0x7f28cc14e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.503+0000 7f28d27ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0c79b0 0x7f28cc0caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.503+0000 7f28d27ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0c7210 con 0x7f28cc0c93e0 2023-07-18T03:38:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.503+0000 7f28d27ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc15f4a0 con 0x7f28b4004040 2023-07-18T03:38:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.503+0000 7f28d27ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc15f780 con 0x7f28cc0c79b0 2023-07-18T03:38:33.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.504+0000 7f28cb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0c93e0 0x7f28cc14e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.504+0000 7f28cb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0c93e0 0x7f28cc14e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45874/0 (socket says 172.21.15.49:45874) 2023-07-18T03:38:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.504+0000 7f28cb7fe700 1 -- 172.21.15.49:0/3454490315 learned_addr learned my addr 172.21.15.49:0/3454490315 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.504+0000 7f28cbfff700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b4004040 0x7f28cc15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.504+0000 7f28d0d9c700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0c79b0 0x7f28cc0caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.504+0000 7f28cb7fe700 1 -- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0c79b0 msgr2=0x7f28cc0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.504+0000 7f28cb7fe700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0c79b0 0x7f28cc0caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.504+0000 7f28cb7fe700 1 -- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b4004040 msgr2=0x7f28cc15b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.504+0000 7f28cb7fe700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b4004040 0x7f28cc15b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.504+0000 7f28cb7fe700 1 -- 172.21.15.49:0/3454490315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc0c7490 con 0x7f28cc0c93e0 2023-07-18T03:38:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.504+0000 7f28cb7fe700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0c93e0 0x7f28cc14e240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f28b4004790 tx=0x7f28c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.505+0000 7f28c8ff9700 1 -- 172.21.15.49:0/3454490315 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0021070 con 0x7f28cc0c93e0 2023-07-18T03:38:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.505+0000 7f28c8ff9700 1 -- 172.21.15.49:0/3454490315 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0004370 con 0x7f28cc0c93e0 2023-07-18T03:38:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.505+0000 7f28c8ff9700 1 -- 172.21.15.49:0/3454490315 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c000e6c0 con 0x7f28cc0c93e0 2023-07-18T03:38:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.505+0000 7f28d27ff700 1 -- 172.21.15.49:0/3454490315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28cc0c9850 con 0x7f28cc0c93e0 2023-07-18T03:38:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.505+0000 7f28d27ff700 1 -- 172.21.15.49:0/3454490315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc0c7e50 con 0x7f28cc0c93e0 2023-07-18T03:38:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.506+0000 7f28d27ff700 1 -- 172.21.15.49:0/3454490315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28cc0c7620 con 0x7f28cc0c93e0 2023-07-18T03:38:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.507+0000 7f28c8ff9700 1 -- 172.21.15.49:0/3454490315 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f28c0025070 con 0x7f28cc0c93e0 2023-07-18T03:38:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.508+0000 7f28c8ff9700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f28b806ace0 0x7f28b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.508+0000 7f28c8ff9700 1 -- 172.21.15.49:0/3454490315 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f28c008a1a0 con 0x7f28cc0c93e0 2023-07-18T03:38:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.508+0000 7f28cbfff700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f28b806ace0 0x7f28b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:33.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.509+0000 7f28cbfff700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f28b806ace0 0x7f28b806d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0c9370 tx=0x7f28bc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:33.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.512+0000 7f28c8ff9700 1 -- 172.21.15.49:0/3454490315 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f28c0055590 con 0x7f28cc0c93e0 2023-07-18T03:38:33.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:33.721+0000 7f28d27ff700 1 -- 172.21.15.49:0/3454490315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f28cc15f8f0 con 0x7f28cc0c93e0 2023-07-18T03:38:34.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.500+0000 7f28c8ff9700 1 -- 172.21.15.49:0/3454490315 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f28c0058bb0 con 0x7f28cc0c93e0 2023-07-18T03:38:34.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.505+0000 7f28d27ff700 1 -- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f28b806ace0 msgr2=0x7f28b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:34.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.505+0000 7f28d27ff700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f28b806ace0 0x7f28b806d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0c9370 tx=0x7f28bc009210 comp rx=0 tx=0).stop 2023-07-18T03:38:34.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.505+0000 7f28d27ff700 1 -- 172.21.15.49:0/3454490315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0c93e0 msgr2=0x7f28cc14e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:34.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.505+0000 7f28d27ff700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0c93e0 0x7f28cc14e240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f28b4004790 tx=0x7f28c00047c0 comp rx=0 tx=0).stop 2023-07-18T03:38:34.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.505+0000 7f28d27ff700 1 -- 172.21.15.49:0/3454490315 shutdown_connections 2023-07-18T03:38:34.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.505+0000 7f28d27ff700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28cc0c79b0 0x7f28cc0caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:34.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.505+0000 7f28d27ff700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b4004040 0x7f28cc15b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:34.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.505+0000 7f28d27ff700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f28b806ace0 0x7f28b806d190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:34.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.505+0000 7f28d27ff700 1 --2- 172.21.15.49:0/3454490315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28cc0c93e0 0x7f28cc14e240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:34.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.505+0000 7f28d27ff700 1 -- 172.21.15.49:0/3454490315 >> 172.21.15.49:0/3454490315 conn(0x7f28cc09c1b0 msgr2=0x7f28cc0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:34.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.506+0000 7f28d27ff700 1 -- 172.21.15.49:0/3454490315 shutdown_connections 2023-07-18T03:38:34.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.506+0000 7f28d27ff700 1 -- 172.21.15.49:0/3454490315 wait complete. 2023-07-18T03:38:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2023-07-18T03:38:34.526 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-18T03:38:34.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.746+0000 7f7716dff700 1 Processor -- start 2023-07-18T03:38:34.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.746+0000 7f7716dff700 1 -- start start 2023-07-18T03:38:34.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.747+0000 7f7716dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f77100d0b10 0x7f77100d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:34.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.747+0000 7f7716dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77100b8e30 con 0x7f77100b0e20 2023-07-18T03:38:34.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.747+0000 7f7716dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77100afaa0 con 0x7f77100b8a60 2023-07-18T03:38:34.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.747+0000 7f7716dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77100d14d0 con 0x7f77100d0b10 2023-07-18T03:38:34.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.747+0000 7f771539c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f77100b0e20 0x7f77100b1260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45926/0 (socket says 172.21.15.49:45926) 2023-07-18T03:38:34.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.747+0000 7f771539c700 1 -- 172.21.15.49:0/1534287661 learned_addr learned my addr 172.21.15.49:0/1534287661 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:34.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.748+0000 7f770ffff700 1 --2- 172.21.15.49:0/1534287661 >> v2:172.21.15.107:3300/0 conn(0x7f77100d0b10 0x7f77100d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:34.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.748+0000 7f770f7fe700 1 -- 172.21.15.49:0/1534287661 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803457336 0 0) 0x7f77100b8e30 con 0x7f77100b0e20 2023-07-18T03:38:34.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.748+0000 7f770f7fe700 1 -- 172.21.15.49:0/1534287661 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77000035b0 con 0x7f77100b0e20 2023-07-18T03:38:34.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.748+0000 7f770f7fe700 1 -- 172.21.15.49:0/1534287661 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644990130 0 0) 0x7f77100afaa0 con 0x7f77100b8a60 2023-07-18T03:38:34.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.748+0000 7f770f7fe700 1 -- 172.21.15.49:0/1534287661 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77100b8e30 con 0x7f77100b8a60 2023-07-18T03:38:34.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.748+0000 7f770ffff700 1 -- 172.21.15.49:0/1534287661 >> v1:172.21.15.107:6789/0 conn(0x7f77100b8a60 legacy=0x7f77100b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:34.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.748+0000 7f770ffff700 1 -- 172.21.15.49:0/1534287661 >> v1:172.21.15.49:6789/0 conn(0x7f77100b0e20 legacy=0x7f77100b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:34.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.748+0000 7f770ffff700 1 -- 172.21.15.49:0/1534287661 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77100afd80 con 0x7f77100d0b10 2023-07-18T03:38:34.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.748+0000 7f770ffff700 1 --2- 172.21.15.49:0/1534287661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100d0b10 0x7f77100d0f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009840 tx=0x7f76fc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be2e0809076dff84 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:34.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.749+0000 7f770f7fe700 1 -- 172.21.15.49:0/1534287661 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc011830 con 0x7f77100d0b10 2023-07-18T03:38:34.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.749+0000 7f770f7fe700 1 -- 172.21.15.49:0/1534287661 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76fc011990 con 0x7f77100d0b10 2023-07-18T03:38:34.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.749+0000 7f770f7fe700 1 -- 172.21.15.49:0/1534287661 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc011d20 con 0x7f77100d0b10 2023-07-18T03:38:34.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.749+0000 7f7716dff700 1 -- 172.21.15.49:0/1534287661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100d0b10 msgr2=0x7f77100d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:34.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.749+0000 7f7716dff700 1 --2- 172.21.15.49:0/1534287661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100d0b10 0x7f77100d0f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009840 tx=0x7f76fc009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:34.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.750+0000 7f7716dff700 1 -- 172.21.15.49:0/1534287661 shutdown_connections 2023-07-18T03:38:34.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.750+0000 7f7716dff700 1 --2- 172.21.15.49:0/1534287661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100d0b10 0x7f77100d0f90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:34.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.750+0000 7f7716dff700 1 -- 172.21.15.49:0/1534287661 >> 172.21.15.49:0/1534287661 conn(0x7f771009c1b0 msgr2=0x7f77100d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:34.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.750+0000 7f7716dff700 1 -- 172.21.15.49:0/1534287661 shutdown_connections 2023-07-18T03:38:34.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.750+0000 7f7716dff700 1 -- 172.21.15.49:0/1534287661 wait complete. 2023-07-18T03:38:34.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.751+0000 7f7716dff700 1 Processor -- start 2023-07-18T03:38:34.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.752+0000 7f7716dff700 1 -- start start 2023-07-18T03:38:34.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f7716dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100b0e20 0x7f771006b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:34.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f7716dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100b8a60 0x7f77100ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:34.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f7716dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100d0b10 0x7f771014f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:34.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f7716dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77100cdc00 con 0x7f77100b8a60 2023-07-18T03:38:34.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f7716dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77100cdea0 con 0x7f77100b0e20 2023-07-18T03:38:34.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f7716dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77100ce180 con 0x7f77100d0b10 2023-07-18T03:38:34.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f770ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100b8a60 0x7f77100ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:34.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f770ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100b8a60 0x7f77100ce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45878/0 (socket says 172.21.15.49:45878) 2023-07-18T03:38:34.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f770ffff700 1 -- 172.21.15.49:0/301789445 learned_addr learned my addr 172.21.15.49:0/301789445 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:34.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f7714b9b700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100b0e20 0x7f771006b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:34.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f770ffff700 1 -- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100d0b10 msgr2=0x7f771014f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:34.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f770ffff700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100d0b10 0x7f771014f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:34.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f770ffff700 1 -- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100b0e20 msgr2=0x7f771006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:34.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f770ffff700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100b0e20 0x7f771006b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:34.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f770ffff700 1 -- 172.21.15.49:0/301789445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc0094a0 con 0x7f77100b8a60 2023-07-18T03:38:34.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.753+0000 7f7714b9b700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100b0e20 0x7f771006b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:34.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.754+0000 7f770ffff700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100b8a60 0x7f77100ce980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76fc014040 tx=0x7f76fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:34.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.754+0000 7f770d7fa700 1 -- 172.21.15.49:0/301789445 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc01a860 con 0x7f77100b8a60 2023-07-18T03:38:34.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.754+0000 7f770d7fa700 1 -- 172.21.15.49:0/301789445 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76fc01a9c0 con 0x7f77100b8a60 2023-07-18T03:38:34.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.754+0000 7f770d7fa700 1 -- 172.21.15.49:0/301789445 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc01ad30 con 0x7f77100b8a60 2023-07-18T03:38:34.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.754+0000 7f7716dff700 1 -- 172.21.15.49:0/301789445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77100ce460 con 0x7f77100b8a60 2023-07-18T03:38:34.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.754+0000 7f7716dff700 1 -- 172.21.15.49:0/301789445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77100cd570 con 0x7f77100b8a60 2023-07-18T03:38:34.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.755+0000 7f7716dff700 1 -- 172.21.15.49:0/301789445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7710074350 con 0x7f77100b8a60 2023-07-18T03:38:34.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.756+0000 7f770d7fa700 1 -- 172.21.15.49:0/301789445 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f76fc022070 con 0x7f77100b8a60 2023-07-18T03:38:34.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.758+0000 7f770d7fa700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f76f406ace0 0x7f76f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:34.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.758+0000 7f770d7fa700 1 -- 172.21.15.49:0/301789445 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f76fc089fe0 con 0x7f77100b8a60 2023-07-18T03:38:34.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.759+0000 7f7714b9b700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f76f406ace0 0x7f76f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:34.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.759+0000 7f7714b9b700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f76f406ace0 0x7f76f406d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7704007fd0 tx=0x7f7704009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:34.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.763+0000 7f770d7fa700 1 -- 172.21.15.49:0/301789445 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f76fc0552c0 con 0x7f77100b8a60 2023-07-18T03:38:34.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:34.971+0000 7f7716dff700 1 -- 172.21.15.49:0/301789445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f77100ce620 con 0x7f77100b8a60 2023-07-18T03:38:35.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.509+0000 7f770d7fa700 1 -- 172.21.15.49:0/301789445 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f76fc029020 con 0x7f77100b8a60 2023-07-18T03:38:35.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.514+0000 7f7716dff700 1 -- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f76f406ace0 msgr2=0x7f76f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:35.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.514+0000 7f7716dff700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f76f406ace0 0x7f76f406d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7704007fd0 tx=0x7f7704009040 comp rx=0 tx=0).stop 2023-07-18T03:38:35.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.514+0000 7f7716dff700 1 -- 172.21.15.49:0/301789445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100b8a60 msgr2=0x7f77100ce980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:35.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.514+0000 7f7716dff700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100b8a60 0x7f77100ce980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76fc014040 tx=0x7f76fc004750 comp rx=0 tx=0).stop 2023-07-18T03:38:35.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.515+0000 7f7716dff700 1 -- 172.21.15.49:0/301789445 shutdown_connections 2023-07-18T03:38:35.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.515+0000 7f7716dff700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100d0b10 0x7f771014f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:35.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.515+0000 7f7716dff700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100b0e20 0x7f771006b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:35.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.515+0000 7f7716dff700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f76f406ace0 0x7f76f406d190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:35.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.515+0000 7f7716dff700 1 --2- 172.21.15.49:0/301789445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100b8a60 0x7f77100ce980 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:35.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.515+0000 7f7716dff700 1 -- 172.21.15.49:0/301789445 >> 172.21.15.49:0/301789445 conn(0x7f771009c1b0 msgr2=0x7f77100d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:35.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.518+0000 7f7716dff700 1 -- 172.21.15.49:0/301789445 shutdown_connections 2023-07-18T03:38:35.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.518+0000 7f7716dff700 1 -- 172.21.15.49:0/301789445 wait complete. 2023-07-18T03:38:35.526 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2023-07-18T03:38:35.543 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-18T03:38:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.755+0000 7fed825ff700 1 Processor -- start 2023-07-18T03:38:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.755+0000 7fed825ff700 1 -- start start 2023-07-18T03:38:35.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.756+0000 7fed825ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fed7c0d0b10 0x7fed7c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:35.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.756+0000 7fed825ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fed7c0b0e20 0x7fed7c0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:35.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.756+0000 7fed825ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c0b8e30 con 0x7fed7c0b8a60 2023-07-18T03:38:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.756+0000 7fed825ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c0d14d0 con 0x7fed7c0d0b10 2023-07-18T03:38:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.756+0000 7fed825ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c0afaa0 con 0x7fed7c0b0e20 2023-07-18T03:38:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.757+0000 7fed7b7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fed7c0d0b10 0x7fed7c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.757+0000 7fed7b7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fed7c0d0b10 0x7fed7c0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45894/0 (socket says 172.21.15.49:45894) 2023-07-18T03:38:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.757+0000 7fed7b7fe700 1 -- 172.21.15.49:0/2528547883 learned_addr learned my addr 172.21.15.49:0/2528547883 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.757+0000 7fed80b9c700 1 --2- 172.21.15.49:0/2528547883 >> v2:172.21.15.107:3300/0 conn(0x7fed7c0b0e20 0x7fed7c0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.757+0000 7fed7b7fe700 1 -- 172.21.15.49:0/2528547883 >> v2:172.21.15.107:3300/0 conn(0x7fed7c0b0e20 msgr2=0x7fed7c0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.757+0000 7fed7b7fe700 1 --2- 172.21.15.49:0/2528547883 >> v2:172.21.15.107:3300/0 conn(0x7fed7c0b0e20 0x7fed7c0b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.757+0000 7fed7b7fe700 1 -- 172.21.15.49:0/2528547883 >> v1:172.21.15.107:6789/0 conn(0x7fed7c0b8a60 legacy=0x7fed7c0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.757+0000 7fed7b7fe700 1 -- 172.21.15.49:0/2528547883 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c0afd80 con 0x7fed7c0d0b10 2023-07-18T03:38:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.757+0000 7fed80b9c700 1 --2- 172.21.15.49:0/2528547883 >> v2:172.21.15.107:3300/0 conn(0x7fed7c0b0e20 0x7fed7c0b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.757+0000 7fed7b7fe700 1 --2- 172.21.15.49:0/2528547883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed7c0d0b10 0x7fed7c0d0f90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0674e0 tx=0x7fed70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d86f5ddf900dc3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:35.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.758+0000 7fed7affd700 1 -- 172.21.15.49:0/2528547883 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7000b870 con 0x7fed7c0d0b10 2023-07-18T03:38:35.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.758+0000 7fed7affd700 1 -- 172.21.15.49:0/2528547883 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7000b9d0 con 0x7fed7c0d0b10 2023-07-18T03:38:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.758+0000 7fed7affd700 1 -- 172.21.15.49:0/2528547883 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7000bd60 con 0x7fed7c0d0b10 2023-07-18T03:38:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.758+0000 7fed825ff700 1 -- 172.21.15.49:0/2528547883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed7c0d0b10 msgr2=0x7fed7c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.758+0000 7fed825ff700 1 --2- 172.21.15.49:0/2528547883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed7c0d0b10 0x7fed7c0d0f90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0674e0 tx=0x7fed70009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.758+0000 7fed825ff700 1 -- 172.21.15.49:0/2528547883 shutdown_connections 2023-07-18T03:38:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.758+0000 7fed825ff700 1 --2- 172.21.15.49:0/2528547883 >> v2:172.21.15.107:3300/0 conn(0x7fed7c0b0e20 0x7fed7c0b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.758+0000 7fed825ff700 1 --2- 172.21.15.49:0/2528547883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed7c0d0b10 0x7fed7c0d0f90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:35.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.758+0000 7fed825ff700 1 -- 172.21.15.49:0/2528547883 >> 172.21.15.49:0/2528547883 conn(0x7fed7c09c1b0 msgr2=0x7fed7c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:35.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.758+0000 7fed825ff700 1 -- 172.21.15.49:0/2528547883 shutdown_connections 2023-07-18T03:38:35.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.758+0000 7fed825ff700 1 -- 172.21.15.49:0/2528547883 wait complete. 2023-07-18T03:38:35.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.760+0000 7fed825ff700 1 Processor -- start 2023-07-18T03:38:35.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.760+0000 7fed825ff700 1 -- start start 2023-07-18T03:38:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.761+0000 7fed825ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed7c0b0e20 0x7fed7c15f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.761+0000 7fed825ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed7c0b8a60 0x7fed7c0cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.761+0000 7fed825ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed7c0d0b10 0x7fed7c152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.761+0000 7fed825ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c15fcb0 con 0x7fed7c0b8a60 2023-07-18T03:38:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.761+0000 7fed825ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0ce720 con 0x7fed7c0d0b10 2023-07-18T03:38:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.761+0000 7fed825ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0cea00 con 0x7fed7c0b0e20 2023-07-18T03:38:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.761+0000 7fed7b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed7c0b8a60 0x7fed7c0cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed7b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed7c0b8a60 0x7fed7c0cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45898/0 (socket says 172.21.15.49:45898) 2023-07-18T03:38:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed7b7fe700 1 -- 172.21.15.49:0/2761990932 learned_addr learned my addr 172.21.15.49:0/2761990932 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed7bfff700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed7c0b0e20 0x7fed7c15f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed80b9c700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed7c0d0b10 0x7fed7c152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed7b7fe700 1 -- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed7c0b0e20 msgr2=0x7fed7c15f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed7b7fe700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed7c0b0e20 0x7fed7c15f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed7b7fe700 1 -- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed7c0d0b10 msgr2=0x7fed7c152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed7b7fe700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed7c0d0b10 0x7fed7c152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed7b7fe700 1 -- 172.21.15.49:0/2761990932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed70009550 con 0x7fed7c0b8a60 2023-07-18T03:38:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed7b7fe700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed7c0b8a60 0x7fed7c0cfaa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fed70000c00 tx=0x7fed70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed78ff9700 1 -- 172.21.15.49:0/2761990932 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70004310 con 0x7fed7c0b8a60 2023-07-18T03:38:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed78ff9700 1 -- 172.21.15.49:0/2761990932 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7001e070 con 0x7fed7c0b8a60 2023-07-18T03:38:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.762+0000 7fed78ff9700 1 -- 172.21.15.49:0/2761990932 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed700036a0 con 0x7fed7c0b8a60 2023-07-18T03:38:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.763+0000 7fed825ff700 1 -- 172.21.15.49:0/2761990932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c0cece0 con 0x7fed7c0b8a60 2023-07-18T03:38:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.763+0000 7fed825ff700 1 -- 172.21.15.49:0/2761990932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c0cffe0 con 0x7fed7c0b8a60 2023-07-18T03:38:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.764+0000 7fed825ff700 1 -- 172.21.15.49:0/2761990932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c0c78b0 con 0x7fed7c0b8a60 2023-07-18T03:38:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.765+0000 7fed78ff9700 1 -- 172.21.15.49:0/2761990932 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed70020030 con 0x7fed7c0b8a60 2023-07-18T03:38:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.766+0000 7fed78ff9700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fed6806ace0 0x7fed6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.766+0000 7fed78ff9700 1 -- 172.21.15.49:0/2761990932 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fed7008a3a0 con 0x7fed7c0b8a60 2023-07-18T03:38:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.766+0000 7fed7bfff700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fed6806ace0 0x7fed6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:35.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.767+0000 7fed7bfff700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fed6806ace0 0x7fed6806d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed6c007fd0 tx=0x7fed6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:35.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:35.770+0000 7fed78ff9700 1 -- 172.21.15.49:0/2761990932 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fed7c0c78b0 con 0x7fed7c0b8a60 2023-07-18T03:38:36.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.023+0000 7fed825ff700 1 -- 172.21.15.49:0/2761990932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fed7c0c78b0 con 0x7fed7c0b8a60 2023-07-18T03:38:36.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.532+0000 7fed78ff9700 1 -- 172.21.15.49:0/2761990932 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fed7c0c78b0 con 0x7fed7c0b8a60 2023-07-18T03:38:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.540+0000 7fed825ff700 1 -- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fed6806ace0 msgr2=0x7fed6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.540+0000 7fed825ff700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fed6806ace0 0x7fed6806d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed6c007fd0 tx=0x7fed6c009040 comp rx=0 tx=0).stop 2023-07-18T03:38:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.540+0000 7fed825ff700 1 -- 172.21.15.49:0/2761990932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed7c0b8a60 msgr2=0x7fed7c0cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.540+0000 7fed825ff700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed7c0b8a60 0x7fed7c0cfaa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fed70000c00 tx=0x7fed70004750 comp rx=0 tx=0).stop 2023-07-18T03:38:36.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.540+0000 7fed825ff700 1 -- 172.21.15.49:0/2761990932 shutdown_connections 2023-07-18T03:38:36.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.540+0000 7fed825ff700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed7c0d0b10 0x7fed7c152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.540+0000 7fed825ff700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed7c0b0e20 0x7fed7c15f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.540+0000 7fed825ff700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fed6806ace0 0x7fed6806d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.540+0000 7fed825ff700 1 --2- 172.21.15.49:0/2761990932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed7c0b8a60 0x7fed7c0cfaa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.540+0000 7fed825ff700 1 -- 172.21.15.49:0/2761990932 >> 172.21.15.49:0/2761990932 conn(0x7fed7c09c1b0 msgr2=0x7fed7c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:36.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.541+0000 7fed825ff700 1 -- 172.21.15.49:0/2761990932 shutdown_connections 2023-07-18T03:38:36.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.541+0000 7fed825ff700 1 -- 172.21.15.49:0/2761990932 wait complete. 2023-07-18T03:38:36.538 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 3 and data pool 4 2023-07-18T03:38:36.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:36.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.768+0000 7fa1d95ff700 1 Processor -- start 2023-07-18T03:38:36.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.769+0000 7fa1d95ff700 1 -- start start 2023-07-18T03:38:36.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.770+0000 7fa1d95ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1d41597c0 0x7fa1d4159c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:36.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.770+0000 7fa1d95ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa1d40cea50 0x7fa1d40ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:36.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.770+0000 7fa1d95ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d40cdf90 con 0x7fa1d41500d0 2023-07-18T03:38:36.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.770+0000 7fa1d95ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d415a2f0 con 0x7fa1d40cea50 2023-07-18T03:38:36.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.770+0000 7fa1d95ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d415a5d0 con 0x7fa1d41597c0 2023-07-18T03:38:36.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.770+0000 7fa1d2ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa1d41500d0 0x7fa1d4070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45940/0 (socket says 172.21.15.49:45940) 2023-07-18T03:38:36.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.770+0000 7fa1d2ffd700 1 -- 172.21.15.49:0/1332234417 learned_addr learned my addr 172.21.15.49:0/1332234417 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:36.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.770+0000 7fa1d37fe700 1 --2- 172.21.15.49:0/1332234417 >> v2:172.21.15.49:3300/0 conn(0x7fa1d40cea50 0x7fa1d40ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:36.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.771+0000 7fa1cbfff700 1 --2- 172.21.15.49:0/1332234417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1d41597c0 0x7fa1d4159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:36.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.771+0000 7fa1d27fc700 1 -- 172.21.15.49:0/1332234417 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761489577 0 0) 0x7fa1d40cdf90 con 0x7fa1d41500d0 2023-07-18T03:38:36.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.771+0000 7fa1d37fe700 1 -- 172.21.15.49:0/1332234417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1d41597c0 msgr2=0x7fa1d4159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:36.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.771+0000 7fa1d37fe700 1 --2- 172.21.15.49:0/1332234417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1d41597c0 0x7fa1d4159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.771+0000 7fa1d37fe700 1 -- 172.21.15.49:0/1332234417 >> v1:172.21.15.49:6789/0 conn(0x7fa1d41500d0 legacy=0x7fa1d4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:36.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.771+0000 7fa1d37fe700 1 -- 172.21.15.49:0/1332234417 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d40ce250 con 0x7fa1d40cea50 2023-07-18T03:38:36.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.771+0000 7fa1d37fe700 1 --2- 172.21.15.49:0/1332234417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d40cea50 0x7fa1d40ceeb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009df0 tx=0x7fa1cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55da502f8828306 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:36.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.771+0000 7fa1d27fc700 1 -- 172.21.15.49:0/1332234417 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00bd50 con 0x7fa1d40cea50 2023-07-18T03:38:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.771+0000 7fa1d27fc700 1 -- 172.21.15.49:0/1332234417 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc003a20 con 0x7fa1d40cea50 2023-07-18T03:38:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.771+0000 7fa1d27fc700 1 -- 172.21.15.49:0/1332234417 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc003dd0 con 0x7fa1d40cea50 2023-07-18T03:38:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.772+0000 7fa1d95ff700 1 -- 172.21.15.49:0/1332234417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d40cea50 msgr2=0x7fa1d40ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.772+0000 7fa1d95ff700 1 --2- 172.21.15.49:0/1332234417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d40cea50 0x7fa1d40ceeb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009df0 tx=0x7fa1cc00b330 comp rx=0 tx=0).stop 2023-07-18T03:38:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.772+0000 7fa1d95ff700 1 -- 172.21.15.49:0/1332234417 shutdown_connections 2023-07-18T03:38:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.772+0000 7fa1d95ff700 1 --2- 172.21.15.49:0/1332234417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d40cea50 0x7fa1d40ceeb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.772+0000 7fa1d95ff700 1 --2- 172.21.15.49:0/1332234417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1d41597c0 0x7fa1d4159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.772+0000 7fa1d95ff700 1 -- 172.21.15.49:0/1332234417 >> 172.21.15.49:0/1332234417 conn(0x7fa1d4069f30 msgr2=0x7fa1d406a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.772+0000 7fa1d95ff700 1 -- 172.21.15.49:0/1332234417 shutdown_connections 2023-07-18T03:38:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.772+0000 7fa1d95ff700 1 -- 172.21.15.49:0/1332234417 wait complete. 2023-07-18T03:38:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.774+0000 7fa1d95ff700 1 Processor -- start 2023-07-18T03:38:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.774+0000 7fa1d95ff700 1 -- start start 2023-07-18T03:38:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.775+0000 7fa1d95ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1d40cea50 0x7fa1d4139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:36.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.775+0000 7fa1d95ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1d41500d0 0x7fa1d41f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:36.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.775+0000 7fa1d95ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d41597c0 0x7fa1d41f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:36.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.775+0000 7fa1d95ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d41f1e90 con 0x7fa1d41597c0 2023-07-18T03:38:36.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.775+0000 7fa1d95ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d41f6490 con 0x7fa1d40cea50 2023-07-18T03:38:36.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.775+0000 7fa1d95ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d413a570 con 0x7fa1d41500d0 2023-07-18T03:38:36.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.775+0000 7fa1d37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d41597c0 0x7fa1d41f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:36.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.775+0000 7fa1d37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d41597c0 0x7fa1d41f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45910/0 (socket says 172.21.15.49:45910) 2023-07-18T03:38:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.775+0000 7fa1d37fe700 1 -- 172.21.15.49:0/2878796384 learned_addr learned my addr 172.21.15.49:0/2878796384 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.775+0000 7fa1d2ffd700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1d40cea50 0x7fa1d4139610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.775+0000 7fa1cbfff700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1d41500d0 0x7fa1d41f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.776+0000 7fa1d37fe700 1 -- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1d41500d0 msgr2=0x7fa1d41f1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.776+0000 7fa1d37fe700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1d41500d0 0x7fa1d41f1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.776+0000 7fa1d37fe700 1 -- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1d40cea50 msgr2=0x7fa1d4139610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:36.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.776+0000 7fa1d37fe700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1d40cea50 0x7fa1d4139610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.776+0000 7fa1d37fe700 1 -- 172.21.15.49:0/2878796384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc009a70 con 0x7fa1d41597c0 2023-07-18T03:38:36.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.776+0000 7fa1d37fe700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d41597c0 0x7fa1d41f5f50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00b730 tx=0x7fa1cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:36.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.776+0000 7fa1cb7fe700 1 -- 172.21.15.49:0/2878796384 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc01e070 con 0x7fa1d41597c0 2023-07-18T03:38:36.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.776+0000 7fa1cb7fe700 1 -- 172.21.15.49:0/2878796384 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc004310 con 0x7fa1d41597c0 2023-07-18T03:38:36.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.776+0000 7fa1cb7fe700 1 -- 172.21.15.49:0/2878796384 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc0046c0 con 0x7fa1d41597c0 2023-07-18T03:38:36.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.776+0000 7fa1d95ff700 1 -- 172.21.15.49:0/2878796384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d4139cd0 con 0x7fa1d41597c0 2023-07-18T03:38:36.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.776+0000 7fa1d95ff700 1 -- 172.21.15.49:0/2878796384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d413a290 con 0x7fa1d41597c0 2023-07-18T03:38:36.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.777+0000 7fa1d95ff700 1 -- 172.21.15.49:0/2878796384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d404e280 con 0x7fa1d41597c0 2023-07-18T03:38:36.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.778+0000 7fa1cb7fe700 1 -- 172.21.15.49:0/2878796384 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa1cc004820 con 0x7fa1d41597c0 2023-07-18T03:38:36.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.779+0000 7fa1cb7fe700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa1b406ace0 0x7fa1b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:36.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.779+0000 7fa1cb7fe700 1 -- 172.21.15.49:0/2878796384 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa1cc08ab30 con 0x7fa1d41597c0 2023-07-18T03:38:36.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.780+0000 7fa1d2ffd700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa1b406ace0 0x7fa1b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:36.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.780+0000 7fa1d2ffd700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa1b406ace0 0x7fa1b406d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4007fd0 tx=0x7fa1c4007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:36.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.784+0000 7fa1cb7fe700 1 -- 172.21.15.49:0/2878796384 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa1cc055c70 con 0x7fa1d41597c0 2023-07-18T03:38:36.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.994+0000 7fa1d95ff700 1 -- 172.21.15.49:0/2878796384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1d4070130 con 0x7fa1d41597c0 2023-07-18T03:38:36.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.995+0000 7fa1cb7fe700 1 -- 172.21.15.49:0/2878796384 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18198 (secure 0 0 0) 0x7fa1cc027090 con 0x7fa1d41597c0 2023-07-18T03:38:36.990 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:36.991 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":21,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:36.024875+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:36.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.997+0000 7fa1d95ff700 1 -- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa1b406ace0 msgr2=0x7fa1b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:36.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.997+0000 7fa1d95ff700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa1b406ace0 0x7fa1b406d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4007fd0 tx=0x7fa1c4007de0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.997+0000 7fa1d95ff700 1 -- 172.21.15.49:0/2878796384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d41597c0 msgr2=0x7fa1d41f5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:36.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.997+0000 7fa1d95ff700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d41597c0 0x7fa1d41f5f50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00b730 tx=0x7fa1cc004ad0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.998+0000 7fa1d95ff700 1 -- 172.21.15.49:0/2878796384 shutdown_connections 2023-07-18T03:38:36.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.998+0000 7fa1d95ff700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa1b406ace0 0x7fa1b406d190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.998+0000 7fa1d95ff700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d41597c0 0x7fa1d41f5f50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.998+0000 7fa1d95ff700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1d40cea50 0x7fa1d4139610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.998+0000 7fa1d95ff700 1 --2- 172.21.15.49:0/2878796384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1d41500d0 0x7fa1d41f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:36.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.998+0000 7fa1d95ff700 1 -- 172.21.15.49:0/2878796384 >> 172.21.15.49:0/2878796384 conn(0x7fa1d4069f30 msgr2=0x7fa1d4070350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:36.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.998+0000 7fa1d95ff700 1 -- 172.21.15.49:0/2878796384 shutdown_connections 2023-07-18T03:38:36.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:36.998+0000 7fa1d95ff700 1 -- 172.21.15.49:0/2878796384 wait complete. 2023-07-18T03:38:37.012 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:37.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.221+0000 7f073afff700 1 Processor -- start 2023-07-18T03:38:37.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.221+0000 7f073afff700 1 -- start start 2023-07-18T03:38:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f073afff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f07340d1220 0x7f07340b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f073afff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f07340b1e00 0x7f07340afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f073afff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07340d2ce0 con 0x7f073415c950 2023-07-18T03:38:37.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f073afff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07340b3350 con 0x7f07340b1e00 2023-07-18T03:38:37.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f073afff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07340b3630 con 0x7f07340d1220 2023-07-18T03:38:37.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f073959c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f07340b1e00 0x7f07340afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:37.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f073959c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f07340b1e00 0x7f07340afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45918/0 (socket says 172.21.15.49:45918) 2023-07-18T03:38:37.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f073959c700 1 -- 172.21.15.49:0/111265968 learned_addr learned my addr 172.21.15.49:0/111265968 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:37.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f0733fff700 1 --2- 172.21.15.49:0/111265968 >> v2:172.21.15.107:3300/0 conn(0x7f07340d1220 0x7f07340b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:37.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f073959c700 1 -- 172.21.15.49:0/111265968 >> v2:172.21.15.107:3300/0 conn(0x7f07340d1220 msgr2=0x7f07340b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:37.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f073959c700 1 --2- 172.21.15.49:0/111265968 >> v2:172.21.15.107:3300/0 conn(0x7f07340d1220 0x7f07340b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f073959c700 1 -- 172.21.15.49:0/111265968 >> v1:172.21.15.107:6789/0 conn(0x7f073415c950 legacy=0x7f07340d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:37.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f073959c700 1 -- 172.21.15.49:0/111265968 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07340b3910 con 0x7f07340b1e00 2023-07-18T03:38:37.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.223+0000 7f0733fff700 1 --2- 172.21.15.49:0/111265968 >> v2:172.21.15.107:3300/0 conn(0x7f07340d1220 0x7f07340b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:37.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.224+0000 7f073959c700 1 --2- 172.21.15.49:0/111265968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07340b1e00 0x7f07340afac0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0734072140 tx=0x7f072c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21a2d4060bce09e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:37.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.224+0000 7f07337fe700 1 -- 172.21.15.49:0/111265968 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c00e070 con 0x7f07340b1e00 2023-07-18T03:38:37.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.224+0000 7f07337fe700 1 -- 172.21.15.49:0/111265968 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c00b9c0 con 0x7f07340b1e00 2023-07-18T03:38:37.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.224+0000 7f07337fe700 1 -- 172.21.15.49:0/111265968 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c00bd70 con 0x7f07340b1e00 2023-07-18T03:38:37.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.224+0000 7f073afff700 1 -- 172.21.15.49:0/111265968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07340b1e00 msgr2=0x7f07340afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:37.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.224+0000 7f073afff700 1 --2- 172.21.15.49:0/111265968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07340b1e00 0x7f07340afac0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0734072140 tx=0x7f072c00b330 comp rx=0 tx=0).stop 2023-07-18T03:38:37.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.225+0000 7f073afff700 1 -- 172.21.15.49:0/111265968 shutdown_connections 2023-07-18T03:38:37.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.225+0000 7f073afff700 1 --2- 172.21.15.49:0/111265968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07340b1e00 0x7f07340afac0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.225+0000 7f073afff700 1 --2- 172.21.15.49:0/111265968 >> v2:172.21.15.107:3300/0 conn(0x7f07340d1220 0x7f07340b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.225+0000 7f073afff700 1 -- 172.21.15.49:0/111265968 >> 172.21.15.49:0/111265968 conn(0x7f073409c1d0 msgr2=0x7f07340d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:37.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.225+0000 7f073afff700 1 -- 172.21.15.49:0/111265968 shutdown_connections 2023-07-18T03:38:37.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.225+0000 7f073afff700 1 -- 172.21.15.49:0/111265968 wait complete. 2023-07-18T03:38:37.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.226+0000 7f073afff700 1 Processor -- start 2023-07-18T03:38:37.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.227+0000 7f073afff700 1 -- start start 2023-07-18T03:38:37.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073afff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07340b1e00 0x7f07340cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:37.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073afff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07340d1220 0x7f07340cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:37.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073afff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f073415c950 0x7f07340ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:37.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073afff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340ca010 con 0x7f073415c950 2023-07-18T03:38:37.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073afff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340ca8b0 con 0x7f07340b1e00 2023-07-18T03:38:37.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073afff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07340cf500 con 0x7f07340d1220 2023-07-18T03:38:37.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073959c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f073415c950 0x7f07340ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:37.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073959c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f073415c950 0x7f07340ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45932/0 (socket says 172.21.15.49:45932) 2023-07-18T03:38:37.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073959c700 1 -- 172.21.15.49:0/3816321257 learned_addr learned my addr 172.21.15.49:0/3816321257 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:37.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f0738d9b700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07340b1e00 0x7f07340cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:37.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f0733fff700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07340d1220 0x7f07340cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:37.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073959c700 1 -- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07340d1220 msgr2=0x7f07340cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:37.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073959c700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07340d1220 0x7f07340cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073959c700 1 -- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07340b1e00 msgr2=0x7f07340cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:37.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073959c700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07340b1e00 0x7f07340cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.228+0000 7f073959c700 1 -- 172.21.15.49:0/3816321257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c009b30 con 0x7f073415c950 2023-07-18T03:38:37.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.229+0000 7f073959c700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f073415c950 0x7f07340ceab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f07340673b0 tx=0x7f072c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:37.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.229+0000 7f07317fa700 1 -- 172.21.15.49:0/3816321257 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c004880 con 0x7f073415c950 2023-07-18T03:38:37.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.229+0000 7f07317fa700 1 -- 172.21.15.49:0/3816321257 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c0092e0 con 0x7f073415c950 2023-07-18T03:38:37.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.229+0000 7f07317fa700 1 -- 172.21.15.49:0/3816321257 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c01c680 con 0x7f073415c950 2023-07-18T03:38:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.229+0000 7f073afff700 1 -- 172.21.15.49:0/3816321257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07340ca260 con 0x7f073415c950 2023-07-18T03:38:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.229+0000 7f073afff700 1 -- 172.21.15.49:0/3816321257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07340cb280 con 0x7f073415c950 2023-07-18T03:38:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.230+0000 7f073afff700 1 -- 172.21.15.49:0/3816321257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07340caa40 con 0x7f073415c950 2023-07-18T03:38:37.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.231+0000 7f07317fa700 1 -- 172.21.15.49:0/3816321257 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f072c01c7e0 con 0x7f073415c950 2023-07-18T03:38:37.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.232+0000 7f07317fa700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f071806ace0 0x7f071806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:37.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.232+0000 7f07317fa700 1 -- 172.21.15.49:0/3816321257 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f072c08b0b0 con 0x7f073415c950 2023-07-18T03:38:37.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.233+0000 7f0738d9b700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f071806ace0 0x7f071806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:37.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.233+0000 7f0738d9b700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f071806ace0 0x7f071806d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f07280017b0 tx=0x7f0728009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:37.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.236+0000 7f07317fa700 1 -- 172.21.15.49:0/3816321257 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f072c056150 con 0x7f073415c950 2023-07-18T03:38:37.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.447+0000 7f073afff700 1 -- 172.21.15.49:0/3816321257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f073404e280 con 0x7f073415c950 2023-07-18T03:38:37.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.448+0000 7f07317fa700 1 -- 172.21.15.49:0/3816321257 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18198 (secure 0 0 0) 0x7f072c059770 con 0x7f073415c950 2023-07-18T03:38:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:37.443 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":21,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:36.024875+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:37.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.451+0000 7f073afff700 1 -- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f071806ace0 msgr2=0x7f071806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:37.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.451+0000 7f073afff700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f071806ace0 0x7f071806d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f07280017b0 tx=0x7f0728009040 comp rx=0 tx=0).stop 2023-07-18T03:38:37.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.451+0000 7f073afff700 1 -- 172.21.15.49:0/3816321257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f073415c950 msgr2=0x7f07340ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:37.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.451+0000 7f073afff700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f073415c950 0x7f07340ceab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f07340673b0 tx=0x7f072c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.451+0000 7f073afff700 1 -- 172.21.15.49:0/3816321257 shutdown_connections 2023-07-18T03:38:37.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.451+0000 7f073afff700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f071806ace0 0x7f071806d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.451+0000 7f073afff700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f073415c950 0x7f07340ceab0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.451+0000 7f073afff700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07340b1e00 0x7f07340cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.451+0000 7f073afff700 1 --2- 172.21.15.49:0/3816321257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07340d1220 0x7f07340cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.451+0000 7f073afff700 1 -- 172.21.15.49:0/3816321257 >> 172.21.15.49:0/3816321257 conn(0x7f073409c1d0 msgr2=0x7f07340afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:37.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.451+0000 7f073afff700 1 -- 172.21.15.49:0/3816321257 shutdown_connections 2023-07-18T03:38:37.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.451+0000 7f073afff700 1 -- 172.21.15.49:0/3816321257 wait complete. 2023-07-18T03:38:37.464 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-18T03:38:37.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.678+0000 7f6ab75ff700 1 Processor -- start 2023-07-18T03:38:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.679+0000 7f6ab75ff700 1 -- start start 2023-07-18T03:38:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.680+0000 7f6ab75ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6ab00d0f90 0x7f6ab00b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.680+0000 7f6ab75ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ab00b8e30 con 0x7f6ab00d0b10 2023-07-18T03:38:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.680+0000 7f6ab75ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ab00afaa0 con 0x7f6ab00b8a60 2023-07-18T03:38:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.680+0000 7f6ab75ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab00afd80 con 0x7f6ab00d0f90 2023-07-18T03:38:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.680+0000 7f6ab4b9a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6ab00d0b10 0x7f6ab00b0e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45948/0 (socket says 172.21.15.49:45948) 2023-07-18T03:38:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.680+0000 7f6ab4b9a700 1 -- 172.21.15.49:0/3504165546 learned_addr learned my addr 172.21.15.49:0/3504165546 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.680+0000 7f6ab5b9c700 1 --2- 172.21.15.49:0/3504165546 >> v2:172.21.15.107:3300/0 conn(0x7f6ab00d0f90 0x7f6ab00b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.681+0000 7f6aa7fff700 1 -- 172.21.15.49:0/3504165546 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904672721 0 0) 0x7f6ab00b8e30 con 0x7f6ab00d0b10 2023-07-18T03:38:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.681+0000 7f6aa7fff700 1 -- 172.21.15.49:0/3504165546 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a9c0035b0 con 0x7f6ab00d0b10 2023-07-18T03:38:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.681+0000 7f6aa7fff700 1 -- 172.21.15.49:0/3504165546 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553116250 0 0) 0x7f6ab00afaa0 con 0x7f6ab00b8a60 2023-07-18T03:38:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.681+0000 7f6aa7fff700 1 -- 172.21.15.49:0/3504165546 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ab00b8e30 con 0x7f6ab00b8a60 2023-07-18T03:38:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.681+0000 7f6aa7fff700 1 -- 172.21.15.49:0/3504165546 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 778503181 0 0) 0x7f6a9c0035b0 con 0x7f6ab00d0b10 2023-07-18T03:38:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.681+0000 7f6ab5b9c700 1 -- 172.21.15.49:0/3504165546 >> v1:172.21.15.107:6789/0 conn(0x7f6ab00b8a60 legacy=0x7f6ab00b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.681+0000 7f6ab5b9c700 1 -- 172.21.15.49:0/3504165546 >> v1:172.21.15.49:6789/0 conn(0x7f6ab00d0b10 legacy=0x7f6ab00b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.681+0000 7f6ab5b9c700 1 -- 172.21.15.49:0/3504165546 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab00b0060 con 0x7f6ab00d0f90 2023-07-18T03:38:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.681+0000 7f6ab5b9c700 1 --2- 172.21.15.49:0/3504165546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab00d0f90 0x7f6ab00b1530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009d90 tx=0x7f6aa800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a06a07c6a0b7c6e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.682+0000 7f6aa7fff700 1 -- 172.21.15.49:0/3504165546 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa800e070 con 0x7f6ab00d0f90 2023-07-18T03:38:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.682+0000 7f6aa7fff700 1 -- 172.21.15.49:0/3504165546 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa8003a20 con 0x7f6ab00d0f90 2023-07-18T03:38:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.682+0000 7f6aa7fff700 1 -- 172.21.15.49:0/3504165546 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0001670 con 0x7f6ab00d0f90 2023-07-18T03:38:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.682+0000 7f6ab75ff700 1 -- 172.21.15.49:0/3504165546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab00d0f90 msgr2=0x7f6ab00b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.682+0000 7f6ab75ff700 1 --2- 172.21.15.49:0/3504165546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab00d0f90 0x7f6ab00b1530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009d90 tx=0x7f6aa800b330 comp rx=0 tx=0).stop 2023-07-18T03:38:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.682+0000 7f6ab75ff700 1 -- 172.21.15.49:0/3504165546 shutdown_connections 2023-07-18T03:38:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.682+0000 7f6ab75ff700 1 --2- 172.21.15.49:0/3504165546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab00d0f90 0x7f6ab00b1530 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.682+0000 7f6ab75ff700 1 -- 172.21.15.49:0/3504165546 >> 172.21.15.49:0/3504165546 conn(0x7f6ab009c1b0 msgr2=0x7f6ab00d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.682+0000 7f6ab75ff700 1 -- 172.21.15.49:0/3504165546 shutdown_connections 2023-07-18T03:38:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.683+0000 7f6ab75ff700 1 -- 172.21.15.49:0/3504165546 wait complete. 2023-07-18T03:38:37.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.684+0000 7f6ab75ff700 1 Processor -- start 2023-07-18T03:38:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.684+0000 7f6ab75ff700 1 -- start start 2023-07-18T03:38:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.685+0000 7f6ab75ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ab00b8a60 0x7f6ab00cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.685+0000 7f6ab75ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab00d0b10 0x7f6ab01f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.685+0000 7f6ab75ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ab00d0f90 0x7f6ab00b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.685+0000 7f6ab75ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab014a7c0 con 0x7f6ab00b8a60 2023-07-18T03:38:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.685+0000 7f6ab75ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00d0450 con 0x7f6ab00d0b10 2023-07-18T03:38:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.685+0000 7f6ab75ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab0149e20 con 0x7f6ab00d0f90 2023-07-18T03:38:37.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.685+0000 7f6ab539b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ab00b8a60 0x7f6ab00cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:37.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.686+0000 7f6ab539b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ab00b8a60 0x7f6ab00cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45934/0 (socket says 172.21.15.49:45934) 2023-07-18T03:38:37.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.686+0000 7f6ab539b700 1 -- 172.21.15.49:0/2406958240 learned_addr learned my addr 172.21.15.49:0/2406958240 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:37.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.686+0000 7f6ab4b9a700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab00d0b10 0x7f6ab01f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:37.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.686+0000 7f6ab5b9c700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ab00d0f90 0x7f6ab00b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:37.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.686+0000 7f6ab539b700 1 -- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ab00d0f90 msgr2=0x7f6ab00b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:37.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.686+0000 7f6ab539b700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ab00d0f90 0x7f6ab00b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.686+0000 7f6ab539b700 1 -- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab00d0b10 msgr2=0x7f6ab01f16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.686+0000 7f6ab539b700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab00d0b10 0x7f6ab01f16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.686+0000 7f6ab539b700 1 -- 172.21.15.49:0/2406958240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa8009a70 con 0x7f6ab00b8a60 2023-07-18T03:38:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.686+0000 7f6ab5b9c700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ab00d0f90 0x7f6ab00b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.686+0000 7f6ab539b700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ab00b8a60 0x7f6ab00cfb50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00c520 tx=0x7f6aac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.686+0000 7f6aa5ffb700 1 -- 172.21.15.49:0/2406958240 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0052d0 con 0x7f6ab00b8a60 2023-07-18T03:38:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.687+0000 7f6aa5ffb700 1 -- 172.21.15.49:0/2406958240 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aac005430 con 0x7f6ab00b8a60 2023-07-18T03:38:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.687+0000 7f6ab75ff700 1 -- 172.21.15.49:0/2406958240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab014a100 con 0x7f6ab00b8a60 2023-07-18T03:38:37.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.687+0000 7f6aa5ffb700 1 -- 172.21.15.49:0/2406958240 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0038c0 con 0x7f6ab00b8a60 2023-07-18T03:38:37.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.687+0000 7f6ab75ff700 1 -- 172.21.15.49:0/2406958240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab00d0090 con 0x7f6ab00b8a60 2023-07-18T03:38:37.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.688+0000 7f6ab75ff700 1 -- 172.21.15.49:0/2406958240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab00d0220 con 0x7f6ab00b8a60 2023-07-18T03:38:37.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.689+0000 7f6aa5ffb700 1 -- 172.21.15.49:0/2406958240 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6aac003a20 con 0x7f6ab00b8a60 2023-07-18T03:38:37.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.690+0000 7f6aa5ffb700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6a9406ace0 0x7f6a9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:37.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.690+0000 7f6aa5ffb700 1 -- 172.21.15.49:0/2406958240 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6aac088b00 con 0x7f6ab00b8a60 2023-07-18T03:38:37.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.690+0000 7f6ab4b9a700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6a9406ace0 0x7f6a9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:37.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.691+0000 7f6ab4b9a700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6a9406ace0 0x7f6a9406d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00ced60 tx=0x7f6aa00015f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.694+0000 7f6aa5ffb700 1 -- 172.21.15.49:0/2406958240 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6aac08ae60 con 0x7f6ab00b8a60 2023-07-18T03:38:37.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:37.949+0000 7f6ab75ff700 1 -- 172.21.15.49:0/2406958240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6ab0074350 con 0x7f6ab00b8a60 2023-07-18T03:38:38.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.537+0000 7f6aa5ffb700 1 -- 172.21.15.49:0/2406958240 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6aac053b10 con 0x7f6ab00b8a60 2023-07-18T03:38:38.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.539+0000 7f6ab75ff700 1 -- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6a9406ace0 msgr2=0x7f6a9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:38.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.539+0000 7f6ab75ff700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6a9406ace0 0x7f6a9406d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00ced60 tx=0x7f6aa00015f0 comp rx=0 tx=0).stop 2023-07-18T03:38:38.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.539+0000 7f6ab75ff700 1 -- 172.21.15.49:0/2406958240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ab00b8a60 msgr2=0x7f6ab00cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:38.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.539+0000 7f6ab75ff700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ab00b8a60 0x7f6ab00cfb50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00c520 tx=0x7f6aac00c8e0 comp rx=0 tx=0).stop 2023-07-18T03:38:38.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.539+0000 7f6ab75ff700 1 -- 172.21.15.49:0/2406958240 shutdown_connections 2023-07-18T03:38:38.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.539+0000 7f6ab75ff700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ab00d0f90 0x7f6ab00b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:38.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.539+0000 7f6ab75ff700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6a9406ace0 0x7f6a9406d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.539+0000 7f6ab75ff700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ab00b8a60 0x7f6ab00cfb50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.539+0000 7f6ab75ff700 1 --2- 172.21.15.49:0/2406958240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab00d0b10 0x7f6ab01f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.539+0000 7f6ab75ff700 1 -- 172.21.15.49:0/2406958240 >> 172.21.15.49:0/2406958240 conn(0x7f6ab009c1b0 msgr2=0x7f6ab00b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.540+0000 7f6ab75ff700 1 -- 172.21.15.49:0/2406958240 shutdown_connections 2023-07-18T03:38:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.540+0000 7f6ab75ff700 1 -- 172.21.15.49:0/2406958240 wait complete. 2023-07-18T03:38:38.553 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-18T03:38:38.554 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-18T03:38:38.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.768+0000 7faded5ff700 1 Processor -- start 2023-07-18T03:38:38.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.768+0000 7faded5ff700 1 -- start start 2023-07-18T03:38:38.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7faded5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fade80d0b10 0x7fade80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:38.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7faded5ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fade80b0e20 0x7fade80b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:38.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7faded5ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade80b8e30 con 0x7fade80b8a60 2023-07-18T03:38:38.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7faded5ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade80d14d0 con 0x7fade80b0e20 2023-07-18T03:38:38.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7faded5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade80afaa0 con 0x7fade80d0b10 2023-07-18T03:38:38.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7fade6ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fade80b8a60 0x7fade80b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45954/0 (socket says 172.21.15.49:45954) 2023-07-18T03:38:38.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7fade6ffd700 1 -- 172.21.15.49:0/1781081253 learned_addr learned my addr 172.21.15.49:0/1781081253 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:38.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7fade77fe700 1 --2- 172.21.15.49:0/1781081253 >> v2:172.21.15.49:3300/0 conn(0x7fade80b0e20 0x7fade80b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:38.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7faddffff700 1 --2- 172.21.15.49:0/1781081253 >> v2:172.21.15.107:3300/0 conn(0x7fade80d0b10 0x7fade80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:38.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7fade77fe700 1 -- 172.21.15.49:0/1781081253 >> v2:172.21.15.107:3300/0 conn(0x7fade80d0b10 msgr2=0x7fade80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:38.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7fade77fe700 1 --2- 172.21.15.49:0/1781081253 >> v2:172.21.15.107:3300/0 conn(0x7fade80d0b10 0x7fade80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:38.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7fade77fe700 1 -- 172.21.15.49:0/1781081253 >> v1:172.21.15.49:6789/0 conn(0x7fade80b8a60 legacy=0x7fade80b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:38.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7fade77fe700 1 -- 172.21.15.49:0/1781081253 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade80afd80 con 0x7fade80b0e20 2023-07-18T03:38:38.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.770+0000 7faddffff700 1 --2- 172.21.15.49:0/1781081253 >> v2:172.21.15.107:3300/0 conn(0x7fade80d0b10 0x7fade80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:38.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.771+0000 7fade77fe700 1 --2- 172.21.15.49:0/1781081253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fade80b0e20 0x7fade80b1260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fade809c8e0 tx=0x7fade000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe7ff2770a5b66 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:38.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.771+0000 7fade67fc700 1 -- 172.21.15.49:0/1781081253 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade000e070 con 0x7fade80b0e20 2023-07-18T03:38:38.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.771+0000 7fade67fc700 1 -- 172.21.15.49:0/1781081253 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade000bbd0 con 0x7fade80b0e20 2023-07-18T03:38:38.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.771+0000 7fade67fc700 1 -- 172.21.15.49:0/1781081253 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0012430 con 0x7fade80b0e20 2023-07-18T03:38:38.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.771+0000 7faded5ff700 1 -- 172.21.15.49:0/1781081253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fade80b0e20 msgr2=0x7fade80b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:38.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.771+0000 7faded5ff700 1 --2- 172.21.15.49:0/1781081253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fade80b0e20 0x7fade80b1260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fade809c8e0 tx=0x7fade000b330 comp rx=0 tx=0).stop 2023-07-18T03:38:38.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.772+0000 7faded5ff700 1 -- 172.21.15.49:0/1781081253 shutdown_connections 2023-07-18T03:38:38.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.772+0000 7faded5ff700 1 --2- 172.21.15.49:0/1781081253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fade80b0e20 0x7fade80b1260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:38.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.772+0000 7faded5ff700 1 --2- 172.21.15.49:0/1781081253 >> v2:172.21.15.107:3300/0 conn(0x7fade80d0b10 0x7fade80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:38.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.772+0000 7faded5ff700 1 -- 172.21.15.49:0/1781081253 >> 172.21.15.49:0/1781081253 conn(0x7fade809c1b0 msgr2=0x7fade80d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:38.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.772+0000 7faded5ff700 1 -- 172.21.15.49:0/1781081253 shutdown_connections 2023-07-18T03:38:38.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.772+0000 7faded5ff700 1 -- 172.21.15.49:0/1781081253 wait complete. 2023-07-18T03:38:38.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.773+0000 7faded5ff700 1 Processor -- start 2023-07-18T03:38:38.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.774+0000 7faded5ff700 1 -- start start 2023-07-18T03:38:38.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7faded5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80b0e20 0x7fade80cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:38.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7faded5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade80b8a60 0x7fade8156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:38.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7faded5ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fade80d0b10 0x7fade81f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:38.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7faded5ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade80cbe80 con 0x7fade80d0b10 2023-07-18T03:38:38.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7faded5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade81f5fd0 con 0x7fade80b0e20 2023-07-18T03:38:38.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7faded5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade81f62b0 con 0x7fade80b8a60 2023-07-18T03:38:38.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7fade77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fade80d0b10 0x7fade81f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:38.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7fade77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fade80d0b10 0x7fade81f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45954/0 (socket says 172.21.15.49:45954) 2023-07-18T03:38:38.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7fade77fe700 1 -- 172.21.15.49:0/1062932589 learned_addr learned my addr 172.21.15.49:0/1062932589 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:38.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7fade6ffd700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80b0e20 0x7fade80cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:38.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7faddffff700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade80b8a60 0x7fade8156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:38.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7fade77fe700 1 -- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade80b8a60 msgr2=0x7fade8156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:38.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7fade77fe700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade80b8a60 0x7fade8156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:38.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7fade77fe700 1 -- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80b0e20 msgr2=0x7fade80cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:38.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7fade77fe700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80b0e20 0x7fade80cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:38.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7fade77fe700 1 -- 172.21.15.49:0/1062932589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0009a70 con 0x7fade80d0b10 2023-07-18T03:38:38.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7faddffff700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade80b8a60 0x7fade8156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:38.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.775+0000 7fade6ffd700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80b0e20 0x7fade80cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.776+0000 7fade77fe700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fade80d0b10 0x7fade81f5530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fade0014040 tx=0x7fade0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.776+0000 7faddf7fe700 1 -- 172.21.15.49:0/1062932589 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0012430 con 0x7fade80d0b10 2023-07-18T03:38:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.776+0000 7faddf7fe700 1 -- 172.21.15.49:0/1062932589 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade0020070 con 0x7fade80d0b10 2023-07-18T03:38:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.776+0000 7faded5ff700 1 -- 172.21.15.49:0/1062932589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade80c67a0 con 0x7fade80d0b10 2023-07-18T03:38:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.776+0000 7faddf7fe700 1 -- 172.21.15.49:0/1062932589 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0004280 con 0x7fade80d0b10 2023-07-18T03:38:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.776+0000 7faded5ff700 1 -- 172.21.15.49:0/1062932589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade80c6d60 con 0x7fade80d0b10 2023-07-18T03:38:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.777+0000 7faded5ff700 1 -- 172.21.15.49:0/1062932589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade8074350 con 0x7fade80d0b10 2023-07-18T03:38:38.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.778+0000 7faddf7fe700 1 -- 172.21.15.49:0/1062932589 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fade00043e0 con 0x7fade80d0b10 2023-07-18T03:38:38.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.779+0000 7faddf7fe700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fadc806ace0 0x7fadc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:38.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.779+0000 7faddf7fe700 1 -- 172.21.15.49:0/1062932589 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fade0022070 con 0x7fade80d0b10 2023-07-18T03:38:38.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.779+0000 7fade6ffd700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fadc806ace0 0x7fadc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:38.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.780+0000 7fade6ffd700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fadc806ace0 0x7fadc806d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005910 tx=0x7fadd8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:38.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:38.783+0000 7faddf7fe700 1 -- 172.21.15.49:0/1062932589 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fade0056740 con 0x7fade80d0b10 2023-07-18T03:38:39.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.039+0000 7faded5ff700 1 -- 172.21.15.49:0/1062932589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fade80b54e0 con 0x7fade80d0b10 2023-07-18T03:38:39.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.542+0000 7faddf7fe700 1 -- 172.21.15.49:0/1062932589 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fade0059d60 con 0x7fade80d0b10 2023-07-18T03:38:39.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.545+0000 7faded5ff700 1 -- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fadc806ace0 msgr2=0x7fadc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:39.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.545+0000 7faded5ff700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fadc806ace0 0x7fadc806d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005910 tx=0x7fadd8009040 comp rx=0 tx=0).stop 2023-07-18T03:38:39.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.545+0000 7faded5ff700 1 -- 172.21.15.49:0/1062932589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fade80d0b10 msgr2=0x7fade81f5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:39.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.545+0000 7faded5ff700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fade80d0b10 0x7fade81f5530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fade0014040 tx=0x7fade0004ad0 comp rx=0 tx=0).stop 2023-07-18T03:38:39.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.545+0000 7faded5ff700 1 -- 172.21.15.49:0/1062932589 shutdown_connections 2023-07-18T03:38:39.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.545+0000 7faded5ff700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fadc806ace0 0x7fadc806d190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:39.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.545+0000 7faded5ff700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fade80d0b10 0x7fade81f5530 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:39.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.545+0000 7faded5ff700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80b0e20 0x7fade80cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:39.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.545+0000 7faded5ff700 1 --2- 172.21.15.49:0/1062932589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade80b8a60 0x7fade8156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:39.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.545+0000 7faded5ff700 1 -- 172.21.15.49:0/1062932589 >> 172.21.15.49:0/1062932589 conn(0x7fade809c1b0 msgr2=0x7fade815c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:39.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.545+0000 7faded5ff700 1 -- 172.21.15.49:0/1062932589 shutdown_connections 2023-07-18T03:38:39.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.545+0000 7faded5ff700 1 -- 172.21.15.49:0/1062932589 wait complete. 2023-07-18T03:38:39.560 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-18T03:38:39.560 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:38:39.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.770+0000 7fc40ddff700 1 Processor -- start 2023-07-18T03:38:39.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.771+0000 7fc40ddff700 1 -- start start 2023-07-18T03:38:39.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.772+0000 7fc40ddff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc40815c950 0x7fc4080b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:39.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.772+0000 7fc40ddff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4080d2ce0 con 0x7fc4080d1220 2023-07-18T03:38:39.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.772+0000 7fc40ddff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4080b3350 con 0x7fc4080b1e00 2023-07-18T03:38:39.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.772+0000 7fc40ddff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4080b3630 con 0x7fc40815c950 2023-07-18T03:38:39.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.772+0000 7fc4077fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc40815c950 0x7fc4080b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:39.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.772+0000 7fc4077fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc40815c950 0x7fc4080b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40172/0 (socket says 172.21.15.49:40172) 2023-07-18T03:38:39.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.772+0000 7fc4077fe700 1 -- 172.21.15.49:0/980273717 learned_addr learned my addr 172.21.15.49:0/980273717 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.772+0000 7fc4077fe700 1 -- 172.21.15.49:0/980273717 >> v1:172.21.15.107:6789/0 conn(0x7fc4080b1e00 legacy=0x7fc4080d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.772+0000 7fc4077fe700 1 -- 172.21.15.49:0/980273717 >> v1:172.21.15.49:6789/0 conn(0x7fc4080d1220 legacy=0x7fc4080afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.772+0000 7fc4077fe700 1 -- 172.21.15.49:0/980273717 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4080b3910 con 0x7fc40815c950 2023-07-18T03:38:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.773+0000 7fc4077fe700 1 --2- 172.21.15.49:0/980273717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc40815c950 0x7fc4080b43f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80098e0 tx=0x7fc3f8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d50ff320c36218 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.773+0000 7fc406ffd700 1 -- 172.21.15.49:0/980273717 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f800b8a0 con 0x7fc40815c950 2023-07-18T03:38:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.773+0000 7fc406ffd700 1 -- 172.21.15.49:0/980273717 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f800ba00 con 0x7fc40815c950 2023-07-18T03:38:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.773+0000 7fc406ffd700 1 -- 172.21.15.49:0/980273717 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f800bdb0 con 0x7fc40815c950 2023-07-18T03:38:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.773+0000 7fc40ddff700 1 -- 172.21.15.49:0/980273717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc40815c950 msgr2=0x7fc4080b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.773+0000 7fc40ddff700 1 --2- 172.21.15.49:0/980273717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc40815c950 0x7fc4080b43f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80098e0 tx=0x7fc3f8009bf0 comp rx=0 tx=0).stop 2023-07-18T03:38:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.774+0000 7fc40ddff700 1 -- 172.21.15.49:0/980273717 shutdown_connections 2023-07-18T03:38:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.774+0000 7fc40ddff700 1 --2- 172.21.15.49:0/980273717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc40815c950 0x7fc4080b43f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.774+0000 7fc40ddff700 1 -- 172.21.15.49:0/980273717 >> 172.21.15.49:0/980273717 conn(0x7fc40809c1d0 msgr2=0x7fc4080d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.774+0000 7fc40ddff700 1 -- 172.21.15.49:0/980273717 shutdown_connections 2023-07-18T03:38:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.774+0000 7fc40ddff700 1 -- 172.21.15.49:0/980273717 wait complete. 2023-07-18T03:38:39.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.775+0000 7fc40ddff700 1 Processor -- start 2023-07-18T03:38:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.776+0000 7fc40ddff700 1 -- start start 2023-07-18T03:38:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.776+0000 7fc40ddff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4080b1e00 0x7fc40814fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc40ddff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4080d1220 0x7fc40815fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc40ddff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc40815c950 0x7fc40815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc40ddff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4081502f0 con 0x7fc4080b1e00 2023-07-18T03:38:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc40ddff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4080c8080 con 0x7fc4080d1220 2023-07-18T03:38:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc40ddff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4080c8360 con 0x7fc40815c950 2023-07-18T03:38:39.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc4077fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4080b1e00 0x7fc40814fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:39.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc4077fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4080b1e00 0x7fc40814fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40184/0 (socket says 172.21.15.49:40184) 2023-07-18T03:38:39.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc4077fe700 1 -- 172.21.15.49:0/2088771284 learned_addr learned my addr 172.21.15.49:0/2088771284 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:39.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc4077fe700 1 -- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc40815c950 msgr2=0x7fc40815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:39.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc4077fe700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc40815c950 0x7fc40815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:39.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc3fffff700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4080d1220 0x7fc40815fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:39.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc4077fe700 1 -- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4080d1220 msgr2=0x7fc40815fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:39.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc4077fe700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4080d1220 0x7fc40815fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:39.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc4077fe700 1 -- 172.21.15.49:0/2088771284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f80094f0 con 0x7fc4080b1e00 2023-07-18T03:38:39.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc3fffff700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4080d1220 0x7fc40815fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:39.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.777+0000 7fc4077fe700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4080b1e00 0x7fc40814fdb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8000c00 tx=0x7fc3f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:39.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.778+0000 7fc404ff9700 1 -- 172.21.15.49:0/2088771284 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8011e00 con 0x7fc4080b1e00 2023-07-18T03:38:39.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.778+0000 7fc404ff9700 1 -- 172.21.15.49:0/2088771284 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f801e070 con 0x7fc4080b1e00 2023-07-18T03:38:39.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.778+0000 7fc404ff9700 1 -- 172.21.15.49:0/2088771284 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f801a660 con 0x7fc4080b1e00 2023-07-18T03:38:39.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.778+0000 7fc40ddff700 1 -- 172.21.15.49:0/2088771284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4080c8640 con 0x7fc4080b1e00 2023-07-18T03:38:39.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.778+0000 7fc40ddff700 1 -- 172.21.15.49:0/2088771284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4080c8c00 con 0x7fc4080b1e00 2023-07-18T03:38:39.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.779+0000 7fc40ddff700 1 -- 172.21.15.49:0/2088771284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40804e280 con 0x7fc4080b1e00 2023-07-18T03:38:39.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.780+0000 7fc404ff9700 1 -- 172.21.15.49:0/2088771284 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc3f801a7c0 con 0x7fc4080b1e00 2023-07-18T03:38:39.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.781+0000 7fc404ff9700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc3e806ace0 0x7fc3e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:39.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.781+0000 7fc404ff9700 1 -- 172.21.15.49:0/2088771284 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3f808a8f0 con 0x7fc4080b1e00 2023-07-18T03:38:39.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.781+0000 7fc3fffff700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc3e806ace0 0x7fc3e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:39.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.782+0000 7fc3fffff700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc3e806ace0 0x7fc3e806d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4080c9a30 tx=0x7fc3f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:39.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:39.785+0000 7fc404ff9700 1 -- 172.21.15.49:0/2088771284 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc3f8055990 con 0x7fc4080b1e00 2023-07-18T03:38:40.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.042+0000 7fc40ddff700 1 -- 172.21.15.49:0/2088771284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc40808c5a0 con 0x7fc4080b1e00 2023-07-18T03:38:40.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.042+0000 7fc404ff9700 1 -- 172.21.15.49:0/2088771284 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4709 (secure 0 0 0) 0x7fc3f8015090 con 0x7fc4080b1e00 2023-07-18T03:38:40.037 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:40.038 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1477016053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1477016053},{"type":"v1","addr":"172.21.15.107:6835","nonce":1477016053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2998451500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2998451500},{"type":"v1","addr":"172.21.15.49:6835","nonce":2998451500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4247281109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4247281109},{"type":"v1","addr":"172.21.15.49:6833","nonce":4247281109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:38:36.025442+0000","modified":"2023-07-18T03:38:39.535929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1127901907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1127901907},{"type":"v1","addr":"172.21.15.107:6837","nonce":1127901907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:38:40.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.045+0000 7fc40ddff700 1 -- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc3e806ace0 msgr2=0x7fc3e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.045+0000 7fc40ddff700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc3e806ace0 0x7fc3e806d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4080c9a30 tx=0x7fc3f0009040 comp rx=0 tx=0).stop 2023-07-18T03:38:40.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.045+0000 7fc40ddff700 1 -- 172.21.15.49:0/2088771284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4080b1e00 msgr2=0x7fc40814fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.045+0000 7fc40ddff700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4080b1e00 0x7fc40814fdb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8000c00 tx=0x7fc3f8004750 comp rx=0 tx=0).stop 2023-07-18T03:38:40.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.045+0000 7fc40ddff700 1 -- 172.21.15.49:0/2088771284 shutdown_connections 2023-07-18T03:38:40.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.045+0000 7fc40ddff700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc40815c950 0x7fc40815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.045+0000 7fc40ddff700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc3e806ace0 0x7fc3e806d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.045+0000 7fc40ddff700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4080b1e00 0x7fc40814fdb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.045+0000 7fc40ddff700 1 --2- 172.21.15.49:0/2088771284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4080d1220 0x7fc40815fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.045+0000 7fc40ddff700 1 -- 172.21.15.49:0/2088771284 >> 172.21.15.49:0/2088771284 conn(0x7fc40809c1d0 msgr2=0x7fc4080b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:40.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.045+0000 7fc40ddff700 1 -- 172.21.15.49:0/2088771284 shutdown_connections 2023-07-18T03:38:40.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.045+0000 7fc40ddff700 1 -- 172.21.15.49:0/2088771284 wait complete. 2023-07-18T03:38:40.042 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2023-07-18T03:38:40.058 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:40.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.269+0000 7ff6f2fff700 1 Processor -- start 2023-07-18T03:38:40.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.269+0000 7ff6f2fff700 1 -- start start 2023-07-18T03:38:40.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f2fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff6ec0b87d0 0x7ff6ec160150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:40.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f2fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff6ec06ae40 0x7ff6ec0d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:40.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f2fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ec0cd4e0 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f2fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec0cdcf0 con 0x7ff6ec06ae40 2023-07-18T03:38:40.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f2fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec0d00d0 con 0x7ff6ec0b87d0 2023-07-18T03:38:40.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f159c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff6ec06ae40 0x7ff6ec0d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:40.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f159c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff6ec06ae40 0x7ff6ec0d0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40198/0 (socket says 172.21.15.49:40198) 2023-07-18T03:38:40.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f0d9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff6ec0b87d0 0x7ff6ec160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:40.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f159c700 1 -- 172.21.15.49:0/2378452437 learned_addr learned my addr 172.21.15.49:0/2378452437 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f159c700 1 -- 172.21.15.49:0/2378452437 >> v2:172.21.15.107:3300/0 conn(0x7ff6ec0b87d0 msgr2=0x7ff6ec160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f159c700 1 --2- 172.21.15.49:0/2378452437 >> v2:172.21.15.107:3300/0 conn(0x7ff6ec0b87d0 0x7ff6ec160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f159c700 1 -- 172.21.15.49:0/2378452437 >> v1:172.21.15.107:6789/0 conn(0x7ff6ec0cfc60 legacy=0x7ff6ec160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.271+0000 7ff6f159c700 1 -- 172.21.15.49:0/2378452437 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0cd730 con 0x7ff6ec06ae40 2023-07-18T03:38:40.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.272+0000 7ff6f159c700 1 --2- 172.21.15.49:0/2378452437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6ec06ae40 0x7ff6ec0d0e20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0b5a50 tx=0x7ff6e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900da6c8a04d2ec3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:40.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.272+0000 7ff6eb7fe700 1 -- 172.21.15.49:0/2378452437 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400e070 con 0x7ff6ec06ae40 2023-07-18T03:38:40.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.272+0000 7ff6eb7fe700 1 -- 172.21.15.49:0/2378452437 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e400b910 con 0x7ff6ec06ae40 2023-07-18T03:38:40.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.272+0000 7ff6eb7fe700 1 -- 172.21.15.49:0/2378452437 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400bcc0 con 0x7ff6ec06ae40 2023-07-18T03:38:40.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.273+0000 7ff6f2fff700 1 -- 172.21.15.49:0/2378452437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6ec06ae40 msgr2=0x7ff6ec0d0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.273+0000 7ff6f2fff700 1 --2- 172.21.15.49:0/2378452437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6ec06ae40 0x7ff6ec0d0e20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0b5a50 tx=0x7ff6e400b330 comp rx=0 tx=0).stop 2023-07-18T03:38:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.273+0000 7ff6f2fff700 1 -- 172.21.15.49:0/2378452437 shutdown_connections 2023-07-18T03:38:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.273+0000 7ff6f2fff700 1 --2- 172.21.15.49:0/2378452437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6ec06ae40 0x7ff6ec0d0e20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.273+0000 7ff6f2fff700 1 --2- 172.21.15.49:0/2378452437 >> v2:172.21.15.107:3300/0 conn(0x7ff6ec0b87d0 0x7ff6ec160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.273+0000 7ff6f2fff700 1 -- 172.21.15.49:0/2378452437 >> 172.21.15.49:0/2378452437 conn(0x7ff6ec06c140 msgr2=0x7ff6ec06c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.273+0000 7ff6f2fff700 1 -- 172.21.15.49:0/2378452437 shutdown_connections 2023-07-18T03:38:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.273+0000 7ff6f2fff700 1 -- 172.21.15.49:0/2378452437 wait complete. 2023-07-18T03:38:40.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.275+0000 7ff6f2fff700 1 Processor -- start 2023-07-18T03:38:40.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.275+0000 7ff6f2fff700 1 -- start start 2023-07-18T03:38:40.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.276+0000 7ff6f2fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ec0b87d0 0x7ff6ec1f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:40.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.276+0000 7ff6f2fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6ec0cfc60 0x7ff6ec15e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.276+0000 7ff6f2fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6ec1f62d0 0x7ff6ec0c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.276+0000 7ff6f2fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec0c9d40 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.276+0000 7ff6f2fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec0c9a60 con 0x7ff6ec0b87d0 2023-07-18T03:38:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.276+0000 7ff6f2fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec1f2190 con 0x7ff6ec1f62d0 2023-07-18T03:38:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.276+0000 7ff6ebfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6ec0cfc60 0x7ff6ec15e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:40.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.276+0000 7ff6ebfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6ec0cfc60 0x7ff6ec15e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40202/0 (socket says 172.21.15.49:40202) 2023-07-18T03:38:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.276+0000 7ff6ebfff700 1 -- 172.21.15.49:0/872892969 learned_addr learned my addr 172.21.15.49:0/872892969 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.276+0000 7ff6f0d9b700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ec0b87d0 0x7ff6ec1f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.276+0000 7ff6f159c700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6ec1f62d0 0x7ff6ec0c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.277+0000 7ff6ebfff700 1 -- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6ec1f62d0 msgr2=0x7ff6ec0c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.277+0000 7ff6ebfff700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6ec1f62d0 0x7ff6ec0c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.277+0000 7ff6ebfff700 1 -- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ec0b87d0 msgr2=0x7ff6ec1f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.277+0000 7ff6ebfff700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ec0b87d0 0x7ff6ec1f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.277+0000 7ff6ebfff700 1 -- 172.21.15.49:0/872892969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e4009b30 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.277+0000 7ff6ebfff700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6ec0cfc60 0x7ff6ec15e2b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800c450 tx=0x7ff6d800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:40.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.277+0000 7ff6e97fa700 1 -- 172.21.15.49:0/872892969 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80052d0 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.277+0000 7ff6e97fa700 1 -- 172.21.15.49:0/872892969 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d8005430 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.277+0000 7ff6e97fa700 1 -- 172.21.15.49:0/872892969 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80038c0 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.277+0000 7ff6f2fff700 1 -- 172.21.15.49:0/872892969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec1f2470 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.277+0000 7ff6f2fff700 1 -- 172.21.15.49:0/872892969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ec15e7f0 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.278+0000 7ff6f2fff700 1 -- 172.21.15.49:0/872892969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ec1f2990 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.279+0000 7ff6e97fa700 1 -- 172.21.15.49:0/872892969 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff6d8003a20 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.280+0000 7ff6e97fa700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff6d006ace0 0x7ff6d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:40.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.281+0000 7ff6e97fa700 1 -- 172.21.15.49:0/872892969 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff6d8088840 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.281+0000 7ff6f0d9b700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff6d006ace0 0x7ff6d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:40.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.281+0000 7ff6f0d9b700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff6d006ace0 0x7ff6d006d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0b28c0 tx=0x7ff6e0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:40.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.285+0000 7ff6e97fa700 1 -- 172.21.15.49:0/872892969 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff6ec1f2990 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.496+0000 7ff6f2fff700 1 -- 172.21.15.49:0/872892969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6ec1f2990 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.497+0000 7ff6e97fa700 1 -- 172.21.15.49:0/872892969 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18185 (secure 0 0 0) 0x7ff6ec1f2990 con 0x7ff6ec0cfc60 2023-07-18T03:38:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:40.494 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:37.510006+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:40.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.499+0000 7ff6f2fff700 1 -- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff6d006ace0 msgr2=0x7ff6d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.499+0000 7ff6f2fff700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff6d006ace0 0x7ff6d006d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0b28c0 tx=0x7ff6e0009210 comp rx=0 tx=0).stop 2023-07-18T03:38:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.499+0000 7ff6f2fff700 1 -- 172.21.15.49:0/872892969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6ec0cfc60 msgr2=0x7ff6ec15e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.499+0000 7ff6f2fff700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6ec0cfc60 0x7ff6ec15e2b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800c450 tx=0x7ff6d800c760 comp rx=0 tx=0).stop 2023-07-18T03:38:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.500+0000 7ff6f2fff700 1 -- 172.21.15.49:0/872892969 shutdown_connections 2023-07-18T03:38:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.500+0000 7ff6f2fff700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6ec1f62d0 0x7ff6ec0c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.500+0000 7ff6f2fff700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6ec0b87d0 0x7ff6ec1f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.500+0000 7ff6f2fff700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff6d006ace0 0x7ff6d006d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.500+0000 7ff6f2fff700 1 --2- 172.21.15.49:0/872892969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6ec0cfc60 0x7ff6ec15e2b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.500+0000 7ff6f2fff700 1 -- 172.21.15.49:0/872892969 >> 172.21.15.49:0/872892969 conn(0x7ff6ec06c140 msgr2=0x7ff6ec0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:40.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.500+0000 7ff6f2fff700 1 -- 172.21.15.49:0/872892969 shutdown_connections 2023-07-18T03:38:40.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.500+0000 7ff6f2fff700 1 -- 172.21.15.49:0/872892969 wait complete. 2023-07-18T03:38:40.513 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-18T03:38:40.514 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:38:40.545 INFO:tasks.ceph.mds.d.smithi107.stderr:2023-07-18T03:38:40.549+0000 7f98bddf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-18T03:38:40.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.724+0000 7f248d1ff700 1 Processor -- start 2023-07-18T03:38:40.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.724+0000 7f248d1ff700 1 -- start start 2023-07-18T03:38:40.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.726+0000 7f248d1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24880b1e00 0x7f24880b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:40.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.726+0000 7f248d1ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24880d2ce0 con 0x7f24880d1220 2023-07-18T03:38:40.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.726+0000 7f248d1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24880b3350 con 0x7f248815c950 2023-07-18T03:38:40.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.726+0000 7f248d1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24880b3630 con 0x7f24880b1e00 2023-07-18T03:38:40.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.726+0000 7f248659c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f24880d1220 0x7f24880afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41518/0 (socket says 172.21.15.49:41518) 2023-07-18T03:38:40.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.726+0000 7f248659c700 1 -- 172.21.15.49:0/3921063662 learned_addr learned my addr 172.21.15.49:0/3921063662 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:40.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.726+0000 7f248759e700 1 --2- 172.21.15.49:0/3921063662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24880b1e00 0x7f24880b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:40.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.726+0000 7f2485d9b700 1 -- 172.21.15.49:0/3921063662 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464085088 0 0) 0x7f24880d2ce0 con 0x7f24880d1220 2023-07-18T03:38:40.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.726+0000 7f2485d9b700 1 -- 172.21.15.49:0/3921063662 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24700035b0 con 0x7f24880d1220 2023-07-18T03:38:40.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.727+0000 7f2485d9b700 1 -- 172.21.15.49:0/3921063662 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208122744 0 0) 0x7f24880b3350 con 0x7f248815c950 2023-07-18T03:38:40.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.727+0000 7f2485d9b700 1 -- 172.21.15.49:0/3921063662 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24880d2ce0 con 0x7f248815c950 2023-07-18T03:38:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.727+0000 7f2485d9b700 1 -- 172.21.15.49:0/3921063662 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1900878953 0 0) 0x7f24700035b0 con 0x7f24880d1220 2023-07-18T03:38:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.727+0000 7f248759e700 1 -- 172.21.15.49:0/3921063662 >> v1:172.21.15.107:6789/0 conn(0x7f248815c950 legacy=0x7f24880d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.727+0000 7f248759e700 1 -- 172.21.15.49:0/3921063662 >> v1:172.21.15.49:6789/0 conn(0x7f24880d1220 legacy=0x7f24880afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.727+0000 7f248759e700 1 -- 172.21.15.49:0/3921063662 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24880b3910 con 0x7f24880b1e00 2023-07-18T03:38:40.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.727+0000 7f248759e700 1 --2- 172.21.15.49:0/3921063662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24880b1e00 0x7f24880b43f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2480009dc0 tx=0x7f248000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6c1fd1a98a6d0f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.727+0000 7f2485d9b700 1 -- 172.21.15.49:0/3921063662 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248000e070 con 0x7f24880b1e00 2023-07-18T03:38:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.727+0000 7f2485d9b700 1 -- 172.21.15.49:0/3921063662 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2480003a20 con 0x7f24880b1e00 2023-07-18T03:38:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.727+0000 7f2485d9b700 1 -- 172.21.15.49:0/3921063662 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c001670 con 0x7f24880b1e00 2023-07-18T03:38:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.728+0000 7f248d1ff700 1 -- 172.21.15.49:0/3921063662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24880b1e00 msgr2=0x7f24880b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.728+0000 7f248d1ff700 1 --2- 172.21.15.49:0/3921063662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24880b1e00 0x7f24880b43f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2480009dc0 tx=0x7f248000b330 comp rx=0 tx=0).stop 2023-07-18T03:38:40.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.728+0000 7f248d1ff700 1 -- 172.21.15.49:0/3921063662 shutdown_connections 2023-07-18T03:38:40.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.728+0000 7f248d1ff700 1 --2- 172.21.15.49:0/3921063662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24880b1e00 0x7f24880b43f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.728+0000 7f248d1ff700 1 -- 172.21.15.49:0/3921063662 >> 172.21.15.49:0/3921063662 conn(0x7f248809c1d0 msgr2=0x7f24880d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:40.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.728+0000 7f248d1ff700 1 -- 172.21.15.49:0/3921063662 shutdown_connections 2023-07-18T03:38:40.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.728+0000 7f248d1ff700 1 -- 172.21.15.49:0/3921063662 wait complete. 2023-07-18T03:38:40.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.730+0000 7f248d1ff700 1 Processor -- start 2023-07-18T03:38:40.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.730+0000 7f248d1ff700 1 -- start start 2023-07-18T03:38:40.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248d1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24880b1e00 0x7f24880ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:40.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248d1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24880d1220 0x7f2488155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:40.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248d1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f248815c950 0x7f24880ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:40.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248d1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881f5ac0 con 0x7f24880d1220 2023-07-18T03:38:40.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248d1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881559b0 con 0x7f248815c950 2023-07-18T03:38:40.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248d1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24880cf1e0 con 0x7f24880b1e00 2023-07-18T03:38:40.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24880d1220 0x7f2488155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:40.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24880d1220 0x7f2488155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40218/0 (socket says 172.21.15.49:40218) 2023-07-18T03:38:40.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248659c700 1 -- 172.21.15.49:0/1348414189 learned_addr learned my addr 172.21.15.49:0/1348414189 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:40.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248759e700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f248815c950 0x7f24880ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:40.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f2486d9d700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24880b1e00 0x7f24880ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:40.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248659c700 1 -- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24880b1e00 msgr2=0x7f24880ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248659c700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24880b1e00 0x7f24880ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248659c700 1 -- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f248815c950 msgr2=0x7f24880ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248659c700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f248815c950 0x7f24880ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248659c700 1 -- 172.21.15.49:0/1348414189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2480009a70 con 0x7f24880d1220 2023-07-18T03:38:40.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f248759e700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f248815c950 0x7f24880ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:40.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.731+0000 7f2486d9d700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24880b1e00 0x7f24880ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:40.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.732+0000 7f248659c700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24880d1220 0x7f2488155470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f247c00c6b0 tx=0x7f247c00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:40.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.732+0000 7f246f7fe700 1 -- 172.21.15.49:0/1348414189 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c004c80 con 0x7f24880d1220 2023-07-18T03:38:40.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.732+0000 7f246f7fe700 1 -- 172.21.15.49:0/1348414189 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c004de0 con 0x7f24880d1220 2023-07-18T03:38:40.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.732+0000 7f246f7fe700 1 -- 172.21.15.49:0/1348414189 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c007310 con 0x7f24880d1220 2023-07-18T03:38:40.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.732+0000 7f248d1ff700 1 -- 172.21.15.49:0/1348414189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24880cf4c0 con 0x7f24880d1220 2023-07-18T03:38:40.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.732+0000 7f248d1ff700 1 -- 172.21.15.49:0/1348414189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24881f5750 con 0x7f24880d1220 2023-07-18T03:38:40.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.733+0000 7f248d1ff700 1 -- 172.21.15.49:0/1348414189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24881f5c70 con 0x7f24880d1220 2023-07-18T03:38:40.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.734+0000 7f246f7fe700 1 -- 172.21.15.49:0/1348414189 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f247c007470 con 0x7f24880d1220 2023-07-18T03:38:40.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.735+0000 7f246f7fe700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f247406ace0 0x7f247406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:40.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.735+0000 7f246f7fe700 1 -- 172.21.15.49:0/1348414189 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f247c0892b0 con 0x7f24880d1220 2023-07-18T03:38:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.735+0000 7f2486d9d700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f247406ace0 0x7f247406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.736+0000 7f2486d9d700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f247406ace0 0x7f247406d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2478001270 tx=0x7f2478009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:40.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.739+0000 7f246f7fe700 1 -- 172.21.15.49:0/1348414189 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f24881f5c70 con 0x7f24880d1220 2023-07-18T03:38:40.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.944+0000 7f248d1ff700 1 -- 172.21.15.49:0/1348414189 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24880b3ed0 con 0x7f247406ace0 2023-07-18T03:38:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.956+0000 7f246f7fe700 1 -- 172.21.15.49:0/1348414189 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286890 (secure 0 0 0) 0x7f24780123d0 con 0x7f247406ace0 2023-07-18T03:38:40.954 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.960+0000 7f248d1ff700 1 -- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f247406ace0 msgr2=0x7f247406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.960+0000 7f248d1ff700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f247406ace0 0x7f247406d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2478001270 tx=0x7f2478009040 comp rx=0 tx=0).stop 2023-07-18T03:38:40.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.960+0000 7f248d1ff700 1 -- 172.21.15.49:0/1348414189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24880d1220 msgr2=0x7f2488155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:40.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.960+0000 7f248d1ff700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24880d1220 0x7f2488155470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f247c00c6b0 tx=0x7f247c00ca70 comp rx=0 tx=0).stop 2023-07-18T03:38:40.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.960+0000 7f248d1ff700 1 -- 172.21.15.49:0/1348414189 shutdown_connections 2023-07-18T03:38:40.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.960+0000 7f248d1ff700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f248815c950 0x7f24880ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.960+0000 7f248d1ff700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24880b1e00 0x7f24880ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.960+0000 7f248d1ff700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f247406ace0 0x7f247406d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.960+0000 7f248d1ff700 1 --2- 172.21.15.49:0/1348414189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24880d1220 0x7f2488155470 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.960+0000 7f248d1ff700 1 -- 172.21.15.49:0/1348414189 >> 172.21.15.49:0/1348414189 conn(0x7f248809c1d0 msgr2=0x7f24880d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.960+0000 7f248d1ff700 1 -- 172.21.15.49:0/1348414189 shutdown_connections 2023-07-18T03:38:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:40.960+0000 7f248d1ff700 1 -- 172.21.15.49:0/1348414189 wait complete. 2023-07-18T03:38:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:38:40.974 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-07-18T03:38:40.097282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230784,"kb_used_data":2880,"kb_used_omap":0,"kb_used_meta":227840,"kb_avail":754743936,"statfs":{"total":773094113280,"available":772857790464,"internally_reserved":0,"allocated":2949120,"data_stored":1795142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":233308160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.598559"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.122105+0000","last_change":"2023-07-18T03:38:36.538478+0000","last_active":"2023-07-18T03:38:38.122105+0000","last_peered":"2023-07-18T03:38:38.122105+0000","last_clean":"2023-07-18T03:38:38.122105+0000","last_became_active":"2023-07-18T03:38:36.538174+0000","last_became_peered":"2023-07-18T03:38:36.538174+0000","last_unstale":"2023-07-18T03:38:38.122105+0000","last_undegraded":"2023-07-18T03:38:38.122105+0000","last_fullsized":"2023-07-18T03:38:38.122105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:10:09.118957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.542016+0000","last_change":"2023-07-18T03:38:36.538553+0000","last_active":"2023-07-18T03:38:36.542016+0000","last_peered":"2023-07-18T03:38:36.542016+0000","last_clean":"2023-07-18T03:38:36.542016+0000","last_became_active":"2023-07-18T03:38:36.538257+0000","last_became_peered":"2023-07-18T03:38:36.538257+0000","last_unstale":"2023-07-18T03:38:36.542016+0000","last_undegraded":"2023-07-18T03:38:36.542016+0000","last_fullsized":"2023-07-18T03:38:36.542016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:32:37.988899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537770+0000","last_change":"2023-07-18T03:38:36.526262+0000","last_active":"2023-07-18T03:38:36.537770+0000","last_peered":"2023-07-18T03:38:36.537770+0000","last_clean":"2023-07-18T03:38:36.537770+0000","last_became_active":"2023-07-18T03:38:36.525920+0000","last_became_peered":"2023-07-18T03:38:36.525920+0000","last_unstale":"2023-07-18T03:38:36.537770+0000","last_undegraded":"2023-07-18T03:38:36.537770+0000","last_fullsized":"2023-07-18T03:38:36.537770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:38.898070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121429+0000","last_change":"2023-07-18T03:38:36.535992+0000","last_active":"2023-07-18T03:38:38.121429+0000","last_peered":"2023-07-18T03:38:38.121429+0000","last_clean":"2023-07-18T03:38:38.121429+0000","last_became_active":"2023-07-18T03:38:36.535587+0000","last_became_peered":"2023-07-18T03:38:36.535587+0000","last_unstale":"2023-07-18T03:38:38.121429+0000","last_undegraded":"2023-07-18T03:38:38.121429+0000","last_fullsized":"2023-07-18T03:38:38.121429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:24.505474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121029+0000","last_change":"2023-07-18T03:38:36.535524+0000","last_active":"2023-07-18T03:38:38.121029+0000","last_peered":"2023-07-18T03:38:38.121029+0000","last_clean":"2023-07-18T03:38:38.121029+0000","last_became_active":"2023-07-18T03:38:36.534975+0000","last_became_peered":"2023-07-18T03:38:36.534975+0000","last_unstale":"2023-07-18T03:38:38.121029+0000","last_undegraded":"2023-07-18T03:38:38.121029+0000","last_fullsized":"2023-07-18T03:38:38.121029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:37:55.979750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.534617+0000","last_change":"2023-07-18T03:38:36.526306+0000","last_active":"2023-07-18T03:38:36.534617+0000","last_peered":"2023-07-18T03:38:36.534617+0000","last_clean":"2023-07-18T03:38:36.534617+0000","last_became_active":"2023-07-18T03:38:36.525982+0000","last_became_peered":"2023-07-18T03:38:36.525982+0000","last_unstale":"2023-07-18T03:38:36.534617+0000","last_undegraded":"2023-07-18T03:38:36.534617+0000","last_fullsized":"2023-07-18T03:38:36.534617+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:32:37.988899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516000+0000","last_change":"2023-07-18T03:38:36.527124+0000","last_active":"2023-07-18T03:38:37.516000+0000","last_peered":"2023-07-18T03:38:37.516000+0000","last_clean":"2023-07-18T03:38:37.516000+0000","last_became_active":"2023-07-18T03:38:36.526827+0000","last_became_peered":"2023-07-18T03:38:36.526827+0000","last_unstale":"2023-07-18T03:38:37.516000+0000","last_undegraded":"2023-07-18T03:38:37.516000+0000","last_fullsized":"2023-07-18T03:38:37.516000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:21:51.299094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537642+0000","last_change":"2023-07-18T03:38:36.529550+0000","last_active":"2023-07-18T03:38:36.537642+0000","last_peered":"2023-07-18T03:38:36.537642+0000","last_clean":"2023-07-18T03:38:36.537642+0000","last_became_active":"2023-07-18T03:38:36.529261+0000","last_became_peered":"2023-07-18T03:38:36.529261+0000","last_unstale":"2023-07-18T03:38:36.537642+0000","last_undegraded":"2023-07-18T03:38:36.537642+0000","last_fullsized":"2023-07-18T03:38:36.537642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:44:27.833608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121235+0000","last_change":"2023-07-18T03:38:36.534909+0000","last_active":"2023-07-18T03:38:38.121235+0000","last_peered":"2023-07-18T03:38:38.121235+0000","last_clean":"2023-07-18T03:38:38.121235+0000","last_became_active":"2023-07-18T03:38:36.534593+0000","last_became_peered":"2023-07-18T03:38:36.534593+0000","last_unstale":"2023-07-18T03:38:38.121235+0000","last_undegraded":"2023-07-18T03:38:38.121235+0000","last_fullsized":"2023-07-18T03:38:38.121235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:45:51.736824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.541969+0000","last_change":"2023-07-18T03:38:36.536965+0000","last_active":"2023-07-18T03:38:36.541969+0000","last_peered":"2023-07-18T03:38:36.541969+0000","last_clean":"2023-07-18T03:38:36.541969+0000","last_became_active":"2023-07-18T03:38:36.536665+0000","last_became_peered":"2023-07-18T03:38:36.536665+0000","last_unstale":"2023-07-18T03:38:36.541969+0000","last_undegraded":"2023-07-18T03:38:36.541969+0000","last_fullsized":"2023-07-18T03:38:36.541969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:59:18.578420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537794+0000","last_change":"2023-07-18T03:38:36.526499+0000","last_active":"2023-07-18T03:38:36.537794+0000","last_peered":"2023-07-18T03:38:36.537794+0000","last_clean":"2023-07-18T03:38:36.537794+0000","last_became_active":"2023-07-18T03:38:36.525871+0000","last_became_peered":"2023-07-18T03:38:36.525871+0000","last_unstale":"2023-07-18T03:38:36.537794+0000","last_undegraded":"2023-07-18T03:38:36.537794+0000","last_fullsized":"2023-07-18T03:38:36.537794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:04.554555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121914+0000","last_change":"2023-07-18T03:38:36.524528+0000","last_active":"2023-07-18T03:38:38.121914+0000","last_peered":"2023-07-18T03:38:38.121914+0000","last_clean":"2023-07-18T03:38:38.121914+0000","last_became_active":"2023-07-18T03:38:36.523892+0000","last_became_peered":"2023-07-18T03:38:36.523892+0000","last_unstale":"2023-07-18T03:38:38.121914+0000","last_undegraded":"2023-07-18T03:38:38.121914+0000","last_fullsized":"2023-07-18T03:38:38.121914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:01:02.396334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121759+0000","last_change":"2023-07-18T03:38:36.533293+0000","last_active":"2023-07-18T03:38:38.121759+0000","last_peered":"2023-07-18T03:38:38.121759+0000","last_clean":"2023-07-18T03:38:38.121759+0000","last_became_active":"2023-07-18T03:38:36.532953+0000","last_became_peered":"2023-07-18T03:38:36.532953+0000","last_unstale":"2023-07-18T03:38:38.121759+0000","last_undegraded":"2023-07-18T03:38:38.121759+0000","last_fullsized":"2023-07-18T03:38:38.121759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:35:14.137991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.542036+0000","last_change":"2023-07-18T03:38:36.536500+0000","last_active":"2023-07-18T03:38:36.542036+0000","last_peered":"2023-07-18T03:38:36.542036+0000","last_clean":"2023-07-18T03:38:36.542036+0000","last_became_active":"2023-07-18T03:38:36.536283+0000","last_became_peered":"2023-07-18T03:38:36.536283+0000","last_unstale":"2023-07-18T03:38:36.542036+0000","last_undegraded":"2023-07-18T03:38:36.542036+0000","last_fullsized":"2023-07-18T03:38:36.542036+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:39.335445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516206+0000","last_change":"2023-07-18T03:38:36.530577+0000","last_active":"2023-07-18T03:38:37.516206+0000","last_peered":"2023-07-18T03:38:37.516206+0000","last_clean":"2023-07-18T03:38:37.516206+0000","last_became_active":"2023-07-18T03:38:36.530268+0000","last_became_peered":"2023-07-18T03:38:36.530268+0000","last_unstale":"2023-07-18T03:38:37.516206+0000","last_undegraded":"2023-07-18T03:38:37.516206+0000","last_fullsized":"2023-07-18T03:38:37.516206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:33:14.215460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537775+0000","last_change":"2023-07-18T03:38:36.526520+0000","last_active":"2023-07-18T03:38:36.537775+0000","last_peered":"2023-07-18T03:38:36.537775+0000","last_clean":"2023-07-18T03:38:36.537775+0000","last_became_active":"2023-07-18T03:38:36.526165+0000","last_became_peered":"2023-07-18T03:38:36.526165+0000","last_unstale":"2023-07-18T03:38:36.537775+0000","last_undegraded":"2023-07-18T03:38:36.537775+0000","last_fullsized":"2023-07-18T03:38:36.537775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:53:16.579302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121313+0000","last_change":"2023-07-18T03:38:36.531837+0000","last_active":"2023-07-18T03:38:38.121313+0000","last_peered":"2023-07-18T03:38:38.121313+0000","last_clean":"2023-07-18T03:38:38.121313+0000","last_became_active":"2023-07-18T03:38:36.531588+0000","last_became_peered":"2023-07-18T03:38:36.531588+0000","last_unstale":"2023-07-18T03:38:38.121313+0000","last_undegraded":"2023-07-18T03:38:38.121313+0000","last_fullsized":"2023-07-18T03:38:38.121313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:57:34.438689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.539935+0000","last_change":"2023-07-18T03:38:36.536129+0000","last_active":"2023-07-18T03:38:36.539935+0000","last_peered":"2023-07-18T03:38:36.539935+0000","last_clean":"2023-07-18T03:38:36.539935+0000","last_became_active":"2023-07-18T03:38:36.535846+0000","last_became_peered":"2023-07-18T03:38:36.535846+0000","last_unstale":"2023-07-18T03:38:36.539935+0000","last_undegraded":"2023-07-18T03:38:36.539935+0000","last_fullsized":"2023-07-18T03:38:36.539935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:27.898176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.122134+0000","last_change":"2023-07-18T03:38:36.529212+0000","last_active":"2023-07-18T03:38:38.122134+0000","last_peered":"2023-07-18T03:38:38.122134+0000","last_clean":"2023-07-18T03:38:38.122134+0000","last_became_active":"2023-07-18T03:38:36.528956+0000","last_became_peered":"2023-07-18T03:38:36.528956+0000","last_unstale":"2023-07-18T03:38:38.122134+0000","last_undegraded":"2023-07-18T03:38:38.122134+0000","last_fullsized":"2023-07-18T03:38:38.122134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:23:08.908073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537935+0000","last_change":"2023-07-18T03:38:36.532777+0000","last_active":"2023-07-18T03:38:36.537935+0000","last_peered":"2023-07-18T03:38:36.537935+0000","last_clean":"2023-07-18T03:38:36.537935+0000","last_became_active":"2023-07-18T03:38:36.532519+0000","last_became_peered":"2023-07-18T03:38:36.532519+0000","last_unstale":"2023-07-18T03:38:36.537935+0000","last_undegraded":"2023-07-18T03:38:36.537935+0000","last_fullsized":"2023-07-18T03:38:36.537935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:54:42.804941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515979+0000","last_change":"2023-07-18T03:38:36.541130+0000","last_active":"2023-07-18T03:38:37.515979+0000","last_peered":"2023-07-18T03:38:37.515979+0000","last_clean":"2023-07-18T03:38:37.515979+0000","last_became_active":"2023-07-18T03:38:36.535483+0000","last_became_peered":"2023-07-18T03:38:36.535483+0000","last_unstale":"2023-07-18T03:38:37.515979+0000","last_undegraded":"2023-07-18T03:38:37.515979+0000","last_fullsized":"2023-07-18T03:38:37.515979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:46.558463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120549+0000","last_change":"2023-07-18T03:38:36.536797+0000","last_active":"2023-07-18T03:38:38.120549+0000","last_peered":"2023-07-18T03:38:38.120549+0000","last_clean":"2023-07-18T03:38:38.120549+0000","last_became_active":"2023-07-18T03:38:36.535389+0000","last_became_peered":"2023-07-18T03:38:36.535389+0000","last_unstale":"2023-07-18T03:38:38.120549+0000","last_undegraded":"2023-07-18T03:38:38.120549+0000","last_fullsized":"2023-07-18T03:38:38.120549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:22:44.555224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.541809+0000","last_change":"2023-07-18T03:38:36.533007+0000","last_active":"2023-07-18T03:38:36.541809+0000","last_peered":"2023-07-18T03:38:36.541809+0000","last_clean":"2023-07-18T03:38:36.541809+0000","last_became_active":"2023-07-18T03:38:36.532714+0000","last_became_peered":"2023-07-18T03:38:36.532714+0000","last_unstale":"2023-07-18T03:38:36.541809+0000","last_undegraded":"2023-07-18T03:38:36.541809+0000","last_fullsized":"2023-07-18T03:38:36.541809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:32:37.988899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515725+0000","last_change":"2023-07-18T03:38:36.541077+0000","last_active":"2023-07-18T03:38:37.515725+0000","last_peered":"2023-07-18T03:38:37.515725+0000","last_clean":"2023-07-18T03:38:37.515725+0000","last_became_active":"2023-07-18T03:38:36.533768+0000","last_became_peered":"2023-07-18T03:38:36.533768+0000","last_unstale":"2023-07-18T03:38:37.515725+0000","last_undegraded":"2023-07-18T03:38:37.515725+0000","last_fullsized":"2023-07-18T03:38:37.515725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:42.081234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.536458+0000","last_change":"2023-07-18T03:38:36.529999+0000","last_active":"2023-07-18T03:38:36.536458+0000","last_peered":"2023-07-18T03:38:36.536458+0000","last_clean":"2023-07-18T03:38:36.536458+0000","last_became_active":"2023-07-18T03:38:36.529489+0000","last_became_peered":"2023-07-18T03:38:36.529489+0000","last_unstale":"2023-07-18T03:38:36.536458+0000","last_undegraded":"2023-07-18T03:38:36.536458+0000","last_fullsized":"2023-07-18T03:38:36.536458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:18.677951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.533773+0000","last_change":"2023-07-18T03:38:36.525279+0000","last_active":"2023-07-18T03:38:36.533773+0000","last_peered":"2023-07-18T03:38:36.533773+0000","last_clean":"2023-07-18T03:38:36.533773+0000","last_became_active":"2023-07-18T03:38:36.525018+0000","last_became_peered":"2023-07-18T03:38:36.525018+0000","last_unstale":"2023-07-18T03:38:36.533773+0000","last_undegraded":"2023-07-18T03:38:36.533773+0000","last_fullsized":"2023-07-18T03:38:36.533773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:27.898176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.534966+0000","last_change":"2023-07-18T03:38:36.531164+0000","last_active":"2023-07-18T03:38:36.534966+0000","last_peered":"2023-07-18T03:38:36.534966+0000","last_clean":"2023-07-18T03:38:36.534966+0000","last_became_active":"2023-07-18T03:38:36.530907+0000","last_became_peered":"2023-07-18T03:38:36.530907+0000","last_unstale":"2023-07-18T03:38:36.534966+0000","last_undegraded":"2023-07-18T03:38:36.534966+0000","last_fullsized":"2023-07-18T03:38:36.534966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:35:45.788407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.554824+0000","last_change":"2023-07-18T03:38:35.529655+0000","last_active":"2023-07-18T03:38:36.554824+0000","last_peered":"2023-07-18T03:38:36.554824+0000","last_clean":"2023-07-18T03:38:36.554824+0000","last_became_active":"2023-07-18T03:38:35.529257+0000","last_became_peered":"2023-07-18T03:38:35.529257+0000","last_unstale":"2023-07-18T03:38:36.554824+0000","last_undegraded":"2023-07-18T03:38:36.554824+0000","last_fullsized":"2023-07-18T03:38:36.554824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:01.259707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.544316+0000","last_change":"2023-07-18T03:38:36.542322+0000","last_active":"2023-07-18T03:38:36.544316+0000","last_peered":"2023-07-18T03:38:36.544316+0000","last_clean":"2023-07-18T03:38:36.544316+0000","last_became_active":"2023-07-18T03:38:36.542038+0000","last_became_peered":"2023-07-18T03:38:36.542038+0000","last_unstale":"2023-07-18T03:38:36.544316+0000","last_undegraded":"2023-07-18T03:38:36.544316+0000","last_fullsized":"2023-07-18T03:38:36.544316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:18.677951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.512583+0000","last_change":"2023-07-18T03:38:35.536467+0000","last_active":"2023-07-18T03:38:36.512583+0000","last_peered":"2023-07-18T03:38:36.512583+0000","last_clean":"2023-07-18T03:38:36.512583+0000","last_became_active":"2023-07-18T03:38:35.536198+0000","last_became_peered":"2023-07-18T03:38:35.536198+0000","last_unstale":"2023-07-18T03:38:36.512583+0000","last_undegraded":"2023-07-18T03:38:36.512583+0000","last_fullsized":"2023-07-18T03:38:36.512583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:44:26.825180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121012+0000","last_change":"2023-07-18T03:38:36.529534+0000","last_active":"2023-07-18T03:38:38.121012+0000","last_peered":"2023-07-18T03:38:38.121012+0000","last_clean":"2023-07-18T03:38:38.121012+0000","last_became_active":"2023-07-18T03:38:36.528958+0000","last_became_peered":"2023-07-18T03:38:36.528958+0000","last_unstale":"2023-07-18T03:38:38.121012+0000","last_undegraded":"2023-07-18T03:38:38.121012+0000","last_fullsized":"2023-07-18T03:38:38.121012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:39.607024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.525271+0000","last_change":"2023-07-18T03:38:35.527369+0000","last_active":"2023-07-18T03:38:36.525271+0000","last_peered":"2023-07-18T03:38:36.525271+0000","last_clean":"2023-07-18T03:38:36.525271+0000","last_became_active":"2023-07-18T03:38:35.527091+0000","last_became_peered":"2023-07-18T03:38:35.527091+0000","last_unstale":"2023-07-18T03:38:36.525271+0000","last_undegraded":"2023-07-18T03:38:36.525271+0000","last_fullsized":"2023-07-18T03:38:36.525271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:37.889642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121460+0000","last_change":"2023-07-18T03:38:36.528845+0000","last_active":"2023-07-18T03:38:38.121460+0000","last_peered":"2023-07-18T03:38:38.121460+0000","last_clean":"2023-07-18T03:38:38.121460+0000","last_became_active":"2023-07-18T03:38:36.528525+0000","last_became_peered":"2023-07-18T03:38:36.528525+0000","last_unstale":"2023-07-18T03:38:38.121460+0000","last_undegraded":"2023-07-18T03:38:38.121460+0000","last_fullsized":"2023-07-18T03:38:38.121460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:16:39.663691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121461+0000","last_change":"2023-07-18T03:38:35.529837+0000","last_active":"2023-07-18T03:38:38.121461+0000","last_peered":"2023-07-18T03:38:38.121461+0000","last_clean":"2023-07-18T03:38:38.121461+0000","last_became_active":"2023-07-18T03:38:35.529486+0000","last_became_peered":"2023-07-18T03:38:35.529486+0000","last_unstale":"2023-07-18T03:38:38.121461+0000","last_undegraded":"2023-07-18T03:38:38.121461+0000","last_fullsized":"2023-07-18T03:38:38.121461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:20:07.344114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.544639+0000","last_change":"2023-07-18T03:38:36.543236+0000","last_active":"2023-07-18T03:38:36.544639+0000","last_peered":"2023-07-18T03:38:36.544639+0000","last_clean":"2023-07-18T03:38:36.544639+0000","last_became_active":"2023-07-18T03:38:36.542960+0000","last_became_peered":"2023-07-18T03:38:36.542960+0000","last_unstale":"2023-07-18T03:38:36.544639+0000","last_undegraded":"2023-07-18T03:38:36.544639+0000","last_fullsized":"2023-07-18T03:38:36.544639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:35:45.788407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.527052+0000","last_change":"2023-07-18T03:38:35.537672+0000","last_active":"2023-07-18T03:38:36.527052+0000","last_peered":"2023-07-18T03:38:36.527052+0000","last_clean":"2023-07-18T03:38:36.527052+0000","last_became_active":"2023-07-18T03:38:35.537362+0000","last_became_peered":"2023-07-18T03:38:35.537362+0000","last_unstale":"2023-07-18T03:38:36.527052+0000","last_undegraded":"2023-07-18T03:38:36.527052+0000","last_fullsized":"2023-07-18T03:38:36.527052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:13:14.473025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.541829+0000","last_change":"2023-07-18T03:38:36.533526+0000","last_active":"2023-07-18T03:38:36.541829+0000","last_peered":"2023-07-18T03:38:36.541829+0000","last_clean":"2023-07-18T03:38:36.541829+0000","last_became_active":"2023-07-18T03:38:36.533278+0000","last_became_peered":"2023-07-18T03:38:36.533278+0000","last_unstale":"2023-07-18T03:38:36.541829+0000","last_undegraded":"2023-07-18T03:38:36.541829+0000","last_fullsized":"2023-07-18T03:38:36.541829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:24:12.848519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516465+0000","last_change":"2023-07-18T03:38:35.536258+0000","last_active":"2023-07-18T03:38:37.516465+0000","last_peered":"2023-07-18T03:38:37.516465+0000","last_clean":"2023-07-18T03:38:37.516465+0000","last_became_active":"2023-07-18T03:38:35.535939+0000","last_became_peered":"2023-07-18T03:38:35.535939+0000","last_unstale":"2023-07-18T03:38:37.516465+0000","last_undegraded":"2023-07-18T03:38:37.516465+0000","last_fullsized":"2023-07-18T03:38:37.516465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:30:23.496663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.540867+0000","last_change":"2023-07-18T03:38:36.533896+0000","last_active":"2023-07-18T03:38:36.540867+0000","last_peered":"2023-07-18T03:38:36.540867+0000","last_clean":"2023-07-18T03:38:36.540867+0000","last_became_active":"2023-07-18T03:38:36.532384+0000","last_became_peered":"2023-07-18T03:38:36.532384+0000","last_unstale":"2023-07-18T03:38:36.540867+0000","last_undegraded":"2023-07-18T03:38:36.540867+0000","last_fullsized":"2023-07-18T03:38:36.540867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:54:42.804941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.558145+0000","last_change":"2023-07-18T03:38:35.536771+0000","last_active":"2023-07-18T03:38:36.558145+0000","last_peered":"2023-07-18T03:38:36.558145+0000","last_clean":"2023-07-18T03:38:36.558145+0000","last_became_active":"2023-07-18T03:38:35.536423+0000","last_became_peered":"2023-07-18T03:38:35.536423+0000","last_unstale":"2023-07-18T03:38:36.558145+0000","last_undegraded":"2023-07-18T03:38:36.558145+0000","last_fullsized":"2023-07-18T03:38:36.558145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:16:38.655263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537811+0000","last_change":"2023-07-18T03:38:36.531396+0000","last_active":"2023-07-18T03:38:36.537811+0000","last_peered":"2023-07-18T03:38:36.537811+0000","last_clean":"2023-07-18T03:38:36.537811+0000","last_became_active":"2023-07-18T03:38:36.531117+0000","last_became_peered":"2023-07-18T03:38:36.531117+0000","last_unstale":"2023-07-18T03:38:36.537811+0000","last_undegraded":"2023-07-18T03:38:36.537811+0000","last_fullsized":"2023-07-18T03:38:36.537811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:24:12.848519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515828+0000","last_change":"2023-07-18T03:38:35.531528+0000","last_active":"2023-07-18T03:38:37.515828+0000","last_peered":"2023-07-18T03:38:37.515828+0000","last_clean":"2023-07-18T03:38:37.515828+0000","last_became_active":"2023-07-18T03:38:35.530889+0000","last_became_peered":"2023-07-18T03:38:35.530889+0000","last_unstale":"2023-07-18T03:38:37.515828+0000","last_undegraded":"2023-07-18T03:38:37.515828+0000","last_fullsized":"2023-07-18T03:38:37.515828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:52:51.944585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.543921+0000","last_change":"2023-07-18T03:38:36.538483+0000","last_active":"2023-07-18T03:38:36.543921+0000","last_peered":"2023-07-18T03:38:36.543921+0000","last_clean":"2023-07-18T03:38:36.543921+0000","last_became_active":"2023-07-18T03:38:36.538133+0000","last_became_peered":"2023-07-18T03:38:36.538133+0000","last_unstale":"2023-07-18T03:38:36.543921+0000","last_undegraded":"2023-07-18T03:38:36.543921+0000","last_fullsized":"2023-07-18T03:38:36.543921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:54:42.804941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120772+0000","last_change":"2023-07-18T03:38:35.518650+0000","last_active":"2023-07-18T03:38:38.120772+0000","last_peered":"2023-07-18T03:38:38.120772+0000","last_clean":"2023-07-18T03:38:38.120772+0000","last_became_active":"2023-07-18T03:38:35.518285+0000","last_became_peered":"2023-07-18T03:38:35.518285+0000","last_unstale":"2023-07-18T03:38:38.120772+0000","last_undegraded":"2023-07-18T03:38:38.120772+0000","last_fullsized":"2023-07-18T03:38:38.120772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:57:59.447632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.542007+0000","last_change":"2023-07-18T03:38:36.538615+0000","last_active":"2023-07-18T03:38:36.542007+0000","last_peered":"2023-07-18T03:38:36.542007+0000","last_clean":"2023-07-18T03:38:36.542007+0000","last_became_active":"2023-07-18T03:38:36.538362+0000","last_became_peered":"2023-07-18T03:38:36.538362+0000","last_unstale":"2023-07-18T03:38:36.542007+0000","last_undegraded":"2023-07-18T03:38:36.542007+0000","last_fullsized":"2023-07-18T03:38:36.542007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:29:26.283162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516038+0000","last_change":"2023-07-18T03:38:35.534762+0000","last_active":"2023-07-18T03:38:37.516038+0000","last_peered":"2023-07-18T03:38:37.516038+0000","last_clean":"2023-07-18T03:38:37.516038+0000","last_became_active":"2023-07-18T03:38:35.534529+0000","last_became_peered":"2023-07-18T03:38:35.534529+0000","last_unstale":"2023-07-18T03:38:37.516038+0000","last_undegraded":"2023-07-18T03:38:37.516038+0000","last_fullsized":"2023-07-18T03:38:37.516038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:27:35.134036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120813+0000","last_change":"2023-07-18T03:38:36.530844+0000","last_active":"2023-07-18T03:38:38.120813+0000","last_peered":"2023-07-18T03:38:38.120813+0000","last_clean":"2023-07-18T03:38:38.120813+0000","last_became_active":"2023-07-18T03:38:36.530545+0000","last_became_peered":"2023-07-18T03:38:36.530545+0000","last_unstale":"2023-07-18T03:38:38.120813+0000","last_undegraded":"2023-07-18T03:38:38.120813+0000","last_fullsized":"2023-07-18T03:38:38.120813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:39:26.376467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120972+0000","last_change":"2023-07-18T03:38:35.530210+0000","last_active":"2023-07-18T03:38:38.120972+0000","last_peered":"2023-07-18T03:38:38.120972+0000","last_clean":"2023-07-18T03:38:38.120972+0000","last_became_active":"2023-07-18T03:38:35.529906+0000","last_became_peered":"2023-07-18T03:38:35.529906+0000","last_unstale":"2023-07-18T03:38:38.120972+0000","last_undegraded":"2023-07-18T03:38:38.120972+0000","last_fullsized":"2023-07-18T03:38:38.120972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:59:17.569992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.540738+0000","last_change":"2023-07-18T03:38:36.534252+0000","last_active":"2023-07-18T03:38:36.540738+0000","last_peered":"2023-07-18T03:38:36.540738+0000","last_clean":"2023-07-18T03:38:36.540738+0000","last_became_active":"2023-07-18T03:38:36.533986+0000","last_became_peered":"2023-07-18T03:38:36.533986+0000","last_unstale":"2023-07-18T03:38:36.540738+0000","last_undegraded":"2023-07-18T03:38:36.540738+0000","last_fullsized":"2023-07-18T03:38:36.540738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:13:15.481453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516720+0000","last_change":"2023-07-18T03:38:35.530100+0000","last_active":"2023-07-18T03:38:37.516720+0000","last_peered":"2023-07-18T03:38:37.516720+0000","last_clean":"2023-07-18T03:38:37.516720+0000","last_became_active":"2023-07-18T03:38:35.529084+0000","last_became_peered":"2023-07-18T03:38:35.529084+0000","last_unstale":"2023-07-18T03:38:37.516720+0000","last_undegraded":"2023-07-18T03:38:37.516720+0000","last_fullsized":"2023-07-18T03:38:37.516720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:25.804673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.540670+0000","last_change":"2023-07-18T03:38:36.531903+0000","last_active":"2023-07-18T03:38:36.540670+0000","last_peered":"2023-07-18T03:38:36.540670+0000","last_clean":"2023-07-18T03:38:36.540670+0000","last_became_active":"2023-07-18T03:38:36.531503+0000","last_became_peered":"2023-07-18T03:38:36.531503+0000","last_unstale":"2023-07-18T03:38:36.540670+0000","last_undegraded":"2023-07-18T03:38:36.540670+0000","last_fullsized":"2023-07-18T03:38:36.540670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:13:03.155792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516435+0000","last_change":"2023-07-18T03:38:35.530991+0000","last_active":"2023-07-18T03:38:37.516435+0000","last_peered":"2023-07-18T03:38:37.516435+0000","last_clean":"2023-07-18T03:38:37.516435+0000","last_became_active":"2023-07-18T03:38:35.530676+0000","last_became_peered":"2023-07-18T03:38:35.530676+0000","last_unstale":"2023-07-18T03:38:37.516435+0000","last_undegraded":"2023-07-18T03:38:37.516435+0000","last_fullsized":"2023-07-18T03:38:37.516435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:43:42.440181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.528815+0000","last_change":"2023-07-18T03:38:36.524535+0000","last_active":"2023-07-18T03:38:36.528815+0000","last_peered":"2023-07-18T03:38:36.528815+0000","last_clean":"2023-07-18T03:38:36.528815+0000","last_became_active":"2023-07-18T03:38:36.524235+0000","last_became_peered":"2023-07-18T03:38:36.524235+0000","last_unstale":"2023-07-18T03:38:36.528815+0000","last_undegraded":"2023-07-18T03:38:36.528815+0000","last_fullsized":"2023-07-18T03:38:36.528815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:13:03.155792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516290+0000","last_change":"2023-07-18T03:38:35.532333+0000","last_active":"2023-07-18T03:38:37.516290+0000","last_peered":"2023-07-18T03:38:37.516290+0000","last_clean":"2023-07-18T03:38:37.516290+0000","last_became_active":"2023-07-18T03:38:35.531899+0000","last_became_peered":"2023-07-18T03:38:35.531899+0000","last_unstale":"2023-07-18T03:38:37.516290+0000","last_undegraded":"2023-07-18T03:38:37.516290+0000","last_fullsized":"2023-07-18T03:38:37.516290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:27:42.625518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121237+0000","last_change":"2023-07-18T03:38:36.540597+0000","last_active":"2023-07-18T03:38:38.121237+0000","last_peered":"2023-07-18T03:38:38.121237+0000","last_clean":"2023-07-18T03:38:38.121237+0000","last_became_active":"2023-07-18T03:38:36.540313+0000","last_became_peered":"2023-07-18T03:38:36.540313+0000","last_unstale":"2023-07-18T03:38:38.121237+0000","last_undegraded":"2023-07-18T03:38:38.121237+0000","last_fullsized":"2023-07-18T03:38:38.121237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:02.268135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.522410+0000","last_change":"2023-07-18T03:38:35.536145+0000","last_active":"2023-07-18T03:38:36.522410+0000","last_peered":"2023-07-18T03:38:36.522410+0000","last_clean":"2023-07-18T03:38:36.522410+0000","last_became_active":"2023-07-18T03:38:35.535827+0000","last_became_peered":"2023-07-18T03:38:35.535827+0000","last_unstale":"2023-07-18T03:38:36.522410+0000","last_undegraded":"2023-07-18T03:38:36.522410+0000","last_fullsized":"2023-07-18T03:38:36.522410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:54:41.796513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120807+0000","last_change":"2023-07-18T03:38:36.526183+0000","last_active":"2023-07-18T03:38:38.120807+0000","last_peered":"2023-07-18T03:38:38.120807+0000","last_clean":"2023-07-18T03:38:38.120807+0000","last_became_active":"2023-07-18T03:38:36.525853+0000","last_became_peered":"2023-07-18T03:38:36.525853+0000","last_unstale":"2023-07-18T03:38:38.120807+0000","last_undegraded":"2023-07-18T03:38:38.120807+0000","last_fullsized":"2023-07-18T03:38:38.120807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:51:42.726329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.550286+0000","last_change":"2023-07-18T03:38:35.534166+0000","last_active":"2023-07-18T03:38:36.550286+0000","last_peered":"2023-07-18T03:38:36.550286+0000","last_clean":"2023-07-18T03:38:36.550286+0000","last_became_active":"2023-07-18T03:38:35.533900+0000","last_became_peered":"2023-07-18T03:38:35.533900+0000","last_unstale":"2023-07-18T03:38:36.550286+0000","last_undegraded":"2023-07-18T03:38:36.550286+0000","last_fullsized":"2023-07-18T03:38:36.550286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:23.497046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.538475+0000","last_change":"2023-07-18T03:38:36.535669+0000","last_active":"2023-07-18T03:38:36.538475+0000","last_peered":"2023-07-18T03:38:36.538475+0000","last_clean":"2023-07-18T03:38:36.538475+0000","last_became_active":"2023-07-18T03:38:36.535338+0000","last_became_peered":"2023-07-18T03:38:36.535338+0000","last_unstale":"2023-07-18T03:38:36.538475+0000","last_undegraded":"2023-07-18T03:38:36.538475+0000","last_fullsized":"2023-07-18T03:38:36.538475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:35:45.788407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121279+0000","last_change":"2023-07-18T03:38:35.530541+0000","last_active":"2023-07-18T03:38:38.121279+0000","last_peered":"2023-07-18T03:38:38.121279+0000","last_clean":"2023-07-18T03:38:38.121279+0000","last_became_active":"2023-07-18T03:38:35.530220+0000","last_became_peered":"2023-07-18T03:38:35.530220+0000","last_unstale":"2023-07-18T03:38:38.121279+0000","last_undegraded":"2023-07-18T03:38:38.121279+0000","last_fullsized":"2023-07-18T03:38:38.121279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:53:15.570874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.527879+0000","last_change":"2023-07-18T03:38:36.523648+0000","last_active":"2023-07-18T03:38:36.527879+0000","last_peered":"2023-07-18T03:38:36.527879+0000","last_clean":"2023-07-18T03:38:36.527879+0000","last_became_active":"2023-07-18T03:38:36.523346+0000","last_became_peered":"2023-07-18T03:38:36.523346+0000","last_unstale":"2023-07-18T03:38:36.527879+0000","last_undegraded":"2023-07-18T03:38:36.527879+0000","last_fullsized":"2023-07-18T03:38:36.527879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:44:27.833608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.516626+0000","last_change":"2023-07-18T03:38:35.534663+0000","last_active":"2023-07-18T03:38:36.516626+0000","last_peered":"2023-07-18T03:38:36.516626+0000","last_clean":"2023-07-18T03:38:36.516626+0000","last_became_active":"2023-07-18T03:38:35.534363+0000","last_became_peered":"2023-07-18T03:38:35.534363+0000","last_unstale":"2023-07-18T03:38:36.516626+0000","last_undegraded":"2023-07-18T03:38:36.516626+0000","last_fullsized":"2023-07-18T03:38:36.516626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:50:45.063449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537860+0000","last_change":"2023-07-18T03:38:36.531919+0000","last_active":"2023-07-18T03:38:36.537860+0000","last_peered":"2023-07-18T03:38:36.537860+0000","last_clean":"2023-07-18T03:38:36.537860+0000","last_became_active":"2023-07-18T03:38:36.531729+0000","last_became_peered":"2023-07-18T03:38:36.531729+0000","last_unstale":"2023-07-18T03:38:36.537860+0000","last_undegraded":"2023-07-18T03:38:36.537860+0000","last_fullsized":"2023-07-18T03:38:36.537860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:18.677951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.525093+0000","last_change":"2023-07-18T03:38:35.538915+0000","last_active":"2023-07-18T03:38:36.525093+0000","last_peered":"2023-07-18T03:38:36.525093+0000","last_clean":"2023-07-18T03:38:36.525093+0000","last_became_active":"2023-07-18T03:38:35.538539+0000","last_became_peered":"2023-07-18T03:38:35.538539+0000","last_unstale":"2023-07-18T03:38:36.525093+0000","last_undegraded":"2023-07-18T03:38:36.525093+0000","last_fullsized":"2023-07-18T03:38:36.525093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:59:17.569992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537140+0000","last_change":"2023-07-18T03:38:36.534839+0000","last_active":"2023-07-18T03:38:36.537140+0000","last_peered":"2023-07-18T03:38:36.537140+0000","last_clean":"2023-07-18T03:38:36.537140+0000","last_became_active":"2023-07-18T03:38:36.532743+0000","last_became_peered":"2023-07-18T03:38:36.532743+0000","last_unstale":"2023-07-18T03:38:36.537140+0000","last_undegraded":"2023-07-18T03:38:36.537140+0000","last_fullsized":"2023-07-18T03:38:36.537140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:24:12.848519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.553664+0000","last_change":"2023-07-18T03:38:35.538083+0000","last_active":"2023-07-18T03:38:36.553664+0000","last_peered":"2023-07-18T03:38:36.553664+0000","last_clean":"2023-07-18T03:38:36.553664+0000","last_became_active":"2023-07-18T03:38:35.537692+0000","last_became_peered":"2023-07-18T03:38:35.537692+0000","last_unstale":"2023-07-18T03:38:36.553664+0000","last_undegraded":"2023-07-18T03:38:36.553664+0000","last_fullsized":"2023-07-18T03:38:36.553664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:53:15.570874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.516656+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:38:36.516656+0000","last_peered":"2023-07-18T03:38:36.516656+0000","last_clean":"2023-07-18T03:38:36.516656+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:38:36.516656+0000","last_undegraded":"2023-07-18T03:38:36.516656+0000","last_fullsized":"2023-07-18T03:38:36.516656+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:17.327799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.536380+0000","last_change":"2023-07-18T03:38:36.531962+0000","last_active":"2023-07-18T03:38:36.536380+0000","last_peered":"2023-07-18T03:38:36.536380+0000","last_clean":"2023-07-18T03:38:36.536380+0000","last_became_active":"2023-07-18T03:38:36.531710+0000","last_became_peered":"2023-07-18T03:38:36.531710+0000","last_unstale":"2023-07-18T03:38:36.536380+0000","last_undegraded":"2023-07-18T03:38:36.536380+0000","last_fullsized":"2023-07-18T03:38:36.536380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:29:26.283162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.526245+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:38:36.526245+0000","last_peered":"2023-07-18T03:38:36.526245+0000","last_clean":"2023-07-18T03:38:36.526245+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:38:36.526245+0000","last_undegraded":"2023-07-18T03:38:36.526245+0000","last_fullsized":"2023-07-18T03:38:36.526245+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:50:38.894203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121101+0000","last_change":"2023-07-18T03:38:35.522198+0000","last_active":"2023-07-18T03:38:38.121101+0000","last_peered":"2023-07-18T03:38:38.121101+0000","last_clean":"2023-07-18T03:38:38.121101+0000","last_became_active":"2023-07-18T03:38:35.521915+0000","last_became_peered":"2023-07-18T03:38:35.521915+0000","last_unstale":"2023-07-18T03:38:38.121101+0000","last_undegraded":"2023-07-18T03:38:38.121101+0000","last_fullsized":"2023-07-18T03:38:38.121101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:06:13.388319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.544904+0000","last_change":"2023-07-18T03:38:36.537103+0000","last_active":"2023-07-18T03:38:36.544904+0000","last_peered":"2023-07-18T03:38:36.544904+0000","last_clean":"2023-07-18T03:38:36.544904+0000","last_became_active":"2023-07-18T03:38:36.534926+0000","last_became_peered":"2023-07-18T03:38:36.534926+0000","last_unstale":"2023-07-18T03:38:36.544904+0000","last_undegraded":"2023-07-18T03:38:36.544904+0000","last_fullsized":"2023-07-18T03:38:36.544904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:01:03.276648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.512308+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:38:36.512308+0000","last_peered":"2023-07-18T03:38:36.512308+0000","last_clean":"2023-07-18T03:38:36.512308+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:38:36.512308+0000","last_undegraded":"2023-07-18T03:38:36.512308+0000","last_fullsized":"2023-07-18T03:38:36.512308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:07:26.667202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515826+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:38:37.515826+0000","last_peered":"2023-07-18T03:38:37.515826+0000","last_clean":"2023-07-18T03:38:37.515826+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:38:37.515826+0000","last_undegraded":"2023-07-18T03:38:37.515826+0000","last_fullsized":"2023-07-18T03:38:37.515826+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:38.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120579+0000","last_change":"2023-07-18T03:38:35.534781+0000","last_active":"2023-07-18T03:38:38.120579+0000","last_peered":"2023-07-18T03:38:38.120579+0000","last_clean":"2023-07-18T03:38:38.120579+0000","last_became_active":"2023-07-18T03:38:35.534485+0000","last_became_peered":"2023-07-18T03:38:35.534485+0000","last_unstale":"2023-07-18T03:38:38.120579+0000","last_undegraded":"2023-07-18T03:38:38.120579+0000","last_fullsized":"2023-07-18T03:38:38.120579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:42:36.215812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120857+0000","last_change":"2023-07-18T03:38:36.527606+0000","last_active":"2023-07-18T03:38:38.120857+0000","last_peered":"2023-07-18T03:38:38.120857+0000","last_clean":"2023-07-18T03:38:38.120857+0000","last_became_active":"2023-07-18T03:38:36.527267+0000","last_became_peered":"2023-07-18T03:38:36.527267+0000","last_unstale":"2023-07-18T03:38:38.120857+0000","last_undegraded":"2023-07-18T03:38:38.120857+0000","last_fullsized":"2023-07-18T03:38:38.120857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:32:24.221027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515516+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:38:37.515516+0000","last_peered":"2023-07-18T03:38:37.515516+0000","last_clean":"2023-07-18T03:38:37.515516+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:38:37.515516+0000","last_undegraded":"2023-07-18T03:38:37.515516+0000","last_fullsized":"2023-07-18T03:38:37.515516+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:02.380287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.512496+0000","last_change":"2023-07-18T03:38:35.531556+0000","last_active":"2023-07-18T03:38:36.512496+0000","last_peered":"2023-07-18T03:38:36.512496+0000","last_clean":"2023-07-18T03:38:36.512496+0000","last_became_active":"2023-07-18T03:38:35.531268+0000","last_became_peered":"2023-07-18T03:38:35.531268+0000","last_unstale":"2023-07-18T03:38:36.512496+0000","last_undegraded":"2023-07-18T03:38:36.512496+0000","last_fullsized":"2023-07-18T03:38:36.512496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:48:25.450768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537739+0000","last_change":"2023-07-18T03:38:36.536571+0000","last_active":"2023-07-18T03:38:36.537739+0000","last_peered":"2023-07-18T03:38:36.537739+0000","last_clean":"2023-07-18T03:38:36.537739+0000","last_became_active":"2023-07-18T03:38:36.536187+0000","last_became_peered":"2023-07-18T03:38:36.536187+0000","last_unstale":"2023-07-18T03:38:36.537739+0000","last_undegraded":"2023-07-18T03:38:36.537739+0000","last_fullsized":"2023-07-18T03:38:36.537739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:39.335445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515534+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:38:37.515534+0000","last_peered":"2023-07-18T03:38:37.515534+0000","last_clean":"2023-07-18T03:38:37.515534+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:38:37.515534+0000","last_undegraded":"2023-07-18T03:38:37.515534+0000","last_fullsized":"2023-07-18T03:38:37.515534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:38:34.441575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.526334+0000","last_change":"2023-07-18T03:38:35.536386+0000","last_active":"2023-07-18T03:38:36.526334+0000","last_peered":"2023-07-18T03:38:36.526334+0000","last_clean":"2023-07-18T03:38:36.526334+0000","last_became_active":"2023-07-18T03:38:35.536066+0000","last_became_peered":"2023-07-18T03:38:35.536066+0000","last_unstale":"2023-07-18T03:38:36.526334+0000","last_undegraded":"2023-07-18T03:38:36.526334+0000","last_fullsized":"2023-07-18T03:38:36.526334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:01.259707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121258+0000","last_change":"2023-07-18T03:38:36.528088+0000","last_active":"2023-07-18T03:38:38.121258+0000","last_peered":"2023-07-18T03:38:38.121258+0000","last_clean":"2023-07-18T03:38:38.121258+0000","last_became_active":"2023-07-18T03:38:36.527760+0000","last_became_peered":"2023-07-18T03:38:36.527760+0000","last_unstale":"2023-07-18T03:38:38.121258+0000","last_undegraded":"2023-07-18T03:38:38.121258+0000","last_fullsized":"2023-07-18T03:38:38.121258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:38.898070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.525054+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:38:36.525054+0000","last_peered":"2023-07-18T03:38:36.525054+0000","last_clean":"2023-07-18T03:38:36.525054+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:38:36.525054+0000","last_undegraded":"2023-07-18T03:38:36.525054+0000","last_fullsized":"2023-07-18T03:38:36.525054+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:20.720501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.516127+0000","last_change":"2023-07-18T03:38:35.526465+0000","last_active":"2023-07-18T03:38:36.516127+0000","last_peered":"2023-07-18T03:38:36.516127+0000","last_clean":"2023-07-18T03:38:36.516127+0000","last_became_active":"2023-07-18T03:38:35.525759+0000","last_became_peered":"2023-07-18T03:38:35.525759+0000","last_unstale":"2023-07-18T03:38:36.516127+0000","last_undegraded":"2023-07-18T03:38:36.516127+0000","last_fullsized":"2023-07-18T03:38:36.516127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:59:17.569992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.532144+0000","last_change":"2023-07-18T03:38:36.523849+0000","last_active":"2023-07-18T03:38:36.532144+0000","last_peered":"2023-07-18T03:38:36.532144+0000","last_clean":"2023-07-18T03:38:36.532144+0000","last_became_active":"2023-07-18T03:38:36.523555+0000","last_became_peered":"2023-07-18T03:38:36.523555+0000","last_unstale":"2023-07-18T03:38:36.532144+0000","last_undegraded":"2023-07-18T03:38:36.532144+0000","last_fullsized":"2023-07-18T03:38:36.532144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:29:26.283162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121324+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:38:38.121324+0000","last_peered":"2023-07-18T03:38:38.121324+0000","last_clean":"2023-07-18T03:38:38.121324+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:38:38.121324+0000","last_undegraded":"2023-07-18T03:38:38.121324+0000","last_fullsized":"2023-07-18T03:38:38.121324+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:50:38.894203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.512887+0000","last_change":"2023-07-18T03:38:35.530710+0000","last_active":"2023-07-18T03:38:36.512887+0000","last_peered":"2023-07-18T03:38:36.512887+0000","last_clean":"2023-07-18T03:38:36.512887+0000","last_became_active":"2023-07-18T03:38:35.530463+0000","last_became_peered":"2023-07-18T03:38:35.530463+0000","last_unstale":"2023-07-18T03:38:36.512887+0000","last_undegraded":"2023-07-18T03:38:36.512887+0000","last_fullsized":"2023-07-18T03:38:36.512887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:35:44.779979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121163+0000","last_change":"2023-07-18T03:38:36.535754+0000","last_active":"2023-07-18T03:38:38.121163+0000","last_peered":"2023-07-18T03:38:38.121163+0000","last_clean":"2023-07-18T03:38:38.121163+0000","last_became_active":"2023-07-18T03:38:36.535290+0000","last_became_peered":"2023-07-18T03:38:36.535290+0000","last_unstale":"2023-07-18T03:38:38.121163+0000","last_undegraded":"2023-07-18T03:38:38.121163+0000","last_fullsized":"2023-07-18T03:38:38.121163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:03.011261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121317+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:38:38.121317+0000","last_peered":"2023-07-18T03:38:38.121317+0000","last_clean":"2023-07-18T03:38:38.121317+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:38:38.121317+0000","last_undegraded":"2023-07-18T03:38:38.121317+0000","last_fullsized":"2023-07-18T03:38:38.121317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:42:57.376881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.525694+0000","last_change":"2023-07-18T03:38:35.536982+0000","last_active":"2023-07-18T03:38:36.525694+0000","last_peered":"2023-07-18T03:38:36.525694+0000","last_clean":"2023-07-18T03:38:36.525694+0000","last_became_active":"2023-07-18T03:38:35.536739+0000","last_became_peered":"2023-07-18T03:38:35.536739+0000","last_unstale":"2023-07-18T03:38:36.525694+0000","last_undegraded":"2023-07-18T03:38:36.525694+0000","last_fullsized":"2023-07-18T03:38:36.525694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:01:02.268220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.540887+0000","last_change":"2023-07-18T03:38:36.532972+0000","last_active":"2023-07-18T03:38:36.540887+0000","last_peered":"2023-07-18T03:38:36.540887+0000","last_clean":"2023-07-18T03:38:36.540887+0000","last_became_active":"2023-07-18T03:38:36.530512+0000","last_became_peered":"2023-07-18T03:38:36.530512+0000","last_unstale":"2023-07-18T03:38:36.540887+0000","last_undegraded":"2023-07-18T03:38:36.540887+0000","last_fullsized":"2023-07-18T03:38:36.540887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:01:03.276648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121293+0000","last_change":"2023-07-18T03:38:35.525179+0000","last_active":"2023-07-18T03:38:38.121293+0000","last_peered":"2023-07-18T03:38:38.121293+0000","last_clean":"2023-07-18T03:38:38.121293+0000","last_became_active":"2023-07-18T03:38:35.524912+0000","last_became_peered":"2023-07-18T03:38:35.524912+0000","last_unstale":"2023-07-18T03:38:38.121293+0000","last_undegraded":"2023-07-18T03:38:38.121293+0000","last_fullsized":"2023-07-18T03:38:38.121293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:23:28.704920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.122118+0000","last_change":"2023-07-18T03:38:36.529814+0000","last_active":"2023-07-18T03:38:38.122118+0000","last_peered":"2023-07-18T03:38:38.122118+0000","last_clean":"2023-07-18T03:38:38.122118+0000","last_became_active":"2023-07-18T03:38:36.529554+0000","last_became_peered":"2023-07-18T03:38:36.529554+0000","last_unstale":"2023-07-18T03:38:38.122118+0000","last_undegraded":"2023-07-18T03:38:38.122118+0000","last_fullsized":"2023-07-18T03:38:38.122118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:30:55.302127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.551181+0000","last_change":"2023-07-18T03:38:35.529892+0000","last_active":"2023-07-18T03:38:36.551181+0000","last_peered":"2023-07-18T03:38:36.551181+0000","last_clean":"2023-07-18T03:38:36.551181+0000","last_became_active":"2023-07-18T03:38:35.529614+0000","last_became_peered":"2023-07-18T03:38:35.529614+0000","last_unstale":"2023-07-18T03:38:36.551181+0000","last_undegraded":"2023-07-18T03:38:36.551181+0000","last_fullsized":"2023-07-18T03:38:36.551181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:17.669523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537984+0000","last_change":"2023-07-18T03:38:36.535652+0000","last_active":"2023-07-18T03:38:36.537984+0000","last_peered":"2023-07-18T03:38:36.537984+0000","last_clean":"2023-07-18T03:38:36.537984+0000","last_became_active":"2023-07-18T03:38:36.535378+0000","last_became_peered":"2023-07-18T03:38:36.535378+0000","last_unstale":"2023-07-18T03:38:36.537984+0000","last_undegraded":"2023-07-18T03:38:36.537984+0000","last_fullsized":"2023-07-18T03:38:36.537984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:39.335445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.550038+0000","last_change":"2023-07-18T03:38:35.534949+0000","last_active":"2023-07-18T03:38:36.550038+0000","last_peered":"2023-07-18T03:38:36.550038+0000","last_clean":"2023-07-18T03:38:36.550038+0000","last_became_active":"2023-07-18T03:38:35.534685+0000","last_became_peered":"2023-07-18T03:38:35.534685+0000","last_unstale":"2023-07-18T03:38:36.550038+0000","last_undegraded":"2023-07-18T03:38:36.550038+0000","last_fullsized":"2023-07-18T03:38:36.550038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:24:11.840091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.538972+0000","last_change":"2023-07-18T03:38:36.535757+0000","last_active":"2023-07-18T03:38:36.538972+0000","last_peered":"2023-07-18T03:38:36.538972+0000","last_clean":"2023-07-18T03:38:36.538972+0000","last_became_active":"2023-07-18T03:38:36.535452+0000","last_became_peered":"2023-07-18T03:38:36.535452+0000","last_unstale":"2023-07-18T03:38:36.538972+0000","last_undegraded":"2023-07-18T03:38:36.538972+0000","last_fullsized":"2023-07-18T03:38:36.538972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:13:15.481453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.549343+0000","last_change":"2023-07-18T03:38:35.532954+0000","last_active":"2023-07-18T03:38:36.549343+0000","last_peered":"2023-07-18T03:38:36.549343+0000","last_clean":"2023-07-18T03:38:36.549343+0000","last_became_active":"2023-07-18T03:38:35.532617+0000","last_became_peered":"2023-07-18T03:38:35.532617+0000","last_unstale":"2023-07-18T03:38:36.549343+0000","last_undegraded":"2023-07-18T03:38:36.549343+0000","last_fullsized":"2023-07-18T03:38:36.549343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:03.546127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.543826+0000","last_change":"2023-07-18T03:38:36.540001+0000","last_active":"2023-07-18T03:38:36.543826+0000","last_peered":"2023-07-18T03:38:36.543826+0000","last_clean":"2023-07-18T03:38:36.543826+0000","last_became_active":"2023-07-18T03:38:36.539743+0000","last_became_peered":"2023-07-18T03:38:36.539743+0000","last_unstale":"2023-07-18T03:38:36.543826+0000","last_undegraded":"2023-07-18T03:38:36.543826+0000","last_fullsized":"2023-07-18T03:38:36.543826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:39.335445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121427+0000","last_change":"2023-07-18T03:38:35.532342+0000","last_active":"2023-07-18T03:38:38.121427+0000","last_peered":"2023-07-18T03:38:38.121427+0000","last_clean":"2023-07-18T03:38:38.121427+0000","last_became_active":"2023-07-18T03:38:35.530138+0000","last_became_peered":"2023-07-18T03:38:35.530138+0000","last_unstale":"2023-07-18T03:38:38.121427+0000","last_undegraded":"2023-07-18T03:38:38.121427+0000","last_fullsized":"2023-07-18T03:38:38.121427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:59:26.646208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.539939+0000","last_change":"2023-07-18T03:38:36.531861+0000","last_active":"2023-07-18T03:38:36.539939+0000","last_peered":"2023-07-18T03:38:36.539939+0000","last_clean":"2023-07-18T03:38:36.539939+0000","last_became_active":"2023-07-18T03:38:36.531563+0000","last_became_peered":"2023-07-18T03:38:36.531563+0000","last_unstale":"2023-07-18T03:38:36.539939+0000","last_undegraded":"2023-07-18T03:38:36.539939+0000","last_fullsized":"2023-07-18T03:38:36.539939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:48:26.459196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121209+0000","last_change":"2023-07-18T03:38:36.537688+0000","last_active":"2023-07-18T03:38:38.121209+0000","last_peered":"2023-07-18T03:38:38.121209+0000","last_clean":"2023-07-18T03:38:38.121209+0000","last_became_active":"2023-07-18T03:38:36.537415+0000","last_became_peered":"2023-07-18T03:38:36.537415+0000","last_unstale":"2023-07-18T03:38:38.121209+0000","last_undegraded":"2023-07-18T03:38:38.121209+0000","last_fullsized":"2023-07-18T03:38:38.121209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:13:15.481453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.544369+0000","last_change":"2023-07-18T03:38:36.542378+0000","last_active":"2023-07-18T03:38:36.544369+0000","last_peered":"2023-07-18T03:38:36.544369+0000","last_clean":"2023-07-18T03:38:36.544369+0000","last_became_active":"2023-07-18T03:38:36.542101+0000","last_became_peered":"2023-07-18T03:38:36.542101+0000","last_unstale":"2023-07-18T03:38:36.544369+0000","last_undegraded":"2023-07-18T03:38:36.544369+0000","last_fullsized":"2023-07-18T03:38:36.544369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:27.898176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537836+0000","last_change":"2023-07-18T03:38:36.531395+0000","last_active":"2023-07-18T03:38:36.537836+0000","last_peered":"2023-07-18T03:38:36.537836+0000","last_clean":"2023-07-18T03:38:36.537836+0000","last_became_active":"2023-07-18T03:38:36.531071+0000","last_became_peered":"2023-07-18T03:38:36.531071+0000","last_unstale":"2023-07-18T03:38:36.537836+0000","last_undegraded":"2023-07-18T03:38:36.537836+0000","last_fullsized":"2023-07-18T03:38:36.537836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:29:26.283162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.541527+0000","last_change":"2023-07-18T03:38:36.534035+0000","last_active":"2023-07-18T03:38:36.541527+0000","last_peered":"2023-07-18T03:38:36.541527+0000","last_clean":"2023-07-18T03:38:36.541527+0000","last_became_active":"2023-07-18T03:38:36.533755+0000","last_became_peered":"2023-07-18T03:38:36.533755+0000","last_unstale":"2023-07-18T03:38:36.541527+0000","last_undegraded":"2023-07-18T03:38:36.541527+0000","last_fullsized":"2023-07-18T03:38:36.541527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:01:03.276648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.534239+0000","last_change":"2023-07-18T03:38:36.524624+0000","last_active":"2023-07-18T03:38:36.534239+0000","last_peered":"2023-07-18T03:38:36.534239+0000","last_clean":"2023-07-18T03:38:36.534239+0000","last_became_active":"2023-07-18T03:38:36.524106+0000","last_became_peered":"2023-07-18T03:38:36.524106+0000","last_unstale":"2023-07-18T03:38:36.534239+0000","last_undegraded":"2023-07-18T03:38:36.534239+0000","last_fullsized":"2023-07-18T03:38:36.534239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:13:03.155792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":843776,"data_stored":673348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":208896,"data_stored":74547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":212992,"data_stored":74637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":802816,"data_stored":664915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":76167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":75129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31264,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340576,"statfs":{"total":96636764160,"available":96604749824,"internally_reserved":0,"allocated":221184,"data_stored":81200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:38:40.983 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:38:41.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.222+0000 7f5f63dff700 1 Processor -- start 2023-07-18T03:38:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.223+0000 7f5f63dff700 1 -- start start 2023-07-18T03:38:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f63dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5f5c15c950 0x7f5f5c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f63dff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5f5c0b1e00 0x7f5f5c0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f63dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f5c0b2350 0x7f5f5c0d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f63dff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f5c0d2ce0 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f63dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f5c0b3350 con 0x7f5f5c15c950 2023-07-18T03:38:41.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f63dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0b3630 con 0x7f5f5c0b2350 2023-07-18T03:38:41.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f6139a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5f5c0b1e00 0x7f5f5c0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f6139a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5f5c0b1e00 0x7f5f5c0d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40230/0 (socket says 172.21.15.49:40230) 2023-07-18T03:38:41.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f6139a700 1 -- 172.21.15.49:0/3745781972 learned_addr learned my addr 172.21.15.49:0/3745781972 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:41.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f61b9b700 1 --2- 172.21.15.49:0/3745781972 >> v2:172.21.15.107:3300/0 conn(0x7f5f5c15c950 0x7f5f5c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f6239c700 1 --2- 172.21.15.49:0/3745781972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f5c0b2350 0x7f5f5c0d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f6139a700 1 -- 172.21.15.49:0/3745781972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f5c0b2350 msgr2=0x7f5f5c0d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:41.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.224+0000 7f5f6139a700 1 --2- 172.21.15.49:0/3745781972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f5c0b2350 0x7f5f5c0d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.225+0000 7f5f6139a700 1 -- 172.21.15.49:0/3745781972 >> v2:172.21.15.107:3300/0 conn(0x7f5f5c15c950 msgr2=0x7f5f5c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:41.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.225+0000 7f5f6139a700 1 --2- 172.21.15.49:0/3745781972 >> v2:172.21.15.107:3300/0 conn(0x7f5f5c15c950 0x7f5f5c0b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.225+0000 7f5f6139a700 1 -- 172.21.15.49:0/3745781972 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c0b3910 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.225+0000 7f5f6139a700 1 --2- 172.21.15.49:0/3745781972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5c0b1e00 0x7f5f5c0d0b10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c08d170 tx=0x7f5f4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f5d8050e9bbf32 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.225+0000 7f5f60b99700 1 -- 172.21.15.49:0/3745781972 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00b870 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.225+0000 7f5f60b99700 1 -- 172.21.15.49:0/3745781972 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c00b9d0 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.225+0000 7f5f60b99700 1 -- 172.21.15.49:0/3745781972 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00bd60 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.226+0000 7f5f63dff700 1 -- 172.21.15.49:0/3745781972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5c0b1e00 msgr2=0x7f5f5c0d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.226+0000 7f5f63dff700 1 --2- 172.21.15.49:0/3745781972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5c0b1e00 0x7f5f5c0d0b10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c08d170 tx=0x7f5f4c009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.226+0000 7f5f63dff700 1 -- 172.21.15.49:0/3745781972 shutdown_connections 2023-07-18T03:38:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.226+0000 7f5f63dff700 1 --2- 172.21.15.49:0/3745781972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f5c0b2350 0x7f5f5c0d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.226+0000 7f5f63dff700 1 --2- 172.21.15.49:0/3745781972 >> v2:172.21.15.107:3300/0 conn(0x7f5f5c15c950 0x7f5f5c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.226+0000 7f5f63dff700 1 --2- 172.21.15.49:0/3745781972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5c0b1e00 0x7f5f5c0d0b10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.226+0000 7f5f63dff700 1 -- 172.21.15.49:0/3745781972 >> 172.21.15.49:0/3745781972 conn(0x7f5f5c09c1d0 msgr2=0x7f5f5c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.226+0000 7f5f63dff700 1 -- 172.21.15.49:0/3745781972 shutdown_connections 2023-07-18T03:38:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.226+0000 7f5f63dff700 1 -- 172.21.15.49:0/3745781972 wait complete. 2023-07-18T03:38:41.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.228+0000 7f5f63dff700 1 Processor -- start 2023-07-18T03:38:41.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.228+0000 7f5f63dff700 1 -- start start 2023-07-18T03:38:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.229+0000 7f5f63dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5c0b1e00 0x7f5f5c0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.229+0000 7f5f63dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f5c0b2350 0x7f5f5c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.229+0000 7f5f63dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c15c950 0x7f5f5c0cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.229+0000 7f5f63dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0ce480 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.229+0000 7f5f63dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c157230 con 0x7f5f5c15c950 2023-07-18T03:38:41.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.229+0000 7f5f63dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c157b90 con 0x7f5f5c0b2350 2023-07-18T03:38:41.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.229+0000 7f5f61b9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5c0b1e00 0x7f5f5c0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.229+0000 7f5f61b9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5c0b1e00 0x7f5f5c0cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40246/0 (socket says 172.21.15.49:40246) 2023-07-18T03:38:41.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.229+0000 7f5f61b9b700 1 -- 172.21.15.49:0/1579950484 learned_addr learned my addr 172.21.15.49:0/1579950484 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:41.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.229+0000 7f5f6139a700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f5c0b2350 0x7f5f5c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.229+0000 7f5f6239c700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c15c950 0x7f5f5c0cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.230+0000 7f5f61b9b700 1 -- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f5c0b2350 msgr2=0x7f5f5c156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.230+0000 7f5f61b9b700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f5c0b2350 0x7f5f5c156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.230+0000 7f5f61b9b700 1 -- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c15c950 msgr2=0x7f5f5c0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.230+0000 7f5f61b9b700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c15c950 0x7f5f5c0cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.230+0000 7f5f61b9b700 1 -- 172.21.15.49:0/1579950484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c009550 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.230+0000 7f5f61b9b700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5c0b1e00 0x7f5f5c0cd570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800d7d0 tx=0x7f5f5800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:41.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.230+0000 7f5f527fc700 1 -- 172.21.15.49:0/1579950484 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58017070 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.230+0000 7f5f527fc700 1 -- 172.21.15.49:0/1579950484 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f58005a20 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.230+0000 7f5f63dff700 1 -- 172.21.15.49:0/1579950484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5c0cdbe0 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.230+0000 7f5f527fc700 1 -- 172.21.15.49:0/1579950484 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58005bd0 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.230+0000 7f5f63dff700 1 -- 172.21.15.49:0/1579950484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c0ce1a0 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.231+0000 7f5f63dff700 1 -- 172.21.15.49:0/1579950484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5c04e280 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.232+0000 7f5f527fc700 1 -- 172.21.15.49:0/1579950484 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f58005d30 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.233+0000 7f5f527fc700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5f4006ace0 0x7f5f4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.233+0000 7f5f527fc700 1 -- 172.21.15.49:0/1579950484 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5f58088eb0 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.234+0000 7f5f6139a700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5f4006ace0 0x7f5f4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.234+0000 7f5f6139a700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5f4006ace0 0x7f5f4006d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c000c00 tx=0x7f5f4c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:41.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.237+0000 7f5f527fc700 1 -- 172.21.15.49:0/1579950484 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5f58053f50 con 0x7f5f5c0b1e00 2023-07-18T03:38:41.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.440+0000 7f5f63dff700 1 -- 172.21.15.49:0/1579950484 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f5c0b3ed0 con 0x7f5f4006ace0 2023-07-18T03:38:41.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.449+0000 7f5f527fc700 1 -- 172.21.15.49:0/1579950484 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286890 (secure 0 0 0) 0x7f5f4c00bb40 con 0x7f5f4006ace0 2023-07-18T03:38:41.447 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:41.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.453+0000 7f5f63dff700 1 -- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5f4006ace0 msgr2=0x7f5f4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.453+0000 7f5f63dff700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5f4006ace0 0x7f5f4006d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c000c00 tx=0x7f5f4c008030 comp rx=0 tx=0).stop 2023-07-18T03:38:41.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.453+0000 7f5f63dff700 1 -- 172.21.15.49:0/1579950484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5c0b1e00 msgr2=0x7f5f5c0cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.453+0000 7f5f63dff700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5c0b1e00 0x7f5f5c0cd570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800d7d0 tx=0x7f5f5800dae0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.453+0000 7f5f63dff700 1 -- 172.21.15.49:0/1579950484 shutdown_connections 2023-07-18T03:38:41.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.453+0000 7f5f63dff700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c15c950 0x7f5f5c0cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.453+0000 7f5f63dff700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5f4006ace0 0x7f5f4006d190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.453+0000 7f5f63dff700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f5c0b1e00 0x7f5f5c0cd570 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.453+0000 7f5f63dff700 1 --2- 172.21.15.49:0/1579950484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f5c0b2350 0x7f5f5c156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.453+0000 7f5f63dff700 1 -- 172.21.15.49:0/1579950484 >> 172.21.15.49:0/1579950484 conn(0x7f5f5c09c1d0 msgr2=0x7f5f5c0b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:41.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.453+0000 7f5f63dff700 1 -- 172.21.15.49:0/1579950484 shutdown_connections 2023-07-18T03:38:41.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.453+0000 7f5f63dff700 1 -- 172.21.15.49:0/1579950484 wait complete. 2023-07-18T03:38:41.451 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:38:41.467 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-07-18T03:38:40.097282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230784,"kb_used_data":2880,"kb_used_omap":0,"kb_used_meta":227840,"kb_avail":754743936,"statfs":{"total":773094113280,"available":772857790464,"internally_reserved":0,"allocated":2949120,"data_stored":1795142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":233308160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.598559"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.122105+0000","last_change":"2023-07-18T03:38:36.538478+0000","last_active":"2023-07-18T03:38:38.122105+0000","last_peered":"2023-07-18T03:38:38.122105+0000","last_clean":"2023-07-18T03:38:38.122105+0000","last_became_active":"2023-07-18T03:38:36.538174+0000","last_became_peered":"2023-07-18T03:38:36.538174+0000","last_unstale":"2023-07-18T03:38:38.122105+0000","last_undegraded":"2023-07-18T03:38:38.122105+0000","last_fullsized":"2023-07-18T03:38:38.122105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:10:09.118957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.542016+0000","last_change":"2023-07-18T03:38:36.538553+0000","last_active":"2023-07-18T03:38:36.542016+0000","last_peered":"2023-07-18T03:38:36.542016+0000","last_clean":"2023-07-18T03:38:36.542016+0000","last_became_active":"2023-07-18T03:38:36.538257+0000","last_became_peered":"2023-07-18T03:38:36.538257+0000","last_unstale":"2023-07-18T03:38:36.542016+0000","last_undegraded":"2023-07-18T03:38:36.542016+0000","last_fullsized":"2023-07-18T03:38:36.542016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:32:37.988899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537770+0000","last_change":"2023-07-18T03:38:36.526262+0000","last_active":"2023-07-18T03:38:36.537770+0000","last_peered":"2023-07-18T03:38:36.537770+0000","last_clean":"2023-07-18T03:38:36.537770+0000","last_became_active":"2023-07-18T03:38:36.525920+0000","last_became_peered":"2023-07-18T03:38:36.525920+0000","last_unstale":"2023-07-18T03:38:36.537770+0000","last_undegraded":"2023-07-18T03:38:36.537770+0000","last_fullsized":"2023-07-18T03:38:36.537770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:38.898070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121429+0000","last_change":"2023-07-18T03:38:36.535992+0000","last_active":"2023-07-18T03:38:38.121429+0000","last_peered":"2023-07-18T03:38:38.121429+0000","last_clean":"2023-07-18T03:38:38.121429+0000","last_became_active":"2023-07-18T03:38:36.535587+0000","last_became_peered":"2023-07-18T03:38:36.535587+0000","last_unstale":"2023-07-18T03:38:38.121429+0000","last_undegraded":"2023-07-18T03:38:38.121429+0000","last_fullsized":"2023-07-18T03:38:38.121429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:24.505474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121029+0000","last_change":"2023-07-18T03:38:36.535524+0000","last_active":"2023-07-18T03:38:38.121029+0000","last_peered":"2023-07-18T03:38:38.121029+0000","last_clean":"2023-07-18T03:38:38.121029+0000","last_became_active":"2023-07-18T03:38:36.534975+0000","last_became_peered":"2023-07-18T03:38:36.534975+0000","last_unstale":"2023-07-18T03:38:38.121029+0000","last_undegraded":"2023-07-18T03:38:38.121029+0000","last_fullsized":"2023-07-18T03:38:38.121029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:37:55.979750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.534617+0000","last_change":"2023-07-18T03:38:36.526306+0000","last_active":"2023-07-18T03:38:36.534617+0000","last_peered":"2023-07-18T03:38:36.534617+0000","last_clean":"2023-07-18T03:38:36.534617+0000","last_became_active":"2023-07-18T03:38:36.525982+0000","last_became_peered":"2023-07-18T03:38:36.525982+0000","last_unstale":"2023-07-18T03:38:36.534617+0000","last_undegraded":"2023-07-18T03:38:36.534617+0000","last_fullsized":"2023-07-18T03:38:36.534617+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:32:37.988899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516000+0000","last_change":"2023-07-18T03:38:36.527124+0000","last_active":"2023-07-18T03:38:37.516000+0000","last_peered":"2023-07-18T03:38:37.516000+0000","last_clean":"2023-07-18T03:38:37.516000+0000","last_became_active":"2023-07-18T03:38:36.526827+0000","last_became_peered":"2023-07-18T03:38:36.526827+0000","last_unstale":"2023-07-18T03:38:37.516000+0000","last_undegraded":"2023-07-18T03:38:37.516000+0000","last_fullsized":"2023-07-18T03:38:37.516000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:21:51.299094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537642+0000","last_change":"2023-07-18T03:38:36.529550+0000","last_active":"2023-07-18T03:38:36.537642+0000","last_peered":"2023-07-18T03:38:36.537642+0000","last_clean":"2023-07-18T03:38:36.537642+0000","last_became_active":"2023-07-18T03:38:36.529261+0000","last_became_peered":"2023-07-18T03:38:36.529261+0000","last_unstale":"2023-07-18T03:38:36.537642+0000","last_undegraded":"2023-07-18T03:38:36.537642+0000","last_fullsized":"2023-07-18T03:38:36.537642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:44:27.833608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121235+0000","last_change":"2023-07-18T03:38:36.534909+0000","last_active":"2023-07-18T03:38:38.121235+0000","last_peered":"2023-07-18T03:38:38.121235+0000","last_clean":"2023-07-18T03:38:38.121235+0000","last_became_active":"2023-07-18T03:38:36.534593+0000","last_became_peered":"2023-07-18T03:38:36.534593+0000","last_unstale":"2023-07-18T03:38:38.121235+0000","last_undegraded":"2023-07-18T03:38:38.121235+0000","last_fullsized":"2023-07-18T03:38:38.121235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:45:51.736824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.541969+0000","last_change":"2023-07-18T03:38:36.536965+0000","last_active":"2023-07-18T03:38:36.541969+0000","last_peered":"2023-07-18T03:38:36.541969+0000","last_clean":"2023-07-18T03:38:36.541969+0000","last_became_active":"2023-07-18T03:38:36.536665+0000","last_became_peered":"2023-07-18T03:38:36.536665+0000","last_unstale":"2023-07-18T03:38:36.541969+0000","last_undegraded":"2023-07-18T03:38:36.541969+0000","last_fullsized":"2023-07-18T03:38:36.541969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:59:18.578420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537794+0000","last_change":"2023-07-18T03:38:36.526499+0000","last_active":"2023-07-18T03:38:36.537794+0000","last_peered":"2023-07-18T03:38:36.537794+0000","last_clean":"2023-07-18T03:38:36.537794+0000","last_became_active":"2023-07-18T03:38:36.525871+0000","last_became_peered":"2023-07-18T03:38:36.525871+0000","last_unstale":"2023-07-18T03:38:36.537794+0000","last_undegraded":"2023-07-18T03:38:36.537794+0000","last_fullsized":"2023-07-18T03:38:36.537794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:04.554555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121914+0000","last_change":"2023-07-18T03:38:36.524528+0000","last_active":"2023-07-18T03:38:38.121914+0000","last_peered":"2023-07-18T03:38:38.121914+0000","last_clean":"2023-07-18T03:38:38.121914+0000","last_became_active":"2023-07-18T03:38:36.523892+0000","last_became_peered":"2023-07-18T03:38:36.523892+0000","last_unstale":"2023-07-18T03:38:38.121914+0000","last_undegraded":"2023-07-18T03:38:38.121914+0000","last_fullsized":"2023-07-18T03:38:38.121914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:01:02.396334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121759+0000","last_change":"2023-07-18T03:38:36.533293+0000","last_active":"2023-07-18T03:38:38.121759+0000","last_peered":"2023-07-18T03:38:38.121759+0000","last_clean":"2023-07-18T03:38:38.121759+0000","last_became_active":"2023-07-18T03:38:36.532953+0000","last_became_peered":"2023-07-18T03:38:36.532953+0000","last_unstale":"2023-07-18T03:38:38.121759+0000","last_undegraded":"2023-07-18T03:38:38.121759+0000","last_fullsized":"2023-07-18T03:38:38.121759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:35:14.137991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.542036+0000","last_change":"2023-07-18T03:38:36.536500+0000","last_active":"2023-07-18T03:38:36.542036+0000","last_peered":"2023-07-18T03:38:36.542036+0000","last_clean":"2023-07-18T03:38:36.542036+0000","last_became_active":"2023-07-18T03:38:36.536283+0000","last_became_peered":"2023-07-18T03:38:36.536283+0000","last_unstale":"2023-07-18T03:38:36.542036+0000","last_undegraded":"2023-07-18T03:38:36.542036+0000","last_fullsized":"2023-07-18T03:38:36.542036+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:39.335445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516206+0000","last_change":"2023-07-18T03:38:36.530577+0000","last_active":"2023-07-18T03:38:37.516206+0000","last_peered":"2023-07-18T03:38:37.516206+0000","last_clean":"2023-07-18T03:38:37.516206+0000","last_became_active":"2023-07-18T03:38:36.530268+0000","last_became_peered":"2023-07-18T03:38:36.530268+0000","last_unstale":"2023-07-18T03:38:37.516206+0000","last_undegraded":"2023-07-18T03:38:37.516206+0000","last_fullsized":"2023-07-18T03:38:37.516206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:33:14.215460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537775+0000","last_change":"2023-07-18T03:38:36.526520+0000","last_active":"2023-07-18T03:38:36.537775+0000","last_peered":"2023-07-18T03:38:36.537775+0000","last_clean":"2023-07-18T03:38:36.537775+0000","last_became_active":"2023-07-18T03:38:36.526165+0000","last_became_peered":"2023-07-18T03:38:36.526165+0000","last_unstale":"2023-07-18T03:38:36.537775+0000","last_undegraded":"2023-07-18T03:38:36.537775+0000","last_fullsized":"2023-07-18T03:38:36.537775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:53:16.579302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121313+0000","last_change":"2023-07-18T03:38:36.531837+0000","last_active":"2023-07-18T03:38:38.121313+0000","last_peered":"2023-07-18T03:38:38.121313+0000","last_clean":"2023-07-18T03:38:38.121313+0000","last_became_active":"2023-07-18T03:38:36.531588+0000","last_became_peered":"2023-07-18T03:38:36.531588+0000","last_unstale":"2023-07-18T03:38:38.121313+0000","last_undegraded":"2023-07-18T03:38:38.121313+0000","last_fullsized":"2023-07-18T03:38:38.121313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:57:34.438689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.539935+0000","last_change":"2023-07-18T03:38:36.536129+0000","last_active":"2023-07-18T03:38:36.539935+0000","last_peered":"2023-07-18T03:38:36.539935+0000","last_clean":"2023-07-18T03:38:36.539935+0000","last_became_active":"2023-07-18T03:38:36.535846+0000","last_became_peered":"2023-07-18T03:38:36.535846+0000","last_unstale":"2023-07-18T03:38:36.539935+0000","last_undegraded":"2023-07-18T03:38:36.539935+0000","last_fullsized":"2023-07-18T03:38:36.539935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:27.898176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.122134+0000","last_change":"2023-07-18T03:38:36.529212+0000","last_active":"2023-07-18T03:38:38.122134+0000","last_peered":"2023-07-18T03:38:38.122134+0000","last_clean":"2023-07-18T03:38:38.122134+0000","last_became_active":"2023-07-18T03:38:36.528956+0000","last_became_peered":"2023-07-18T03:38:36.528956+0000","last_unstale":"2023-07-18T03:38:38.122134+0000","last_undegraded":"2023-07-18T03:38:38.122134+0000","last_fullsized":"2023-07-18T03:38:38.122134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:23:08.908073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537935+0000","last_change":"2023-07-18T03:38:36.532777+0000","last_active":"2023-07-18T03:38:36.537935+0000","last_peered":"2023-07-18T03:38:36.537935+0000","last_clean":"2023-07-18T03:38:36.537935+0000","last_became_active":"2023-07-18T03:38:36.532519+0000","last_became_peered":"2023-07-18T03:38:36.532519+0000","last_unstale":"2023-07-18T03:38:36.537935+0000","last_undegraded":"2023-07-18T03:38:36.537935+0000","last_fullsized":"2023-07-18T03:38:36.537935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:54:42.804941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515979+0000","last_change":"2023-07-18T03:38:36.541130+0000","last_active":"2023-07-18T03:38:37.515979+0000","last_peered":"2023-07-18T03:38:37.515979+0000","last_clean":"2023-07-18T03:38:37.515979+0000","last_became_active":"2023-07-18T03:38:36.535483+0000","last_became_peered":"2023-07-18T03:38:36.535483+0000","last_unstale":"2023-07-18T03:38:37.515979+0000","last_undegraded":"2023-07-18T03:38:37.515979+0000","last_fullsized":"2023-07-18T03:38:37.515979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:46.558463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120549+0000","last_change":"2023-07-18T03:38:36.536797+0000","last_active":"2023-07-18T03:38:38.120549+0000","last_peered":"2023-07-18T03:38:38.120549+0000","last_clean":"2023-07-18T03:38:38.120549+0000","last_became_active":"2023-07-18T03:38:36.535389+0000","last_became_peered":"2023-07-18T03:38:36.535389+0000","last_unstale":"2023-07-18T03:38:38.120549+0000","last_undegraded":"2023-07-18T03:38:38.120549+0000","last_fullsized":"2023-07-18T03:38:38.120549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:22:44.555224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.541809+0000","last_change":"2023-07-18T03:38:36.533007+0000","last_active":"2023-07-18T03:38:36.541809+0000","last_peered":"2023-07-18T03:38:36.541809+0000","last_clean":"2023-07-18T03:38:36.541809+0000","last_became_active":"2023-07-18T03:38:36.532714+0000","last_became_peered":"2023-07-18T03:38:36.532714+0000","last_unstale":"2023-07-18T03:38:36.541809+0000","last_undegraded":"2023-07-18T03:38:36.541809+0000","last_fullsized":"2023-07-18T03:38:36.541809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:32:37.988899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515725+0000","last_change":"2023-07-18T03:38:36.541077+0000","last_active":"2023-07-18T03:38:37.515725+0000","last_peered":"2023-07-18T03:38:37.515725+0000","last_clean":"2023-07-18T03:38:37.515725+0000","last_became_active":"2023-07-18T03:38:36.533768+0000","last_became_peered":"2023-07-18T03:38:36.533768+0000","last_unstale":"2023-07-18T03:38:37.515725+0000","last_undegraded":"2023-07-18T03:38:37.515725+0000","last_fullsized":"2023-07-18T03:38:37.515725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:42.081234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.536458+0000","last_change":"2023-07-18T03:38:36.529999+0000","last_active":"2023-07-18T03:38:36.536458+0000","last_peered":"2023-07-18T03:38:36.536458+0000","last_clean":"2023-07-18T03:38:36.536458+0000","last_became_active":"2023-07-18T03:38:36.529489+0000","last_became_peered":"2023-07-18T03:38:36.529489+0000","last_unstale":"2023-07-18T03:38:36.536458+0000","last_undegraded":"2023-07-18T03:38:36.536458+0000","last_fullsized":"2023-07-18T03:38:36.536458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:18.677951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.533773+0000","last_change":"2023-07-18T03:38:36.525279+0000","last_active":"2023-07-18T03:38:36.533773+0000","last_peered":"2023-07-18T03:38:36.533773+0000","last_clean":"2023-07-18T03:38:36.533773+0000","last_became_active":"2023-07-18T03:38:36.525018+0000","last_became_peered":"2023-07-18T03:38:36.525018+0000","last_unstale":"2023-07-18T03:38:36.533773+0000","last_undegraded":"2023-07-18T03:38:36.533773+0000","last_fullsized":"2023-07-18T03:38:36.533773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:27.898176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.534966+0000","last_change":"2023-07-18T03:38:36.531164+0000","last_active":"2023-07-18T03:38:36.534966+0000","last_peered":"2023-07-18T03:38:36.534966+0000","last_clean":"2023-07-18T03:38:36.534966+0000","last_became_active":"2023-07-18T03:38:36.530907+0000","last_became_peered":"2023-07-18T03:38:36.530907+0000","last_unstale":"2023-07-18T03:38:36.534966+0000","last_undegraded":"2023-07-18T03:38:36.534966+0000","last_fullsized":"2023-07-18T03:38:36.534966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:35:45.788407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.554824+0000","last_change":"2023-07-18T03:38:35.529655+0000","last_active":"2023-07-18T03:38:36.554824+0000","last_peered":"2023-07-18T03:38:36.554824+0000","last_clean":"2023-07-18T03:38:36.554824+0000","last_became_active":"2023-07-18T03:38:35.529257+0000","last_became_peered":"2023-07-18T03:38:35.529257+0000","last_unstale":"2023-07-18T03:38:36.554824+0000","last_undegraded":"2023-07-18T03:38:36.554824+0000","last_fullsized":"2023-07-18T03:38:36.554824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:01.259707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.544316+0000","last_change":"2023-07-18T03:38:36.542322+0000","last_active":"2023-07-18T03:38:36.544316+0000","last_peered":"2023-07-18T03:38:36.544316+0000","last_clean":"2023-07-18T03:38:36.544316+0000","last_became_active":"2023-07-18T03:38:36.542038+0000","last_became_peered":"2023-07-18T03:38:36.542038+0000","last_unstale":"2023-07-18T03:38:36.544316+0000","last_undegraded":"2023-07-18T03:38:36.544316+0000","last_fullsized":"2023-07-18T03:38:36.544316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:18.677951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.512583+0000","last_change":"2023-07-18T03:38:35.536467+0000","last_active":"2023-07-18T03:38:36.512583+0000","last_peered":"2023-07-18T03:38:36.512583+0000","last_clean":"2023-07-18T03:38:36.512583+0000","last_became_active":"2023-07-18T03:38:35.536198+0000","last_became_peered":"2023-07-18T03:38:35.536198+0000","last_unstale":"2023-07-18T03:38:36.512583+0000","last_undegraded":"2023-07-18T03:38:36.512583+0000","last_fullsized":"2023-07-18T03:38:36.512583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:44:26.825180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121012+0000","last_change":"2023-07-18T03:38:36.529534+0000","last_active":"2023-07-18T03:38:38.121012+0000","last_peered":"2023-07-18T03:38:38.121012+0000","last_clean":"2023-07-18T03:38:38.121012+0000","last_became_active":"2023-07-18T03:38:36.528958+0000","last_became_peered":"2023-07-18T03:38:36.528958+0000","last_unstale":"2023-07-18T03:38:38.121012+0000","last_undegraded":"2023-07-18T03:38:38.121012+0000","last_fullsized":"2023-07-18T03:38:38.121012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:39.607024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.525271+0000","last_change":"2023-07-18T03:38:35.527369+0000","last_active":"2023-07-18T03:38:36.525271+0000","last_peered":"2023-07-18T03:38:36.525271+0000","last_clean":"2023-07-18T03:38:36.525271+0000","last_became_active":"2023-07-18T03:38:35.527091+0000","last_became_peered":"2023-07-18T03:38:35.527091+0000","last_unstale":"2023-07-18T03:38:36.525271+0000","last_undegraded":"2023-07-18T03:38:36.525271+0000","last_fullsized":"2023-07-18T03:38:36.525271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:37.889642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121460+0000","last_change":"2023-07-18T03:38:36.528845+0000","last_active":"2023-07-18T03:38:38.121460+0000","last_peered":"2023-07-18T03:38:38.121460+0000","last_clean":"2023-07-18T03:38:38.121460+0000","last_became_active":"2023-07-18T03:38:36.528525+0000","last_became_peered":"2023-07-18T03:38:36.528525+0000","last_unstale":"2023-07-18T03:38:38.121460+0000","last_undegraded":"2023-07-18T03:38:38.121460+0000","last_fullsized":"2023-07-18T03:38:38.121460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:16:39.663691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121461+0000","last_change":"2023-07-18T03:38:35.529837+0000","last_active":"2023-07-18T03:38:38.121461+0000","last_peered":"2023-07-18T03:38:38.121461+0000","last_clean":"2023-07-18T03:38:38.121461+0000","last_became_active":"2023-07-18T03:38:35.529486+0000","last_became_peered":"2023-07-18T03:38:35.529486+0000","last_unstale":"2023-07-18T03:38:38.121461+0000","last_undegraded":"2023-07-18T03:38:38.121461+0000","last_fullsized":"2023-07-18T03:38:38.121461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:20:07.344114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.544639+0000","last_change":"2023-07-18T03:38:36.543236+0000","last_active":"2023-07-18T03:38:36.544639+0000","last_peered":"2023-07-18T03:38:36.544639+0000","last_clean":"2023-07-18T03:38:36.544639+0000","last_became_active":"2023-07-18T03:38:36.542960+0000","last_became_peered":"2023-07-18T03:38:36.542960+0000","last_unstale":"2023-07-18T03:38:36.544639+0000","last_undegraded":"2023-07-18T03:38:36.544639+0000","last_fullsized":"2023-07-18T03:38:36.544639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:35:45.788407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.527052+0000","last_change":"2023-07-18T03:38:35.537672+0000","last_active":"2023-07-18T03:38:36.527052+0000","last_peered":"2023-07-18T03:38:36.527052+0000","last_clean":"2023-07-18T03:38:36.527052+0000","last_became_active":"2023-07-18T03:38:35.537362+0000","last_became_peered":"2023-07-18T03:38:35.537362+0000","last_unstale":"2023-07-18T03:38:36.527052+0000","last_undegraded":"2023-07-18T03:38:36.527052+0000","last_fullsized":"2023-07-18T03:38:36.527052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:13:14.473025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.541829+0000","last_change":"2023-07-18T03:38:36.533526+0000","last_active":"2023-07-18T03:38:36.541829+0000","last_peered":"2023-07-18T03:38:36.541829+0000","last_clean":"2023-07-18T03:38:36.541829+0000","last_became_active":"2023-07-18T03:38:36.533278+0000","last_became_peered":"2023-07-18T03:38:36.533278+0000","last_unstale":"2023-07-18T03:38:36.541829+0000","last_undegraded":"2023-07-18T03:38:36.541829+0000","last_fullsized":"2023-07-18T03:38:36.541829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:24:12.848519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516465+0000","last_change":"2023-07-18T03:38:35.536258+0000","last_active":"2023-07-18T03:38:37.516465+0000","last_peered":"2023-07-18T03:38:37.516465+0000","last_clean":"2023-07-18T03:38:37.516465+0000","last_became_active":"2023-07-18T03:38:35.535939+0000","last_became_peered":"2023-07-18T03:38:35.535939+0000","last_unstale":"2023-07-18T03:38:37.516465+0000","last_undegraded":"2023-07-18T03:38:37.516465+0000","last_fullsized":"2023-07-18T03:38:37.516465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:30:23.496663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.540867+0000","last_change":"2023-07-18T03:38:36.533896+0000","last_active":"2023-07-18T03:38:36.540867+0000","last_peered":"2023-07-18T03:38:36.540867+0000","last_clean":"2023-07-18T03:38:36.540867+0000","last_became_active":"2023-07-18T03:38:36.532384+0000","last_became_peered":"2023-07-18T03:38:36.532384+0000","last_unstale":"2023-07-18T03:38:36.540867+0000","last_undegraded":"2023-07-18T03:38:36.540867+0000","last_fullsized":"2023-07-18T03:38:36.540867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:54:42.804941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.558145+0000","last_change":"2023-07-18T03:38:35.536771+0000","last_active":"2023-07-18T03:38:36.558145+0000","last_peered":"2023-07-18T03:38:36.558145+0000","last_clean":"2023-07-18T03:38:36.558145+0000","last_became_active":"2023-07-18T03:38:35.536423+0000","last_became_peered":"2023-07-18T03:38:35.536423+0000","last_unstale":"2023-07-18T03:38:36.558145+0000","last_undegraded":"2023-07-18T03:38:36.558145+0000","last_fullsized":"2023-07-18T03:38:36.558145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:16:38.655263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537811+0000","last_change":"2023-07-18T03:38:36.531396+0000","last_active":"2023-07-18T03:38:36.537811+0000","last_peered":"2023-07-18T03:38:36.537811+0000","last_clean":"2023-07-18T03:38:36.537811+0000","last_became_active":"2023-07-18T03:38:36.531117+0000","last_became_peered":"2023-07-18T03:38:36.531117+0000","last_unstale":"2023-07-18T03:38:36.537811+0000","last_undegraded":"2023-07-18T03:38:36.537811+0000","last_fullsized":"2023-07-18T03:38:36.537811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:24:12.848519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515828+0000","last_change":"2023-07-18T03:38:35.531528+0000","last_active":"2023-07-18T03:38:37.515828+0000","last_peered":"2023-07-18T03:38:37.515828+0000","last_clean":"2023-07-18T03:38:37.515828+0000","last_became_active":"2023-07-18T03:38:35.530889+0000","last_became_peered":"2023-07-18T03:38:35.530889+0000","last_unstale":"2023-07-18T03:38:37.515828+0000","last_undegraded":"2023-07-18T03:38:37.515828+0000","last_fullsized":"2023-07-18T03:38:37.515828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:52:51.944585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.543921+0000","last_change":"2023-07-18T03:38:36.538483+0000","last_active":"2023-07-18T03:38:36.543921+0000","last_peered":"2023-07-18T03:38:36.543921+0000","last_clean":"2023-07-18T03:38:36.543921+0000","last_became_active":"2023-07-18T03:38:36.538133+0000","last_became_peered":"2023-07-18T03:38:36.538133+0000","last_unstale":"2023-07-18T03:38:36.543921+0000","last_undegraded":"2023-07-18T03:38:36.543921+0000","last_fullsized":"2023-07-18T03:38:36.543921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:54:42.804941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120772+0000","last_change":"2023-07-18T03:38:35.518650+0000","last_active":"2023-07-18T03:38:38.120772+0000","last_peered":"2023-07-18T03:38:38.120772+0000","last_clean":"2023-07-18T03:38:38.120772+0000","last_became_active":"2023-07-18T03:38:35.518285+0000","last_became_peered":"2023-07-18T03:38:35.518285+0000","last_unstale":"2023-07-18T03:38:38.120772+0000","last_undegraded":"2023-07-18T03:38:38.120772+0000","last_fullsized":"2023-07-18T03:38:38.120772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:57:59.447632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.542007+0000","last_change":"2023-07-18T03:38:36.538615+0000","last_active":"2023-07-18T03:38:36.542007+0000","last_peered":"2023-07-18T03:38:36.542007+0000","last_clean":"2023-07-18T03:38:36.542007+0000","last_became_active":"2023-07-18T03:38:36.538362+0000","last_became_peered":"2023-07-18T03:38:36.538362+0000","last_unstale":"2023-07-18T03:38:36.542007+0000","last_undegraded":"2023-07-18T03:38:36.542007+0000","last_fullsized":"2023-07-18T03:38:36.542007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:29:26.283162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516038+0000","last_change":"2023-07-18T03:38:35.534762+0000","last_active":"2023-07-18T03:38:37.516038+0000","last_peered":"2023-07-18T03:38:37.516038+0000","last_clean":"2023-07-18T03:38:37.516038+0000","last_became_active":"2023-07-18T03:38:35.534529+0000","last_became_peered":"2023-07-18T03:38:35.534529+0000","last_unstale":"2023-07-18T03:38:37.516038+0000","last_undegraded":"2023-07-18T03:38:37.516038+0000","last_fullsized":"2023-07-18T03:38:37.516038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:27:35.134036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120813+0000","last_change":"2023-07-18T03:38:36.530844+0000","last_active":"2023-07-18T03:38:38.120813+0000","last_peered":"2023-07-18T03:38:38.120813+0000","last_clean":"2023-07-18T03:38:38.120813+0000","last_became_active":"2023-07-18T03:38:36.530545+0000","last_became_peered":"2023-07-18T03:38:36.530545+0000","last_unstale":"2023-07-18T03:38:38.120813+0000","last_undegraded":"2023-07-18T03:38:38.120813+0000","last_fullsized":"2023-07-18T03:38:38.120813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:39:26.376467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120972+0000","last_change":"2023-07-18T03:38:35.530210+0000","last_active":"2023-07-18T03:38:38.120972+0000","last_peered":"2023-07-18T03:38:38.120972+0000","last_clean":"2023-07-18T03:38:38.120972+0000","last_became_active":"2023-07-18T03:38:35.529906+0000","last_became_peered":"2023-07-18T03:38:35.529906+0000","last_unstale":"2023-07-18T03:38:38.120972+0000","last_undegraded":"2023-07-18T03:38:38.120972+0000","last_fullsized":"2023-07-18T03:38:38.120972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:59:17.569992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.540738+0000","last_change":"2023-07-18T03:38:36.534252+0000","last_active":"2023-07-18T03:38:36.540738+0000","last_peered":"2023-07-18T03:38:36.540738+0000","last_clean":"2023-07-18T03:38:36.540738+0000","last_became_active":"2023-07-18T03:38:36.533986+0000","last_became_peered":"2023-07-18T03:38:36.533986+0000","last_unstale":"2023-07-18T03:38:36.540738+0000","last_undegraded":"2023-07-18T03:38:36.540738+0000","last_fullsized":"2023-07-18T03:38:36.540738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:13:15.481453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516720+0000","last_change":"2023-07-18T03:38:35.530100+0000","last_active":"2023-07-18T03:38:37.516720+0000","last_peered":"2023-07-18T03:38:37.516720+0000","last_clean":"2023-07-18T03:38:37.516720+0000","last_became_active":"2023-07-18T03:38:35.529084+0000","last_became_peered":"2023-07-18T03:38:35.529084+0000","last_unstale":"2023-07-18T03:38:37.516720+0000","last_undegraded":"2023-07-18T03:38:37.516720+0000","last_fullsized":"2023-07-18T03:38:37.516720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:25.804673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.540670+0000","last_change":"2023-07-18T03:38:36.531903+0000","last_active":"2023-07-18T03:38:36.540670+0000","last_peered":"2023-07-18T03:38:36.540670+0000","last_clean":"2023-07-18T03:38:36.540670+0000","last_became_active":"2023-07-18T03:38:36.531503+0000","last_became_peered":"2023-07-18T03:38:36.531503+0000","last_unstale":"2023-07-18T03:38:36.540670+0000","last_undegraded":"2023-07-18T03:38:36.540670+0000","last_fullsized":"2023-07-18T03:38:36.540670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:13:03.155792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516435+0000","last_change":"2023-07-18T03:38:35.530991+0000","last_active":"2023-07-18T03:38:37.516435+0000","last_peered":"2023-07-18T03:38:37.516435+0000","last_clean":"2023-07-18T03:38:37.516435+0000","last_became_active":"2023-07-18T03:38:35.530676+0000","last_became_peered":"2023-07-18T03:38:35.530676+0000","last_unstale":"2023-07-18T03:38:37.516435+0000","last_undegraded":"2023-07-18T03:38:37.516435+0000","last_fullsized":"2023-07-18T03:38:37.516435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:43:42.440181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.528815+0000","last_change":"2023-07-18T03:38:36.524535+0000","last_active":"2023-07-18T03:38:36.528815+0000","last_peered":"2023-07-18T03:38:36.528815+0000","last_clean":"2023-07-18T03:38:36.528815+0000","last_became_active":"2023-07-18T03:38:36.524235+0000","last_became_peered":"2023-07-18T03:38:36.524235+0000","last_unstale":"2023-07-18T03:38:36.528815+0000","last_undegraded":"2023-07-18T03:38:36.528815+0000","last_fullsized":"2023-07-18T03:38:36.528815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:13:03.155792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516290+0000","last_change":"2023-07-18T03:38:35.532333+0000","last_active":"2023-07-18T03:38:37.516290+0000","last_peered":"2023-07-18T03:38:37.516290+0000","last_clean":"2023-07-18T03:38:37.516290+0000","last_became_active":"2023-07-18T03:38:35.531899+0000","last_became_peered":"2023-07-18T03:38:35.531899+0000","last_unstale":"2023-07-18T03:38:37.516290+0000","last_undegraded":"2023-07-18T03:38:37.516290+0000","last_fullsized":"2023-07-18T03:38:37.516290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:27:42.625518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121237+0000","last_change":"2023-07-18T03:38:36.540597+0000","last_active":"2023-07-18T03:38:38.121237+0000","last_peered":"2023-07-18T03:38:38.121237+0000","last_clean":"2023-07-18T03:38:38.121237+0000","last_became_active":"2023-07-18T03:38:36.540313+0000","last_became_peered":"2023-07-18T03:38:36.540313+0000","last_unstale":"2023-07-18T03:38:38.121237+0000","last_undegraded":"2023-07-18T03:38:38.121237+0000","last_fullsized":"2023-07-18T03:38:38.121237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:02.268135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.522410+0000","last_change":"2023-07-18T03:38:35.536145+0000","last_active":"2023-07-18T03:38:36.522410+0000","last_peered":"2023-07-18T03:38:36.522410+0000","last_clean":"2023-07-18T03:38:36.522410+0000","last_became_active":"2023-07-18T03:38:35.535827+0000","last_became_peered":"2023-07-18T03:38:35.535827+0000","last_unstale":"2023-07-18T03:38:36.522410+0000","last_undegraded":"2023-07-18T03:38:36.522410+0000","last_fullsized":"2023-07-18T03:38:36.522410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:54:41.796513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120807+0000","last_change":"2023-07-18T03:38:36.526183+0000","last_active":"2023-07-18T03:38:38.120807+0000","last_peered":"2023-07-18T03:38:38.120807+0000","last_clean":"2023-07-18T03:38:38.120807+0000","last_became_active":"2023-07-18T03:38:36.525853+0000","last_became_peered":"2023-07-18T03:38:36.525853+0000","last_unstale":"2023-07-18T03:38:38.120807+0000","last_undegraded":"2023-07-18T03:38:38.120807+0000","last_fullsized":"2023-07-18T03:38:38.120807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:51:42.726329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.550286+0000","last_change":"2023-07-18T03:38:35.534166+0000","last_active":"2023-07-18T03:38:36.550286+0000","last_peered":"2023-07-18T03:38:36.550286+0000","last_clean":"2023-07-18T03:38:36.550286+0000","last_became_active":"2023-07-18T03:38:35.533900+0000","last_became_peered":"2023-07-18T03:38:35.533900+0000","last_unstale":"2023-07-18T03:38:36.550286+0000","last_undegraded":"2023-07-18T03:38:36.550286+0000","last_fullsized":"2023-07-18T03:38:36.550286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:23.497046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.538475+0000","last_change":"2023-07-18T03:38:36.535669+0000","last_active":"2023-07-18T03:38:36.538475+0000","last_peered":"2023-07-18T03:38:36.538475+0000","last_clean":"2023-07-18T03:38:36.538475+0000","last_became_active":"2023-07-18T03:38:36.535338+0000","last_became_peered":"2023-07-18T03:38:36.535338+0000","last_unstale":"2023-07-18T03:38:36.538475+0000","last_undegraded":"2023-07-18T03:38:36.538475+0000","last_fullsized":"2023-07-18T03:38:36.538475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:35:45.788407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121279+0000","last_change":"2023-07-18T03:38:35.530541+0000","last_active":"2023-07-18T03:38:38.121279+0000","last_peered":"2023-07-18T03:38:38.121279+0000","last_clean":"2023-07-18T03:38:38.121279+0000","last_became_active":"2023-07-18T03:38:35.530220+0000","last_became_peered":"2023-07-18T03:38:35.530220+0000","last_unstale":"2023-07-18T03:38:38.121279+0000","last_undegraded":"2023-07-18T03:38:38.121279+0000","last_fullsized":"2023-07-18T03:38:38.121279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:53:15.570874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.527879+0000","last_change":"2023-07-18T03:38:36.523648+0000","last_active":"2023-07-18T03:38:36.527879+0000","last_peered":"2023-07-18T03:38:36.527879+0000","last_clean":"2023-07-18T03:38:36.527879+0000","last_became_active":"2023-07-18T03:38:36.523346+0000","last_became_peered":"2023-07-18T03:38:36.523346+0000","last_unstale":"2023-07-18T03:38:36.527879+0000","last_undegraded":"2023-07-18T03:38:36.527879+0000","last_fullsized":"2023-07-18T03:38:36.527879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:44:27.833608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.516626+0000","last_change":"2023-07-18T03:38:35.534663+0000","last_active":"2023-07-18T03:38:36.516626+0000","last_peered":"2023-07-18T03:38:36.516626+0000","last_clean":"2023-07-18T03:38:36.516626+0000","last_became_active":"2023-07-18T03:38:35.534363+0000","last_became_peered":"2023-07-18T03:38:35.534363+0000","last_unstale":"2023-07-18T03:38:36.516626+0000","last_undegraded":"2023-07-18T03:38:36.516626+0000","last_fullsized":"2023-07-18T03:38:36.516626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:50:45.063449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537860+0000","last_change":"2023-07-18T03:38:36.531919+0000","last_active":"2023-07-18T03:38:36.537860+0000","last_peered":"2023-07-18T03:38:36.537860+0000","last_clean":"2023-07-18T03:38:36.537860+0000","last_became_active":"2023-07-18T03:38:36.531729+0000","last_became_peered":"2023-07-18T03:38:36.531729+0000","last_unstale":"2023-07-18T03:38:36.537860+0000","last_undegraded":"2023-07-18T03:38:36.537860+0000","last_fullsized":"2023-07-18T03:38:36.537860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:18.677951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.525093+0000","last_change":"2023-07-18T03:38:35.538915+0000","last_active":"2023-07-18T03:38:36.525093+0000","last_peered":"2023-07-18T03:38:36.525093+0000","last_clean":"2023-07-18T03:38:36.525093+0000","last_became_active":"2023-07-18T03:38:35.538539+0000","last_became_peered":"2023-07-18T03:38:35.538539+0000","last_unstale":"2023-07-18T03:38:36.525093+0000","last_undegraded":"2023-07-18T03:38:36.525093+0000","last_fullsized":"2023-07-18T03:38:36.525093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:59:17.569992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537140+0000","last_change":"2023-07-18T03:38:36.534839+0000","last_active":"2023-07-18T03:38:36.537140+0000","last_peered":"2023-07-18T03:38:36.537140+0000","last_clean":"2023-07-18T03:38:36.537140+0000","last_became_active":"2023-07-18T03:38:36.532743+0000","last_became_peered":"2023-07-18T03:38:36.532743+0000","last_unstale":"2023-07-18T03:38:36.537140+0000","last_undegraded":"2023-07-18T03:38:36.537140+0000","last_fullsized":"2023-07-18T03:38:36.537140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:24:12.848519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.553664+0000","last_change":"2023-07-18T03:38:35.538083+0000","last_active":"2023-07-18T03:38:36.553664+0000","last_peered":"2023-07-18T03:38:36.553664+0000","last_clean":"2023-07-18T03:38:36.553664+0000","last_became_active":"2023-07-18T03:38:35.537692+0000","last_became_peered":"2023-07-18T03:38:35.537692+0000","last_unstale":"2023-07-18T03:38:36.553664+0000","last_undegraded":"2023-07-18T03:38:36.553664+0000","last_fullsized":"2023-07-18T03:38:36.553664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:53:15.570874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.516656+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:38:36.516656+0000","last_peered":"2023-07-18T03:38:36.516656+0000","last_clean":"2023-07-18T03:38:36.516656+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:38:36.516656+0000","last_undegraded":"2023-07-18T03:38:36.516656+0000","last_fullsized":"2023-07-18T03:38:36.516656+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:17.327799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.536380+0000","last_change":"2023-07-18T03:38:36.531962+0000","last_active":"2023-07-18T03:38:36.536380+0000","last_peered":"2023-07-18T03:38:36.536380+0000","last_clean":"2023-07-18T03:38:36.536380+0000","last_became_active":"2023-07-18T03:38:36.531710+0000","last_became_peered":"2023-07-18T03:38:36.531710+0000","last_unstale":"2023-07-18T03:38:36.536380+0000","last_undegraded":"2023-07-18T03:38:36.536380+0000","last_fullsized":"2023-07-18T03:38:36.536380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:29:26.283162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.526245+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:38:36.526245+0000","last_peered":"2023-07-18T03:38:36.526245+0000","last_clean":"2023-07-18T03:38:36.526245+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:38:36.526245+0000","last_undegraded":"2023-07-18T03:38:36.526245+0000","last_fullsized":"2023-07-18T03:38:36.526245+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:50:38.894203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121101+0000","last_change":"2023-07-18T03:38:35.522198+0000","last_active":"2023-07-18T03:38:38.121101+0000","last_peered":"2023-07-18T03:38:38.121101+0000","last_clean":"2023-07-18T03:38:38.121101+0000","last_became_active":"2023-07-18T03:38:35.521915+0000","last_became_peered":"2023-07-18T03:38:35.521915+0000","last_unstale":"2023-07-18T03:38:38.121101+0000","last_undegraded":"2023-07-18T03:38:38.121101+0000","last_fullsized":"2023-07-18T03:38:38.121101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:06:13.388319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.544904+0000","last_change":"2023-07-18T03:38:36.537103+0000","last_active":"2023-07-18T03:38:36.544904+0000","last_peered":"2023-07-18T03:38:36.544904+0000","last_clean":"2023-07-18T03:38:36.544904+0000","last_became_active":"2023-07-18T03:38:36.534926+0000","last_became_peered":"2023-07-18T03:38:36.534926+0000","last_unstale":"2023-07-18T03:38:36.544904+0000","last_undegraded":"2023-07-18T03:38:36.544904+0000","last_fullsized":"2023-07-18T03:38:36.544904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:01:03.276648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.512308+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:38:36.512308+0000","last_peered":"2023-07-18T03:38:36.512308+0000","last_clean":"2023-07-18T03:38:36.512308+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:38:36.512308+0000","last_undegraded":"2023-07-18T03:38:36.512308+0000","last_fullsized":"2023-07-18T03:38:36.512308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:07:26.667202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515826+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:38:37.515826+0000","last_peered":"2023-07-18T03:38:37.515826+0000","last_clean":"2023-07-18T03:38:37.515826+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:38:37.515826+0000","last_undegraded":"2023-07-18T03:38:37.515826+0000","last_fullsized":"2023-07-18T03:38:37.515826+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:38.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120579+0000","last_change":"2023-07-18T03:38:35.534781+0000","last_active":"2023-07-18T03:38:38.120579+0000","last_peered":"2023-07-18T03:38:38.120579+0000","last_clean":"2023-07-18T03:38:38.120579+0000","last_became_active":"2023-07-18T03:38:35.534485+0000","last_became_peered":"2023-07-18T03:38:35.534485+0000","last_unstale":"2023-07-18T03:38:38.120579+0000","last_undegraded":"2023-07-18T03:38:38.120579+0000","last_fullsized":"2023-07-18T03:38:38.120579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:42:36.215812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120857+0000","last_change":"2023-07-18T03:38:36.527606+0000","last_active":"2023-07-18T03:38:38.120857+0000","last_peered":"2023-07-18T03:38:38.120857+0000","last_clean":"2023-07-18T03:38:38.120857+0000","last_became_active":"2023-07-18T03:38:36.527267+0000","last_became_peered":"2023-07-18T03:38:36.527267+0000","last_unstale":"2023-07-18T03:38:38.120857+0000","last_undegraded":"2023-07-18T03:38:38.120857+0000","last_fullsized":"2023-07-18T03:38:38.120857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:32:24.221027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515516+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:38:37.515516+0000","last_peered":"2023-07-18T03:38:37.515516+0000","last_clean":"2023-07-18T03:38:37.515516+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:38:37.515516+0000","last_undegraded":"2023-07-18T03:38:37.515516+0000","last_fullsized":"2023-07-18T03:38:37.515516+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:02.380287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.512496+0000","last_change":"2023-07-18T03:38:35.531556+0000","last_active":"2023-07-18T03:38:36.512496+0000","last_peered":"2023-07-18T03:38:36.512496+0000","last_clean":"2023-07-18T03:38:36.512496+0000","last_became_active":"2023-07-18T03:38:35.531268+0000","last_became_peered":"2023-07-18T03:38:35.531268+0000","last_unstale":"2023-07-18T03:38:36.512496+0000","last_undegraded":"2023-07-18T03:38:36.512496+0000","last_fullsized":"2023-07-18T03:38:36.512496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:48:25.450768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537739+0000","last_change":"2023-07-18T03:38:36.536571+0000","last_active":"2023-07-18T03:38:36.537739+0000","last_peered":"2023-07-18T03:38:36.537739+0000","last_clean":"2023-07-18T03:38:36.537739+0000","last_became_active":"2023-07-18T03:38:36.536187+0000","last_became_peered":"2023-07-18T03:38:36.536187+0000","last_unstale":"2023-07-18T03:38:36.537739+0000","last_undegraded":"2023-07-18T03:38:36.537739+0000","last_fullsized":"2023-07-18T03:38:36.537739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:39.335445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515534+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:38:37.515534+0000","last_peered":"2023-07-18T03:38:37.515534+0000","last_clean":"2023-07-18T03:38:37.515534+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:38:37.515534+0000","last_undegraded":"2023-07-18T03:38:37.515534+0000","last_fullsized":"2023-07-18T03:38:37.515534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:38:34.441575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.526334+0000","last_change":"2023-07-18T03:38:35.536386+0000","last_active":"2023-07-18T03:38:36.526334+0000","last_peered":"2023-07-18T03:38:36.526334+0000","last_clean":"2023-07-18T03:38:36.526334+0000","last_became_active":"2023-07-18T03:38:35.536066+0000","last_became_peered":"2023-07-18T03:38:35.536066+0000","last_unstale":"2023-07-18T03:38:36.526334+0000","last_undegraded":"2023-07-18T03:38:36.526334+0000","last_fullsized":"2023-07-18T03:38:36.526334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:01.259707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121258+0000","last_change":"2023-07-18T03:38:36.528088+0000","last_active":"2023-07-18T03:38:38.121258+0000","last_peered":"2023-07-18T03:38:38.121258+0000","last_clean":"2023-07-18T03:38:38.121258+0000","last_became_active":"2023-07-18T03:38:36.527760+0000","last_became_peered":"2023-07-18T03:38:36.527760+0000","last_unstale":"2023-07-18T03:38:38.121258+0000","last_undegraded":"2023-07-18T03:38:38.121258+0000","last_fullsized":"2023-07-18T03:38:38.121258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:38.898070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.525054+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:38:36.525054+0000","last_peered":"2023-07-18T03:38:36.525054+0000","last_clean":"2023-07-18T03:38:36.525054+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:38:36.525054+0000","last_undegraded":"2023-07-18T03:38:36.525054+0000","last_fullsized":"2023-07-18T03:38:36.525054+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:20.720501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.516127+0000","last_change":"2023-07-18T03:38:35.526465+0000","last_active":"2023-07-18T03:38:36.516127+0000","last_peered":"2023-07-18T03:38:36.516127+0000","last_clean":"2023-07-18T03:38:36.516127+0000","last_became_active":"2023-07-18T03:38:35.525759+0000","last_became_peered":"2023-07-18T03:38:35.525759+0000","last_unstale":"2023-07-18T03:38:36.516127+0000","last_undegraded":"2023-07-18T03:38:36.516127+0000","last_fullsized":"2023-07-18T03:38:36.516127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:59:17.569992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.532144+0000","last_change":"2023-07-18T03:38:36.523849+0000","last_active":"2023-07-18T03:38:36.532144+0000","last_peered":"2023-07-18T03:38:36.532144+0000","last_clean":"2023-07-18T03:38:36.532144+0000","last_became_active":"2023-07-18T03:38:36.523555+0000","last_became_peered":"2023-07-18T03:38:36.523555+0000","last_unstale":"2023-07-18T03:38:36.532144+0000","last_undegraded":"2023-07-18T03:38:36.532144+0000","last_fullsized":"2023-07-18T03:38:36.532144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:29:26.283162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121324+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:38:38.121324+0000","last_peered":"2023-07-18T03:38:38.121324+0000","last_clean":"2023-07-18T03:38:38.121324+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:38:38.121324+0000","last_undegraded":"2023-07-18T03:38:38.121324+0000","last_fullsized":"2023-07-18T03:38:38.121324+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:50:38.894203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.512887+0000","last_change":"2023-07-18T03:38:35.530710+0000","last_active":"2023-07-18T03:38:36.512887+0000","last_peered":"2023-07-18T03:38:36.512887+0000","last_clean":"2023-07-18T03:38:36.512887+0000","last_became_active":"2023-07-18T03:38:35.530463+0000","last_became_peered":"2023-07-18T03:38:35.530463+0000","last_unstale":"2023-07-18T03:38:36.512887+0000","last_undegraded":"2023-07-18T03:38:36.512887+0000","last_fullsized":"2023-07-18T03:38:36.512887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:35:44.779979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121163+0000","last_change":"2023-07-18T03:38:36.535754+0000","last_active":"2023-07-18T03:38:38.121163+0000","last_peered":"2023-07-18T03:38:38.121163+0000","last_clean":"2023-07-18T03:38:38.121163+0000","last_became_active":"2023-07-18T03:38:36.535290+0000","last_became_peered":"2023-07-18T03:38:36.535290+0000","last_unstale":"2023-07-18T03:38:38.121163+0000","last_undegraded":"2023-07-18T03:38:38.121163+0000","last_fullsized":"2023-07-18T03:38:38.121163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:03.011261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121317+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:38:38.121317+0000","last_peered":"2023-07-18T03:38:38.121317+0000","last_clean":"2023-07-18T03:38:38.121317+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:38:38.121317+0000","last_undegraded":"2023-07-18T03:38:38.121317+0000","last_fullsized":"2023-07-18T03:38:38.121317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:42:57.376881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.525694+0000","last_change":"2023-07-18T03:38:35.536982+0000","last_active":"2023-07-18T03:38:36.525694+0000","last_peered":"2023-07-18T03:38:36.525694+0000","last_clean":"2023-07-18T03:38:36.525694+0000","last_became_active":"2023-07-18T03:38:35.536739+0000","last_became_peered":"2023-07-18T03:38:35.536739+0000","last_unstale":"2023-07-18T03:38:36.525694+0000","last_undegraded":"2023-07-18T03:38:36.525694+0000","last_fullsized":"2023-07-18T03:38:36.525694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:01:02.268220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.540887+0000","last_change":"2023-07-18T03:38:36.532972+0000","last_active":"2023-07-18T03:38:36.540887+0000","last_peered":"2023-07-18T03:38:36.540887+0000","last_clean":"2023-07-18T03:38:36.540887+0000","last_became_active":"2023-07-18T03:38:36.530512+0000","last_became_peered":"2023-07-18T03:38:36.530512+0000","last_unstale":"2023-07-18T03:38:36.540887+0000","last_undegraded":"2023-07-18T03:38:36.540887+0000","last_fullsized":"2023-07-18T03:38:36.540887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:01:03.276648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121293+0000","last_change":"2023-07-18T03:38:35.525179+0000","last_active":"2023-07-18T03:38:38.121293+0000","last_peered":"2023-07-18T03:38:38.121293+0000","last_clean":"2023-07-18T03:38:38.121293+0000","last_became_active":"2023-07-18T03:38:35.524912+0000","last_became_peered":"2023-07-18T03:38:35.524912+0000","last_unstale":"2023-07-18T03:38:38.121293+0000","last_undegraded":"2023-07-18T03:38:38.121293+0000","last_fullsized":"2023-07-18T03:38:38.121293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:23:28.704920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.122118+0000","last_change":"2023-07-18T03:38:36.529814+0000","last_active":"2023-07-18T03:38:38.122118+0000","last_peered":"2023-07-18T03:38:38.122118+0000","last_clean":"2023-07-18T03:38:38.122118+0000","last_became_active":"2023-07-18T03:38:36.529554+0000","last_became_peered":"2023-07-18T03:38:36.529554+0000","last_unstale":"2023-07-18T03:38:38.122118+0000","last_undegraded":"2023-07-18T03:38:38.122118+0000","last_fullsized":"2023-07-18T03:38:38.122118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:30:55.302127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.551181+0000","last_change":"2023-07-18T03:38:35.529892+0000","last_active":"2023-07-18T03:38:36.551181+0000","last_peered":"2023-07-18T03:38:36.551181+0000","last_clean":"2023-07-18T03:38:36.551181+0000","last_became_active":"2023-07-18T03:38:35.529614+0000","last_became_peered":"2023-07-18T03:38:35.529614+0000","last_unstale":"2023-07-18T03:38:36.551181+0000","last_undegraded":"2023-07-18T03:38:36.551181+0000","last_fullsized":"2023-07-18T03:38:36.551181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:17.669523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537984+0000","last_change":"2023-07-18T03:38:36.535652+0000","last_active":"2023-07-18T03:38:36.537984+0000","last_peered":"2023-07-18T03:38:36.537984+0000","last_clean":"2023-07-18T03:38:36.537984+0000","last_became_active":"2023-07-18T03:38:36.535378+0000","last_became_peered":"2023-07-18T03:38:36.535378+0000","last_unstale":"2023-07-18T03:38:36.537984+0000","last_undegraded":"2023-07-18T03:38:36.537984+0000","last_fullsized":"2023-07-18T03:38:36.537984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:39.335445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.550038+0000","last_change":"2023-07-18T03:38:35.534949+0000","last_active":"2023-07-18T03:38:36.550038+0000","last_peered":"2023-07-18T03:38:36.550038+0000","last_clean":"2023-07-18T03:38:36.550038+0000","last_became_active":"2023-07-18T03:38:35.534685+0000","last_became_peered":"2023-07-18T03:38:35.534685+0000","last_unstale":"2023-07-18T03:38:36.550038+0000","last_undegraded":"2023-07-18T03:38:36.550038+0000","last_fullsized":"2023-07-18T03:38:36.550038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:24:11.840091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.538972+0000","last_change":"2023-07-18T03:38:36.535757+0000","last_active":"2023-07-18T03:38:36.538972+0000","last_peered":"2023-07-18T03:38:36.538972+0000","last_clean":"2023-07-18T03:38:36.538972+0000","last_became_active":"2023-07-18T03:38:36.535452+0000","last_became_peered":"2023-07-18T03:38:36.535452+0000","last_unstale":"2023-07-18T03:38:36.538972+0000","last_undegraded":"2023-07-18T03:38:36.538972+0000","last_fullsized":"2023-07-18T03:38:36.538972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:13:15.481453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.549343+0000","last_change":"2023-07-18T03:38:35.532954+0000","last_active":"2023-07-18T03:38:36.549343+0000","last_peered":"2023-07-18T03:38:36.549343+0000","last_clean":"2023-07-18T03:38:36.549343+0000","last_became_active":"2023-07-18T03:38:35.532617+0000","last_became_peered":"2023-07-18T03:38:35.532617+0000","last_unstale":"2023-07-18T03:38:36.549343+0000","last_undegraded":"2023-07-18T03:38:36.549343+0000","last_fullsized":"2023-07-18T03:38:36.549343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:03.546127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.543826+0000","last_change":"2023-07-18T03:38:36.540001+0000","last_active":"2023-07-18T03:38:36.543826+0000","last_peered":"2023-07-18T03:38:36.543826+0000","last_clean":"2023-07-18T03:38:36.543826+0000","last_became_active":"2023-07-18T03:38:36.539743+0000","last_became_peered":"2023-07-18T03:38:36.539743+0000","last_unstale":"2023-07-18T03:38:36.543826+0000","last_undegraded":"2023-07-18T03:38:36.543826+0000","last_fullsized":"2023-07-18T03:38:36.543826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:39.335445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121427+0000","last_change":"2023-07-18T03:38:35.532342+0000","last_active":"2023-07-18T03:38:38.121427+0000","last_peered":"2023-07-18T03:38:38.121427+0000","last_clean":"2023-07-18T03:38:38.121427+0000","last_became_active":"2023-07-18T03:38:35.530138+0000","last_became_peered":"2023-07-18T03:38:35.530138+0000","last_unstale":"2023-07-18T03:38:38.121427+0000","last_undegraded":"2023-07-18T03:38:38.121427+0000","last_fullsized":"2023-07-18T03:38:38.121427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:59:26.646208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.539939+0000","last_change":"2023-07-18T03:38:36.531861+0000","last_active":"2023-07-18T03:38:36.539939+0000","last_peered":"2023-07-18T03:38:36.539939+0000","last_clean":"2023-07-18T03:38:36.539939+0000","last_became_active":"2023-07-18T03:38:36.531563+0000","last_became_peered":"2023-07-18T03:38:36.531563+0000","last_unstale":"2023-07-18T03:38:36.539939+0000","last_undegraded":"2023-07-18T03:38:36.539939+0000","last_fullsized":"2023-07-18T03:38:36.539939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:48:26.459196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121209+0000","last_change":"2023-07-18T03:38:36.537688+0000","last_active":"2023-07-18T03:38:38.121209+0000","last_peered":"2023-07-18T03:38:38.121209+0000","last_clean":"2023-07-18T03:38:38.121209+0000","last_became_active":"2023-07-18T03:38:36.537415+0000","last_became_peered":"2023-07-18T03:38:36.537415+0000","last_unstale":"2023-07-18T03:38:38.121209+0000","last_undegraded":"2023-07-18T03:38:38.121209+0000","last_fullsized":"2023-07-18T03:38:38.121209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:13:15.481453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.544369+0000","last_change":"2023-07-18T03:38:36.542378+0000","last_active":"2023-07-18T03:38:36.544369+0000","last_peered":"2023-07-18T03:38:36.544369+0000","last_clean":"2023-07-18T03:38:36.544369+0000","last_became_active":"2023-07-18T03:38:36.542101+0000","last_became_peered":"2023-07-18T03:38:36.542101+0000","last_unstale":"2023-07-18T03:38:36.544369+0000","last_undegraded":"2023-07-18T03:38:36.544369+0000","last_fullsized":"2023-07-18T03:38:36.544369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:27.898176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.537836+0000","last_change":"2023-07-18T03:38:36.531395+0000","last_active":"2023-07-18T03:38:36.537836+0000","last_peered":"2023-07-18T03:38:36.537836+0000","last_clean":"2023-07-18T03:38:36.537836+0000","last_became_active":"2023-07-18T03:38:36.531071+0000","last_became_peered":"2023-07-18T03:38:36.531071+0000","last_unstale":"2023-07-18T03:38:36.537836+0000","last_undegraded":"2023-07-18T03:38:36.537836+0000","last_fullsized":"2023-07-18T03:38:36.537836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:29:26.283162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.541527+0000","last_change":"2023-07-18T03:38:36.534035+0000","last_active":"2023-07-18T03:38:36.541527+0000","last_peered":"2023-07-18T03:38:36.541527+0000","last_clean":"2023-07-18T03:38:36.541527+0000","last_became_active":"2023-07-18T03:38:36.533755+0000","last_became_peered":"2023-07-18T03:38:36.533755+0000","last_unstale":"2023-07-18T03:38:36.541527+0000","last_undegraded":"2023-07-18T03:38:36.541527+0000","last_fullsized":"2023-07-18T03:38:36.541527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:01:03.276648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-18T03:38:36.534239+0000","last_change":"2023-07-18T03:38:36.524624+0000","last_active":"2023-07-18T03:38:36.534239+0000","last_peered":"2023-07-18T03:38:36.534239+0000","last_clean":"2023-07-18T03:38:36.534239+0000","last_became_active":"2023-07-18T03:38:36.524106+0000","last_became_peered":"2023-07-18T03:38:36.524106+0000","last_unstale":"2023-07-18T03:38:36.534239+0000","last_undegraded":"2023-07-18T03:38:36.534239+0000","last_fullsized":"2023-07-18T03:38:36.534239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:13:03.155792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":843776,"data_stored":673348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":208896,"data_stored":74547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":212992,"data_stored":74637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":802816,"data_stored":664915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":76167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":75129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31264,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340576,"statfs":{"total":96636764160,"available":96604749824,"internally_reserved":0,"allocated":221184,"data_stored":81200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:38:41.473 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-18T03:38:41.473 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-18T03:38:41.474 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-07-18T03:38:41.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-07-18T03:38:41.474 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-07-18T03:38:41.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.715+0000 7fb6525ff700 1 Processor -- start 2023-07-18T03:38:41.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.716+0000 7fb6525ff700 1 -- start start 2023-07-18T03:38:41.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.717+0000 7fb6525ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb64c15c950 0x7fb64c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.717+0000 7fb6525ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb64c0b1e00 0x7fb64c0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.717+0000 7fb6525ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb64c0b2350 0x7fb64c0d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.717+0000 7fb6525ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64c0d2ce0 con 0x7fb64c0b2350 2023-07-18T03:38:41.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.717+0000 7fb6525ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64c0b3350 con 0x7fb64c15c950 2023-07-18T03:38:41.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.717+0000 7fb6525ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0b3630 con 0x7fb64c0b1e00 2023-07-18T03:38:41.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.717+0000 7fb650b9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb64c0b2350 0x7fb64c0d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.717+0000 7fb64bfff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb64c15c950 0x7fb64c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.717+0000 7fb64b7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb64c0b1e00 0x7fb64c0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.717+0000 7fb650b9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb64c0b2350 0x7fb64c0d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40256/0 (socket says 172.21.15.49:40256) 2023-07-18T03:38:41.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.717+0000 7fb64bfff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb64c15c950 0x7fb64c0b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45034/0 (socket says 172.21.15.49:45034) 2023-07-18T03:38:41.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.717+0000 7fb650b9c700 1 -- 172.21.15.49:0/2576747226 learned_addr learned my addr 172.21.15.49:0/2576747226 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:41.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.718+0000 7fb650b9c700 1 -- 172.21.15.49:0/2576747226 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb64c0b1e00 msgr2=0x7fb64c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:41.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.718+0000 7fb650b9c700 1 --2- 172.21.15.49:0/2576747226 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb64c0b1e00 0x7fb64c0d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.718+0000 7fb650b9c700 1 -- 172.21.15.49:0/2576747226 >> v2:172.21.15.107:3300/0 conn(0x7fb64c15c950 msgr2=0x7fb64c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:41.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.718+0000 7fb650b9c700 1 --2- 172.21.15.49:0/2576747226 >> v2:172.21.15.107:3300/0 conn(0x7fb64c15c950 0x7fb64c0b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.718+0000 7fb650b9c700 1 -- 172.21.15.49:0/2576747226 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c0b3910 con 0x7fb64c0b2350 2023-07-18T03:38:41.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.718+0000 7fb64bfff700 1 --2- 172.21.15.49:0/2576747226 >> v2:172.21.15.107:3300/0 conn(0x7fb64c15c950 0x7fb64c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:41.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.718+0000 7fb650b9c700 1 --2- 172.21.15.49:0/2576747226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c0b2350 0x7fb64c0d1050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb64c08d170 tx=0x7fb64400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89ea6bd4d3299f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:41.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.718+0000 7fb64affd700 1 -- 172.21.15.49:0/2576747226 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64400e070 con 0x7fb64c0b2350 2023-07-18T03:38:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.718+0000 7fb64affd700 1 -- 172.21.15.49:0/2576747226 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64400b910 con 0x7fb64c0b2350 2023-07-18T03:38:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.718+0000 7fb64affd700 1 -- 172.21.15.49:0/2576747226 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64400bcc0 con 0x7fb64c0b2350 2023-07-18T03:38:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.719+0000 7fb6525ff700 1 -- 172.21.15.49:0/2576747226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c0b2350 msgr2=0x7fb64c0d1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.719+0000 7fb6525ff700 1 --2- 172.21.15.49:0/2576747226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c0b2350 0x7fb64c0d1050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb64c08d170 tx=0x7fb64400b330 comp rx=0 tx=0).stop 2023-07-18T03:38:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.719+0000 7fb6525ff700 1 -- 172.21.15.49:0/2576747226 shutdown_connections 2023-07-18T03:38:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.719+0000 7fb6525ff700 1 --2- 172.21.15.49:0/2576747226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c0b2350 0x7fb64c0d1050 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.719+0000 7fb6525ff700 1 --2- 172.21.15.49:0/2576747226 >> v2:172.21.15.107:3300/0 conn(0x7fb64c15c950 0x7fb64c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.719+0000 7fb6525ff700 1 --2- 172.21.15.49:0/2576747226 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb64c0b1e00 0x7fb64c0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.719+0000 7fb6525ff700 1 -- 172.21.15.49:0/2576747226 >> 172.21.15.49:0/2576747226 conn(0x7fb64c09c1d0 msgr2=0x7fb64c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:41.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.719+0000 7fb6525ff700 1 -- 172.21.15.49:0/2576747226 shutdown_connections 2023-07-18T03:38:41.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.719+0000 7fb6525ff700 1 -- 172.21.15.49:0/2576747226 wait complete. 2023-07-18T03:38:41.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.721+0000 7fb6525ff700 1 Processor -- start 2023-07-18T03:38:41.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.721+0000 7fb6525ff700 1 -- start start 2023-07-18T03:38:41.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.722+0000 7fb6525ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c0b1e00 0x7fb64c0b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.722+0000 7fb6525ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb64c0b2350 0x7fb64c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.722+0000 7fb6525ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb64c15c950 0x7fb64c0b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.722+0000 7fb6525ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0b71f0 con 0x7fb64c0b1e00 2023-07-18T03:38:41.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.722+0000 7fb6525ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0b6150 con 0x7fb64c15c950 2023-07-18T03:38:41.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.722+0000 7fb6525ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0b6430 con 0x7fb64c0b2350 2023-07-18T03:38:41.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.722+0000 7fb64bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c0b1e00 0x7fb64c0b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.722+0000 7fb64bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c0b1e00 0x7fb64c0b84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40264/0 (socket says 172.21.15.49:40264) 2023-07-18T03:38:41.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.722+0000 7fb64bfff700 1 -- 172.21.15.49:0/3399054880 learned_addr learned my addr 172.21.15.49:0/3399054880 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:41.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.723+0000 7fb650b9c700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb64c15c950 0x7fb64c0b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.723+0000 7fb64b7fe700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb64c0b2350 0x7fb64c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.723+0000 7fb64bfff700 1 -- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb64c0b2350 msgr2=0x7fb64c0b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.723+0000 7fb64bfff700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb64c0b2350 0x7fb64c0b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.723+0000 7fb64bfff700 1 -- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb64c15c950 msgr2=0x7fb64c0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.723+0000 7fb64bfff700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb64c15c950 0x7fb64c0b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.723+0000 7fb64bfff700 1 -- 172.21.15.49:0/3399054880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c009430 con 0x7fb64c0b1e00 2023-07-18T03:38:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.723+0000 7fb64bfff700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c0b1e00 0x7fb64c0b84d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00ebe0 tx=0x7fb63c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.723+0000 7fb648ff9700 1 -- 172.21.15.49:0/3399054880 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c009740 con 0x7fb64c0b1e00 2023-07-18T03:38:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.723+0000 7fb648ff9700 1 -- 172.21.15.49:0/3399054880 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c005a20 con 0x7fb64c0b1e00 2023-07-18T03:38:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.723+0000 7fb648ff9700 1 -- 172.21.15.49:0/3399054880 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c005db0 con 0x7fb64c0b1e00 2023-07-18T03:38:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.723+0000 7fb6525ff700 1 -- 172.21.15.49:0/3399054880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb644009b30 con 0x7fb64c0b1e00 2023-07-18T03:38:41.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.724+0000 7fb6525ff700 1 -- 172.21.15.49:0/3399054880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64c0b69f0 con 0x7fb64c0b1e00 2023-07-18T03:38:41.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.724+0000 7fb6525ff700 1 -- 172.21.15.49:0/3399054880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64c0b7040 con 0x7fb64c0b1e00 2023-07-18T03:38:41.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.725+0000 7fb648ff9700 1 -- 172.21.15.49:0/3399054880 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb63c00c850 con 0x7fb64c0b1e00 2023-07-18T03:38:41.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.727+0000 7fb648ff9700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb63806ace0 0x7fb63806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.727+0000 7fb648ff9700 1 -- 172.21.15.49:0/3399054880 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb63c0895f0 con 0x7fb64c0b1e00 2023-07-18T03:38:41.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.727+0000 7fb64b7fe700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb63806ace0 0x7fb63806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.728+0000 7fb64b7fe700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb63806ace0 0x7fb63806d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0c9810 tx=0x7fb640009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:41.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.731+0000 7fb648ff9700 1 -- 172.21.15.49:0/3399054880 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb63c054690 con 0x7fb64c0b1e00 2023-07-18T03:38:41.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:41.999+0000 7fb6525ff700 1 -- 172.21.15.49:0/3399054880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb64c04e280 con 0x7fb64c0b1e00 2023-07-18T03:38:41.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.002+0000 7fb648ff9700 1 -- 172.21.15.49:0/3399054880 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170848 (secure 0 0 0) 0x7fb63c057cb0 con 0x7fb64c0b1e00 2023-07-18T03:38:41.999 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:42.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.006+0000 7fb6525ff700 1 -- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb63806ace0 msgr2=0x7fb63806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.006+0000 7fb6525ff700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb63806ace0 0x7fb63806d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0c9810 tx=0x7fb640009210 comp rx=0 tx=0).stop 2023-07-18T03:38:42.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.006+0000 7fb6525ff700 1 -- 172.21.15.49:0/3399054880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c0b1e00 msgr2=0x7fb64c0b84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.006+0000 7fb6525ff700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c0b1e00 0x7fb64c0b84d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00ebe0 tx=0x7fb63c00eef0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.006+0000 7fb6525ff700 1 -- 172.21.15.49:0/3399054880 shutdown_connections 2023-07-18T03:38:42.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.006+0000 7fb6525ff700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb64c15c950 0x7fb64c0b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.006+0000 7fb6525ff700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb63806ace0 0x7fb63806d190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.006+0000 7fb6525ff700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c0b1e00 0x7fb64c0b84d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.006+0000 7fb6525ff700 1 --2- 172.21.15.49:0/3399054880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb64c0b2350 0x7fb64c0b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.006+0000 7fb6525ff700 1 -- 172.21.15.49:0/3399054880 >> 172.21.15.49:0/3399054880 conn(0x7fb64c09c1d0 msgr2=0x7fb64c0d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:42.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.006+0000 7fb6525ff700 1 -- 172.21.15.49:0/3399054880 shutdown_connections 2023-07-18T03:38:42.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.006+0000 7fb6525ff700 1 -- 172.21.15.49:0/3399054880 wait complete. 2023-07-18T03:38:42.019 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2972169254},{"type":"v1","addr":"172.21.15.107:6833","nonce":2972169254}]},"active_addr":"172.21.15.107:6833/2972169254","active_change":"2023-07-18T03:38:22.084553+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3830272791}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":634829302}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":4059678764}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2773168669}]}]} 2023-07-18T03:38:42.024 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-07-18T03:38:42.024 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-07-18T03:38:42.025 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:42.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.234+0000 7f27ea7ff700 1 Processor -- start 2023-07-18T03:38:42.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.235+0000 7f27ea7ff700 1 -- start start 2023-07-18T03:38:42.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.236+0000 7f27ea7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f27e40d1220 0x7f27e40b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:42.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.236+0000 7f27ea7ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e40d2ce0 con 0x7f27e40b1e00 2023-07-18T03:38:42.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.236+0000 7f27ea7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e40b3350 con 0x7f27e415c950 2023-07-18T03:38:42.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.236+0000 7f27ea7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e40b3630 con 0x7f27e40d1220 2023-07-18T03:38:42.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.236+0000 7f27e8d9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f27e40b1e00 0x7f27e40afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41530/0 (socket says 172.21.15.49:41530) 2023-07-18T03:38:42.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.236+0000 7f27e8d9c700 1 -- 172.21.15.49:0/3616244346 learned_addr learned my addr 172.21.15.49:0/3616244346 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:42.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.236+0000 7f27e37fe700 1 --2- 172.21.15.49:0/3616244346 >> v2:172.21.15.107:3300/0 conn(0x7f27e40d1220 0x7f27e40b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:42.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.236+0000 7f27e2ffd700 1 -- 172.21.15.49:0/3616244346 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741843820 0 0) 0x7f27e40d2ce0 con 0x7f27e40b1e00 2023-07-18T03:38:42.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.237+0000 7f27e2ffd700 1 -- 172.21.15.49:0/3616244346 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27cc0035b0 con 0x7f27e40b1e00 2023-07-18T03:38:42.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.237+0000 7f27e2ffd700 1 -- 172.21.15.49:0/3616244346 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144889124 0 0) 0x7f27e40b3350 con 0x7f27e415c950 2023-07-18T03:38:42.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.237+0000 7f27e2ffd700 1 -- 172.21.15.49:0/3616244346 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27e40d2ce0 con 0x7f27e415c950 2023-07-18T03:38:42.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.237+0000 7f27e37fe700 1 -- 172.21.15.49:0/3616244346 >> v1:172.21.15.107:6789/0 conn(0x7f27e415c950 legacy=0x7f27e40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.237+0000 7f27e37fe700 1 -- 172.21.15.49:0/3616244346 >> v1:172.21.15.49:6789/0 conn(0x7f27e40b1e00 legacy=0x7f27e40afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.237+0000 7f27e37fe700 1 -- 172.21.15.49:0/3616244346 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e40b3910 con 0x7f27e40d1220 2023-07-18T03:38:42.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.237+0000 7f27e37fe700 1 --2- 172.21.15.49:0/3616244346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e40d1220 0x7f27e40b43f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009840 tx=0x7f27d8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=839aa573ad60406c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:42.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.237+0000 7f27e2ffd700 1 -- 172.21.15.49:0/3616244346 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8011830 con 0x7f27e40d1220 2023-07-18T03:38:42.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.237+0000 7f27e2ffd700 1 -- 172.21.15.49:0/3616244346 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d8011990 con 0x7f27e40d1220 2023-07-18T03:38:42.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.237+0000 7f27e2ffd700 1 -- 172.21.15.49:0/3616244346 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8011d20 con 0x7f27e40d1220 2023-07-18T03:38:42.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.238+0000 7f27ea7ff700 1 -- 172.21.15.49:0/3616244346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e40d1220 msgr2=0x7f27e40b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.238+0000 7f27ea7ff700 1 --2- 172.21.15.49:0/3616244346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e40d1220 0x7f27e40b43f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009840 tx=0x7f27d8009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:42.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.238+0000 7f27ea7ff700 1 -- 172.21.15.49:0/3616244346 shutdown_connections 2023-07-18T03:38:42.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.238+0000 7f27ea7ff700 1 --2- 172.21.15.49:0/3616244346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e40d1220 0x7f27e40b43f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.238+0000 7f27ea7ff700 1 -- 172.21.15.49:0/3616244346 >> 172.21.15.49:0/3616244346 conn(0x7f27e409c1d0 msgr2=0x7f27e40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:42.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.238+0000 7f27ea7ff700 1 -- 172.21.15.49:0/3616244346 shutdown_connections 2023-07-18T03:38:42.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.238+0000 7f27ea7ff700 1 -- 172.21.15.49:0/3616244346 wait complete. 2023-07-18T03:38:42.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.240+0000 7f27ea7ff700 1 Processor -- start 2023-07-18T03:38:42.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.240+0000 7f27ea7ff700 1 -- start start 2023-07-18T03:38:42.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.241+0000 7f27ea7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e40b1e00 0x7f27e40cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:42.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.241+0000 7f27ea7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27e40d1220 0x7f27e40cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:42.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.241+0000 7f27ea7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27e415c950 0x7f27e40cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:42.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.241+0000 7f27ea7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40cc020 con 0x7f27e40d1220 2023-07-18T03:38:42.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.241+0000 7f27ea7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40ce000 con 0x7f27e40b1e00 2023-07-18T03:38:42.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.241+0000 7f27ea7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41579f0 con 0x7f27e415c950 2023-07-18T03:38:42.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.241+0000 7f27e37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27e40d1220 0x7f27e40cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:42.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.241+0000 7f27e37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27e40d1220 0x7f27e40cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40266/0 (socket says 172.21.15.49:40266) 2023-07-18T03:38:42.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.241+0000 7f27e37fe700 1 -- 172.21.15.49:0/4141381965 learned_addr learned my addr 172.21.15.49:0/4141381965 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:42.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.241+0000 7f27e3fff700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e40b1e00 0x7f27e40cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:42.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.241+0000 7f27e8d9c700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27e415c950 0x7f27e40cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:42.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.242+0000 7f27e37fe700 1 -- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27e415c950 msgr2=0x7f27e40cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.242+0000 7f27e37fe700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27e415c950 0x7f27e40cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.242+0000 7f27e37fe700 1 -- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e40b1e00 msgr2=0x7f27e40cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.242+0000 7f27e37fe700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e40b1e00 0x7f27e40cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.242+0000 7f27e37fe700 1 -- 172.21.15.49:0/4141381965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d80094a0 con 0x7f27e40d1220 2023-07-18T03:38:42.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.242+0000 7f27e37fe700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27e40d1220 0x7f27e40cb5a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009b90 tx=0x7f27d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:42.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.242+0000 7f27e0ff9700 1 -- 172.21.15.49:0/4141381965 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d801a760 con 0x7f27e40d1220 2023-07-18T03:38:42.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.242+0000 7f27e0ff9700 1 -- 172.21.15.49:0/4141381965 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d801a8c0 con 0x7f27e40d1220 2023-07-18T03:38:42.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.242+0000 7f27ea7ff700 1 -- 172.21.15.49:0/4141381965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e40cc210 con 0x7f27e40d1220 2023-07-18T03:38:42.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.242+0000 7f27e0ff9700 1 -- 172.21.15.49:0/4141381965 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d801ac70 con 0x7f27e40d1220 2023-07-18T03:38:42.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.242+0000 7f27ea7ff700 1 -- 172.21.15.49:0/4141381965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e40cc7d0 con 0x7f27e40d1220 2023-07-18T03:38:42.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.243+0000 7f27ea7ff700 1 -- 172.21.15.49:0/4141381965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e40d35c0 con 0x7f27e40d1220 2023-07-18T03:38:42.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.244+0000 7f27e0ff9700 1 -- 172.21.15.49:0/4141381965 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f27d8022070 con 0x7f27e40d1220 2023-07-18T03:38:42.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.245+0000 7f27e0ff9700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f27d006ace0 0x7f27d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:42.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.245+0000 7f27e0ff9700 1 -- 172.21.15.49:0/4141381965 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f27d808a450 con 0x7f27e40d1220 2023-07-18T03:38:42.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.246+0000 7f27e3fff700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f27d006ace0 0x7f27d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:42.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.246+0000 7f27e3fff700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f27d006ace0 0x7f27d006d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27d4007fd0 tx=0x7f27d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:42.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.250+0000 7f27e0ff9700 1 -- 172.21.15.49:0/4141381965 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f27d80554f0 con 0x7f27e40d1220 2023-07-18T03:38:42.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.458+0000 7f27ea7ff700 1 -- 172.21.15.49:0/4141381965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27e406a3e0 con 0x7f27e40d1220 2023-07-18T03:38:42.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.460+0000 7f27e0ff9700 1 -- 172.21.15.49:0/4141381965 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f27d8058b10 con 0x7f27e40d1220 2023-07-18T03:38:42.456 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:42.456 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:37.510006+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:42.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.462+0000 7f27ea7ff700 1 -- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f27d006ace0 msgr2=0x7f27d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.462+0000 7f27ea7ff700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f27d006ace0 0x7f27d006d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27d4007fd0 tx=0x7f27d4009040 comp rx=0 tx=0).stop 2023-07-18T03:38:42.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.462+0000 7f27ea7ff700 1 -- 172.21.15.49:0/4141381965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27e40d1220 msgr2=0x7f27e40cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.462+0000 7f27ea7ff700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27e40d1220 0x7f27e40cb5a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009b90 tx=0x7f27d8004750 comp rx=0 tx=0).stop 2023-07-18T03:38:42.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.462+0000 7f27ea7ff700 1 -- 172.21.15.49:0/4141381965 shutdown_connections 2023-07-18T03:38:42.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.462+0000 7f27ea7ff700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27e415c950 0x7f27e40cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.462+0000 7f27ea7ff700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e40b1e00 0x7f27e40cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.462+0000 7f27ea7ff700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f27d006ace0 0x7f27d006d190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.462+0000 7f27ea7ff700 1 --2- 172.21.15.49:0/4141381965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27e40d1220 0x7f27e40cb5a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.462+0000 7f27ea7ff700 1 -- 172.21.15.49:0/4141381965 >> 172.21.15.49:0/4141381965 conn(0x7f27e409c1d0 msgr2=0x7f27e40b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.462+0000 7f27ea7ff700 1 -- 172.21.15.49:0/4141381965 shutdown_connections 2023-07-18T03:38:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.462+0000 7f27ea7ff700 1 -- 172.21.15.49:0/4141381965 wait complete. 2023-07-18T03:38:42.476 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-07-18T03:38:42.476 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:42.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.685+0000 7fd36adff700 1 Processor -- start 2023-07-18T03:38:42.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.685+0000 7fd36adff700 1 -- start start 2023-07-18T03:38:42.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.687+0000 7fd36adff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd3640b1e00 0x7fd3640b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:42.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.687+0000 7fd36adff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3640d2ce0 con 0x7fd36415c950 2023-07-18T03:38:42.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.687+0000 7fd36adff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3640b3350 con 0x7fd3640d1220 2023-07-18T03:38:42.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.687+0000 7fd36adff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3640b3630 con 0x7fd3640b1e00 2023-07-18T03:38:42.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.687+0000 7fd368b9b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd36415c950 0x7fd3640d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41546/0 (socket says 172.21.15.49:41546) 2023-07-18T03:38:42.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.687+0000 7fd368b9b700 1 -- 172.21.15.49:0/1809874095 learned_addr learned my addr 172.21.15.49:0/1809874095 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:42.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.687+0000 7fd36939c700 1 --2- 172.21.15.49:0/1809874095 >> v2:172.21.15.107:3300/0 conn(0x7fd3640b1e00 0x7fd3640b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:42.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.687+0000 7fd3637fe700 1 -- 172.21.15.49:0/1809874095 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015129905 0 0) 0x7fd3640d2ce0 con 0x7fd36415c950 2023-07-18T03:38:42.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.687+0000 7fd3637fe700 1 -- 172.21.15.49:0/1809874095 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3540035b0 con 0x7fd36415c950 2023-07-18T03:38:42.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.688+0000 7fd3637fe700 1 -- 172.21.15.49:0/1809874095 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731491817 0 0) 0x7fd3640b3350 con 0x7fd3640d1220 2023-07-18T03:38:42.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.688+0000 7fd3637fe700 1 -- 172.21.15.49:0/1809874095 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3640d2ce0 con 0x7fd3640d1220 2023-07-18T03:38:42.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.688+0000 7fd36939c700 1 -- 172.21.15.49:0/1809874095 >> v1:172.21.15.107:6789/0 conn(0x7fd3640d1220 legacy=0x7fd3640afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.688+0000 7fd36939c700 1 -- 172.21.15.49:0/1809874095 >> v1:172.21.15.49:6789/0 conn(0x7fd36415c950 legacy=0x7fd3640d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.688+0000 7fd3637fe700 1 -- 172.21.15.49:0/1809874095 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3905573045 0 0) 0x7fd3540035b0 con 0x7fd36415c950 2023-07-18T03:38:42.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.688+0000 7fd36939c700 1 -- 172.21.15.49:0/1809874095 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640b3910 con 0x7fd3640b1e00 2023-07-18T03:38:42.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.688+0000 7fd36939c700 1 --2- 172.21.15.49:0/1809874095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3640b1e00 0x7fd3640b43f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009d90 tx=0x7fd35c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3382621b1be4e4f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:42.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.688+0000 7fd3637fe700 1 -- 172.21.15.49:0/1809874095 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00e070 con 0x7fd3640b1e00 2023-07-18T03:38:42.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.688+0000 7fd3637fe700 1 -- 172.21.15.49:0/1809874095 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c003a20 con 0x7fd3640b1e00 2023-07-18T03:38:42.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.688+0000 7fd3637fe700 1 -- 172.21.15.49:0/1809874095 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3580017f0 con 0x7fd3640b1e00 2023-07-18T03:38:42.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.689+0000 7fd36adff700 1 -- 172.21.15.49:0/1809874095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3640b1e00 msgr2=0x7fd3640b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.689+0000 7fd36adff700 1 --2- 172.21.15.49:0/1809874095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3640b1e00 0x7fd3640b43f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009d90 tx=0x7fd35c00b330 comp rx=0 tx=0).stop 2023-07-18T03:38:42.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.689+0000 7fd36adff700 1 -- 172.21.15.49:0/1809874095 shutdown_connections 2023-07-18T03:38:42.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.689+0000 7fd36adff700 1 --2- 172.21.15.49:0/1809874095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3640b1e00 0x7fd3640b43f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.689+0000 7fd36adff700 1 -- 172.21.15.49:0/1809874095 >> 172.21.15.49:0/1809874095 conn(0x7fd36409c1d0 msgr2=0x7fd3640d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:42.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.689+0000 7fd36adff700 1 -- 172.21.15.49:0/1809874095 shutdown_connections 2023-07-18T03:38:42.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.689+0000 7fd36adff700 1 -- 172.21.15.49:0/1809874095 wait complete. 2023-07-18T03:38:42.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.691+0000 7fd36adff700 1 Processor -- start 2023-07-18T03:38:42.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.691+0000 7fd36adff700 1 -- start start 2023-07-18T03:38:42.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.692+0000 7fd36adff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3640b1e00 0x7fd36415f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:42.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.692+0000 7fd36adff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3640d1220 0x7fd3640cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:42.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.692+0000 7fd36adff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd36415c950 0x7fd364152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:42.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.692+0000 7fd36adff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640ca010 con 0x7fd36415c950 2023-07-18T03:38:42.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.692+0000 7fd36adff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640cbc50 con 0x7fd3640d1220 2023-07-18T03:38:42.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.692+0000 7fd36adff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640cc700 con 0x7fd3640b1e00 2023-07-18T03:38:42.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.692+0000 7fd36939c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd36415c950 0x7fd364152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:42.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.692+0000 7fd36939c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd36415c950 0x7fd364152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40280/0 (socket says 172.21.15.49:40280) 2023-07-18T03:38:42.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.692+0000 7fd36939c700 1 -- 172.21.15.49:0/3045353458 learned_addr learned my addr 172.21.15.49:0/3045353458 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:42.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.692+0000 7fd363fff700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3640d1220 0x7fd3640cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:42.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.692+0000 7fd368b9b700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3640b1e00 0x7fd36415f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:42.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.693+0000 7fd36939c700 1 -- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3640b1e00 msgr2=0x7fd36415f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.693+0000 7fd36939c700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3640b1e00 0x7fd36415f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.693+0000 7fd36939c700 1 -- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3640d1220 msgr2=0x7fd3640cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.693+0000 7fd36939c700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3640d1220 0x7fd3640cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.693+0000 7fd36939c700 1 -- 172.21.15.49:0/3045353458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c009a70 con 0x7fd36415c950 2023-07-18T03:38:42.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.693+0000 7fd36939c700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd36415c950 0x7fd364152680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00b370 tx=0x7fd35c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:42.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.693+0000 7fd3617fa700 1 -- 172.21.15.49:0/3045353458 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0044c0 con 0x7fd36415c950 2023-07-18T03:38:42.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.693+0000 7fd3617fa700 1 -- 172.21.15.49:0/3045353458 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c004620 con 0x7fd36415c950 2023-07-18T03:38:42.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.693+0000 7fd3617fa700 1 -- 172.21.15.49:0/3045353458 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c01d470 con 0x7fd36415c950 2023-07-18T03:38:42.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.693+0000 7fd36adff700 1 -- 172.21.15.49:0/3045353458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3640cbe60 con 0x7fd36415c950 2023-07-18T03:38:42.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.693+0000 7fd36adff700 1 -- 172.21.15.49:0/3045353458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3640cc420 con 0x7fd36415c950 2023-07-18T03:38:42.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.694+0000 7fd36adff700 1 -- 172.21.15.49:0/3045353458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36404e280 con 0x7fd36415c950 2023-07-18T03:38:42.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.695+0000 7fd3617fa700 1 -- 172.21.15.49:0/3045353458 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd35c01d5d0 con 0x7fd36415c950 2023-07-18T03:38:42.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.696+0000 7fd3617fa700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd34806ace0 0x7fd34806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:42.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.697+0000 7fd3617fa700 1 -- 172.21.15.49:0/3045353458 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd35c08b0c0 con 0x7fd36415c950 2023-07-18T03:38:42.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.697+0000 7fd368b9b700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd34806ace0 0x7fd34806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:42.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.698+0000 7fd368b9b700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd34806ace0 0x7fd34806d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd358007fd0 tx=0x7fd358009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:42.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.701+0000 7fd3617fa700 1 -- 172.21.15.49:0/3045353458 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd35c0561e0 con 0x7fd36415c950 2023-07-18T03:38:42.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.915+0000 7fd36adff700 1 -- 172.21.15.49:0/3045353458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd36408c5a0 con 0x7fd36415c950 2023-07-18T03:38:42.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.916+0000 7fd3617fa700 1 -- 172.21.15.49:0/3045353458 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18185 (secure 0 0 0) 0x7fd35c059800 con 0x7fd36415c950 2023-07-18T03:38:42.911 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:42.911 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:37.510006+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:42.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.918+0000 7fd36adff700 1 -- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd34806ace0 msgr2=0x7fd34806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.918+0000 7fd36adff700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd34806ace0 0x7fd34806d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd358007fd0 tx=0x7fd358009040 comp rx=0 tx=0).stop 2023-07-18T03:38:42.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.918+0000 7fd36adff700 1 -- 172.21.15.49:0/3045353458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd36415c950 msgr2=0x7fd364152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.918+0000 7fd36adff700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd36415c950 0x7fd364152680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00b370 tx=0x7fd35c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.919+0000 7fd36adff700 1 -- 172.21.15.49:0/3045353458 shutdown_connections 2023-07-18T03:38:42.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.919+0000 7fd36adff700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd34806ace0 0x7fd34806d190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.919+0000 7fd36adff700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd36415c950 0x7fd364152680 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.919+0000 7fd36adff700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3640b1e00 0x7fd36415f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.919+0000 7fd36adff700 1 --2- 172.21.15.49:0/3045353458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3640d1220 0x7fd3640cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.919+0000 7fd36adff700 1 -- 172.21.15.49:0/3045353458 >> 172.21.15.49:0/3045353458 conn(0x7fd36409c1d0 msgr2=0x7fd3641585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:42.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.919+0000 7fd36adff700 1 -- 172.21.15.49:0/3045353458 shutdown_connections 2023-07-18T03:38:42.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:42.919+0000 7fd36adff700 1 -- 172.21.15.49:0/3045353458 wait complete. 2023-07-18T03:38:42.932 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-07-18T03:38:42.933 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-07-18T03:38:42.933 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-07-18T03:38:42.934 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-07-18T03:38:42.934 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-07-18T03:38:42.935 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-07-18T03:38:42.935 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-07-18T03:38:42.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-07-18T03:38:43.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.314+0000 7fc828dff700 1 Processor -- start 2023-07-18T03:38:43.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.315+0000 7fc828dff700 1 -- start start 2023-07-18T03:38:43.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.317+0000 7fc828dff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc8240ae0c0 0x7fc82406d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.317+0000 7fc828dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8240cc850 0x7fc8240cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.317+0000 7fc828dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8240cbe70 con 0x7fc8240cba10 2023-07-18T03:38:43.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.317+0000 7fc828dff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82406bfc0 con 0x7fc8240ae0c0 2023-07-18T03:38:43.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.317+0000 7fc828dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240ca120 con 0x7fc8240cc850 2023-07-18T03:38:43.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.318+0000 7fc822d9d700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fc8240cba10 0x7fc824160200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:53430/0 (socket says 172.21.15.49:53430) 2023-07-18T03:38:43.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.318+0000 7fc822d9d700 1 -- 172.21.15.49:0/1917993815 learned_addr learned my addr 172.21.15.49:0/1917993815 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:43.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.318+0000 7f7d3f5ff700 1 Processor -- start 2023-07-18T03:38:43.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.318+0000 7f7d3f5ff700 1 -- start start 2023-07-18T03:38:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.318+0000 7fc821d9b700 1 --2- 172.21.15.49:0/1917993815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8240cc850 0x7fc8240cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.318+0000 7fc82259c700 1 --2- 172.21.15.49:0/1917993815 >> v2:172.21.15.49:3300/0 conn(0x7fc8240ae0c0 0x7fc82406d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.319+0000 7fc82159a700 1 -- 172.21.15.49:0/1917993815 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235092900 0 0) 0x7fc8240cbe70 con 0x7fc8240cba10 2023-07-18T03:38:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.319+0000 7fc82159a700 1 -- 172.21.15.49:0/1917993815 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc80c0035b0 con 0x7fc8240cba10 2023-07-18T03:38:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.319+0000 7fc82259c700 1 -- 172.21.15.49:0/1917993815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8240cc850 msgr2=0x7fc8240cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.319+0000 7fc82259c700 1 --2- 172.21.15.49:0/1917993815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8240cc850 0x7fc8240cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.319+0000 7fc82259c700 1 -- 172.21.15.49:0/1917993815 >> v1:172.21.15.107:6789/0 conn(0x7fc8240cba10 legacy=0x7fc824160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.319+0000 7fc82259c700 1 -- 172.21.15.49:0/1917993815 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82415cb10 con 0x7fc8240ae0c0 2023-07-18T03:38:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.320+0000 7fc82259c700 1 --2- 172.21.15.49:0/1917993815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8240ae0c0 0x7fc82406d040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc818009840 tx=0x7fc818009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0821ad8995c54eb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.320+0000 7fc82159a700 1 -- 172.21.15.49:0/1917993815 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818004230 con 0x7fc8240ae0c0 2023-07-18T03:38:43.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.320+0000 7fc82159a700 1 -- 172.21.15.49:0/1917993815 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc818004390 con 0x7fc8240ae0c0 2023-07-18T03:38:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.320+0000 7fc82159a700 1 -- 172.21.15.49:0/1917993815 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8180036a0 con 0x7fc8240ae0c0 2023-07-18T03:38:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.322+0000 7fc828dff700 1 -- 172.21.15.49:0/1917993815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8240ae0c0 msgr2=0x7fc82406d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.322+0000 7fc828dff700 1 --2- 172.21.15.49:0/1917993815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8240ae0c0 0x7fc82406d040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc818009840 tx=0x7fc818009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.322+0000 7f7d3f5ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7d380cb360 0x7f7d380cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.322+0000 7f7d3f5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7d38157700 0x7f7d38157b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.322+0000 7f7d3f5ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3806b8e0 con 0x7f7d380cab10 2023-07-18T03:38:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.322+0000 7f7d3f5ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d38071b10 con 0x7f7d380cb360 2023-07-18T03:38:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.322+0000 7f7d3f5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3806bbc0 con 0x7f7d38157700 2023-07-18T03:38:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.321+0000 7f993e5ff700 1 Processor -- start 2023-07-18T03:38:43.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.321+0000 7f993e5ff700 1 -- start start 2023-07-18T03:38:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.323+0000 7f7d3db9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7d38157700 0x7f7d38157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.323+0000 7f7d3d39b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7d380cab10 0x7f7d3815b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41562/0 (socket says 172.21.15.49:41562) 2023-07-18T03:38:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.323+0000 7f7d3d39b700 1 -- 172.21.15.49:0/702743083 learned_addr learned my addr 172.21.15.49:0/702743083 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:43.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.323+0000 7f7d3cb9a700 1 --2- 172.21.15.49:0/702743083 >> v2:172.21.15.49:3300/0 conn(0x7f7d380cb360 0x7f7d380cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.323+0000 7fb4683ff700 1 Processor -- start 2023-07-18T03:38:43.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.323+0000 7fb4683ff700 1 -- start start 2023-07-18T03:38:43.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7fc828dff700 1 -- 172.21.15.49:0/1917993815 shutdown_connections 2023-07-18T03:38:43.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7fc828dff700 1 --2- 172.21.15.49:0/1917993815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8240ae0c0 0x7fc82406d040 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7fc828dff700 1 --2- 172.21.15.49:0/1917993815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8240cc850 0x7fc8240cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7fc828dff700 1 -- 172.21.15.49:0/1917993815 >> 172.21.15.49:0/1917993815 conn(0x7fc8240b05d0 msgr2=0x7fc8240b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7fc828dff700 1 -- 172.21.15.49:0/1917993815 shutdown_connections 2023-07-18T03:38:43.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7f7d3db9c700 1 -- 172.21.15.49:0/702743083 >> v2:172.21.15.49:3300/0 conn(0x7f7d380cb360 msgr2=0x7f7d380cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7f7d3db9c700 1 --2- 172.21.15.49:0/702743083 >> v2:172.21.15.49:3300/0 conn(0x7f7d380cb360 0x7f7d380cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7f7d3db9c700 1 -- 172.21.15.49:0/702743083 >> v1:172.21.15.49:6789/0 conn(0x7f7d380cab10 legacy=0x7f7d3815b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7f7d3db9c700 1 -- 172.21.15.49:0/702743083 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3806bea0 con 0x7f7d38157700 2023-07-18T03:38:43.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7f7d2ffff700 1 -- 172.21.15.49:0/702743083 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476750615 0 0) 0x7f7d3806b8e0 con 0x7f7d380cab10 2023-07-18T03:38:43.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7f993e5ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9938160c30 0x7f99380cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7f993e5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99380cdbe0 0x7f99380ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7f993e5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99380ce570 con 0x7f9938164710 2023-07-18T03:38:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7f993e5ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9938161000 con 0x7f9938160c30 2023-07-18T03:38:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7f993e5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9938165400 con 0x7f99380cdbe0 2023-07-18T03:38:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.325+0000 7f713e9ff700 1 Processor -- start 2023-07-18T03:38:43.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.325+0000 7f9937fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9938160c30 0x7f99380cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.325+0000 7f9937fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9938160c30 0x7f99380cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40308/0 (socket says 172.21.15.49:40308) 2023-07-18T03:38:43.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.325+0000 7f9937fff700 1 -- 172.21.15.49:0/442990674 learned_addr learned my addr 172.21.15.49:0/442990674 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7f9937fff700 1 -- 172.21.15.49:0/442990674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99380cdbe0 msgr2=0x7f99380ce030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:38:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7f9937fff700 1 --2- 172.21.15.49:0/442990674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99380cdbe0 0x7f99380ce030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7f9937fff700 1 -- 172.21.15.49:0/442990674 >> v1:172.21.15.107:6789/0 conn(0x7f9938164710 legacy=0x7f9938164b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7f9937fff700 1 -- 172.21.15.49:0/442990674 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99380cc810 con 0x7f9938160c30 2023-07-18T03:38:43.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7f7d3db9c700 1 --2- 172.21.15.49:0/702743083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d38157700 0x7f7d38157b80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d3806f7c0 tx=0x7f7d3000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec323aa5fa8baa66 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7f9937fff700 1 --2- 172.21.15.49:0/442990674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9938160c30 0x7f99380cd2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f992c0098f0 tx=0x7f992c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b472b3aade68f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.325+0000 7fb4683ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4600cd940 0x7fb4600cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7fb4683ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb46006cd60 0x7fb4600cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7fb4683ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4600cd530 con 0x7fb460160d50 2023-07-18T03:38:43.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7fb4683ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4601615d0 con 0x7fb46006cd60 2023-07-18T03:38:43.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7fb4683ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600ce250 con 0x7fb4600cd940 2023-07-18T03:38:43.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7fb46699c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb46006cd60 0x7fb4600cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7fb46699c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb46006cd60 0x7fb4600cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45106/0 (socket says 172.21.15.49:45106) 2023-07-18T03:38:43.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7fb46699c700 1 -- 172.21.15.49:0/3950278182 learned_addr learned my addr 172.21.15.49:0/3950278182 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.326+0000 7fb46599a700 1 --1- 172.21.15.49:0/3950278182 >> v1:172.21.15.49:6789/0 conn(0x7fb460160d50 0x7fb460161a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41574/0 (socket says 172.21.15.49:41574) 2023-07-18T03:38:43.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7fb46619b700 1 --2- 172.21.15.49:0/3950278182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4600cd940 0x7fb4600cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7f9936ffd700 1 -- 172.21.15.49:0/442990674 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c004230 con 0x7f9938160c30 2023-07-18T03:38:43.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7f9936ffd700 1 -- 172.21.15.49:0/442990674 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c004390 con 0x7f9938160c30 2023-07-18T03:38:43.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7f9936ffd700 1 -- 172.21.15.49:0/442990674 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c0036a0 con 0x7f9938160c30 2023-07-18T03:38:43.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.325+0000 7f713e9ff700 1 -- start start 2023-07-18T03:38:43.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7fb46699c700 1 -- 172.21.15.49:0/3950278182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4600cd940 msgr2=0x7fb4600cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7fb46699c700 1 --2- 172.21.15.49:0/3950278182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4600cd940 0x7fb4600cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7fb46699c700 1 -- 172.21.15.49:0/3950278182 >> v1:172.21.15.49:6789/0 conn(0x7fb460160d50 legacy=0x7fb460161a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7fb46699c700 1 -- 172.21.15.49:0/3950278182 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4600ccf70 con 0x7fb46006cd60 2023-07-18T03:38:43.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7f713e9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7138160d50 0x7f7138161a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7f713e9ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f713806cd60 0x7f71380cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7f713e9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71380cd530 con 0x7f71380cd940 2023-07-18T03:38:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7f713e9ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71381615d0 con 0x7f713806cd60 2023-07-18T03:38:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.327+0000 7f713e9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71380ccf70 con 0x7f7138160d50 2023-07-18T03:38:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7f71377fe700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7138160d50 0x7f7138161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.324+0000 7fc828dff700 1 -- 172.21.15.49:0/1917993815 wait complete. 2023-07-18T03:38:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7f7d2ffff700 1 -- 172.21.15.49:0/702743083 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000e070 con 0x7f7d38157700 2023-07-18T03:38:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7f7d2ffff700 1 -- 172.21.15.49:0/702743083 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3000bb20 con 0x7f7d38157700 2023-07-18T03:38:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7f7d2ffff700 1 -- 172.21.15.49:0/702743083 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30012430 con 0x7f7d38157700 2023-07-18T03:38:43.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7f71377fe700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7138160d50 0x7f7138161a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45108/0 (socket says 172.21.15.49:45108) 2023-07-18T03:38:43.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7f71377fe700 1 -- 172.21.15.49:0/2833343550 learned_addr learned my addr 172.21.15.49:0/2833343550 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7fc828dff700 1 Processor -- start 2023-07-18T03:38:43.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7fb46699c700 1 --2- 172.21.15.49:0/3950278182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb46006cd60 0x7fb4600cc690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb458009e50 tx=0x7fb45800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1353d9317e617249 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7f71377fe700 1 -- 172.21.15.49:0/2833343550 >> v2:172.21.15.49:3300/0 conn(0x7f713806cd60 msgr2=0x7f71380cc690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:38:43.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7f71377fe700 1 --2- 172.21.15.49:0/2833343550 >> v2:172.21.15.49:3300/0 conn(0x7f713806cd60 0x7f71380cc690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7f71377fe700 1 -- 172.21.15.49:0/2833343550 >> v1:172.21.15.107:6789/0 conn(0x7f71380cd940 legacy=0x7f71380cdd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:38:43.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7f71377fe700 1 -- 172.21.15.49:0/2833343550 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71380cd250 con 0x7f7138160d50 2023-07-18T03:38:43.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.329+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/702743083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d38157700 msgr2=0x7f7d38157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.329+0000 7f7d3f5ff700 1 --2- 172.21.15.49:0/702743083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d38157700 0x7f7d38157b80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d3806f7c0 tx=0x7f7d3000b330 comp rx=0 tx=0).stop 2023-07-18T03:38:43.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.329+0000 7fb465199700 1 -- 172.21.15.49:0/3950278182 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800e070 con 0x7fb46006cd60 2023-07-18T03:38:43.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.329+0000 7fb465199700 1 -- 172.21.15.49:0/3950278182 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45800b9c0 con 0x7fb46006cd60 2023-07-18T03:38:43.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.329+0000 7fb465199700 1 -- 172.21.15.49:0/3950278182 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800bd50 con 0x7fb46006cd60 2023-07-18T03:38:43.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.329+0000 7f71377fe700 1 --2- 172.21.15.49:0/2833343550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138160d50 0x7f7138161a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71380cea20 tx=0x7f712c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad69bb343b5e7819 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.329+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/702743083 shutdown_connections 2023-07-18T03:38:43.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.329+0000 7f7d3f5ff700 1 --2- 172.21.15.49:0/702743083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d38157700 0x7f7d38157b80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.329+0000 7f7d3f5ff700 1 --2- 172.21.15.49:0/702743083 >> v2:172.21.15.49:3300/0 conn(0x7f7d380cb360 0x7f7d380cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.329+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/702743083 >> 172.21.15.49:0/702743083 conn(0x7f7d380b05d0 msgr2=0x7f7d380b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.330+0000 7f993e5ff700 1 -- 172.21.15.49:0/442990674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9938160c30 msgr2=0x7f99380cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.330+0000 7f993e5ff700 1 --2- 172.21.15.49:0/442990674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9938160c30 0x7f99380cd2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f992c0098f0 tx=0x7f992c009c00 comp rx=0 tx=0).stop 2023-07-18T03:38:43.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.328+0000 7fc828dff700 1 -- start start 2023-07-18T03:38:43.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.330+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/702743083 shutdown_connections 2023-07-18T03:38:43.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.330+0000 7fb4683ff700 1 -- 172.21.15.49:0/3950278182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb46006cd60 msgr2=0x7fb4600cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.330+0000 7fb4683ff700 1 --2- 172.21.15.49:0/3950278182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb46006cd60 0x7fb4600cc690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb458009e50 tx=0x7fb45800b330 comp rx=0 tx=0).stop 2023-07-18T03:38:43.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7fb4683ff700 1 -- 172.21.15.49:0/3950278182 shutdown_connections 2023-07-18T03:38:43.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7fb4683ff700 1 --2- 172.21.15.49:0/3950278182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4600cd940 0x7fb4600cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7fb4683ff700 1 --2- 172.21.15.49:0/3950278182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb46006cd60 0x7fb4600cc690 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7fb4683ff700 1 -- 172.21.15.49:0/3950278182 >> 172.21.15.49:0/3950278182 conn(0x7fb4600b05d0 msgr2=0x7fb4600b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.330+0000 7fc828dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8240ae0c0 0x7fc82414dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.330+0000 7fc828dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8240cba10 0x7fc8240c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.330+0000 7fc828dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8240cc850 0x7fc8240af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.330+0000 7fc828dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241ff570 con 0x7fc8240ae0c0 2023-07-18T03:38:43.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.330+0000 7fc828dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240c24a0 con 0x7fc8240cc850 2023-07-18T03:38:43.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.330+0000 7fc828dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241fef70 con 0x7fc8240cba10 2023-07-18T03:38:43.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7fc82259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8240ae0c0 0x7fc82414dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7fc82259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8240ae0c0 0x7fc82414dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40316/0 (socket says 172.21.15.49:40316) 2023-07-18T03:38:43.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7fc82259c700 1 -- 172.21.15.49:0/1854362433 learned_addr learned my addr 172.21.15.49:0/1854362433 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7f7136ffd700 1 -- 172.21.15.49:0/2833343550 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c004230 con 0x7f7138160d50 2023-07-18T03:38:43.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7f7136ffd700 1 -- 172.21.15.49:0/2833343550 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c004390 con 0x7f7138160d50 2023-07-18T03:38:43.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7f7136ffd700 1 -- 172.21.15.49:0/2833343550 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0036a0 con 0x7f7138160d50 2023-07-18T03:38:43.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7fc821d9b700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8240cba10 0x7fc8240c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7fb4683ff700 1 -- 172.21.15.49:0/3950278182 shutdown_connections 2023-07-18T03:38:43.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7fc822d9d700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8240cc850 0x7fc8240af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7fc82259c700 1 -- 172.21.15.49:0/1854362433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8240cba10 msgr2=0x7fc8240c2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7fc82259c700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8240cba10 0x7fc8240c2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7fc82259c700 1 -- 172.21.15.49:0/1854362433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8240cc850 msgr2=0x7fc8240af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7fc82259c700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8240cc850 0x7fc8240af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7fc82259c700 1 -- 172.21.15.49:0/1854362433 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8180094a0 con 0x7fc8240ae0c0 2023-07-18T03:38:43.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7fc821d9b700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8240cba10 0x7fc8240c2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:43.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.333+0000 7fc82259c700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8240ae0c0 0x7fc82414dd50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc8180090b0 tx=0x7fc818004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.330+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/702743083 wait complete. 2023-07-18T03:38:43.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.333+0000 7f7d3f5ff700 1 Processor -- start 2023-07-18T03:38:43.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7f713e9ff700 1 -- 172.21.15.49:0/2833343550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138160d50 msgr2=0x7f7138161a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7f713e9ff700 1 --2- 172.21.15.49:0/2833343550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138160d50 0x7f7138161a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71380cea20 tx=0x7f712c009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7f713e9ff700 1 -- 172.21.15.49:0/2833343550 shutdown_connections 2023-07-18T03:38:43.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7f713e9ff700 1 --2- 172.21.15.49:0/2833343550 >> v2:172.21.15.49:3300/0 conn(0x7f713806cd60 0x7f71380cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7f713e9ff700 1 --2- 172.21.15.49:0/2833343550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138160d50 0x7f7138161a60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7f713e9ff700 1 -- 172.21.15.49:0/2833343550 >> 172.21.15.49:0/2833343550 conn(0x7f71380b05d0 msgr2=0x7f71380b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.333+0000 7f713e9ff700 1 -- 172.21.15.49:0/2833343550 shutdown_connections 2023-07-18T03:38:43.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.333+0000 7f713e9ff700 1 -- 172.21.15.49:0/2833343550 wait complete. 2023-07-18T03:38:43.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.331+0000 7fb4683ff700 1 -- 172.21.15.49:0/3950278182 wait complete. 2023-07-18T03:38:43.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7f993e5ff700 1 -- 172.21.15.49:0/442990674 shutdown_connections 2023-07-18T03:38:43.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7f993e5ff700 1 --2- 172.21.15.49:0/442990674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9938160c30 0x7f99380cd2f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7f993e5ff700 1 --2- 172.21.15.49:0/442990674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99380cdbe0 0x7f99380ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7f993e5ff700 1 -- 172.21.15.49:0/442990674 >> 172.21.15.49:0/442990674 conn(0x7f99380b05d0 msgr2=0x7f99380b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7f993e5ff700 1 -- 172.21.15.49:0/442990674 shutdown_connections 2023-07-18T03:38:43.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.334+0000 7fc812ffd700 1 -- 172.21.15.49:0/1854362433 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818003e60 con 0x7fc8240ae0c0 2023-07-18T03:38:43.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.334+0000 7fc812ffd700 1 -- 172.21.15.49:0/1854362433 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8180045c0 con 0x7fc8240ae0c0 2023-07-18T03:38:43.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.334+0000 7fc812ffd700 1 -- 172.21.15.49:0/1854362433 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818011830 con 0x7fc8240ae0c0 2023-07-18T03:38:43.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.334+0000 7fc828dff700 1 -- 172.21.15.49:0/1854362433 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82414e290 con 0x7fc8240ae0c0 2023-07-18T03:38:43.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.334+0000 7fc828dff700 1 -- 172.21.15.49:0/1854362433 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8241f3020 con 0x7fc8240ae0c0 2023-07-18T03:38:43.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.334+0000 7fb4683ff700 1 Processor -- start 2023-07-18T03:38:43.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.332+0000 7f993e5ff700 1 -- 172.21.15.49:0/442990674 wait complete. 2023-07-18T03:38:43.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.335+0000 7f713e9ff700 1 Processor -- start 2023-07-18T03:38:43.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.336+0000 7f713e9ff700 1 -- start start 2023-07-18T03:38:43.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.336+0000 7f993e5ff700 1 Processor -- start 2023-07-18T03:38:43.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.333+0000 7f7d3f5ff700 1 -- start start 2023-07-18T03:38:43.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.335+0000 7f7d3f5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d380cab10 0x7f7d380b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.335+0000 7f7d3f5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d380cb360 0x7f7d3814f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.335+0000 7f7d3f5ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d38157700 0x7f7d3814fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.335+0000 7f7d3f5ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d38150190 con 0x7f7d38157700 2023-07-18T03:38:43.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.335+0000 7f7d3f5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d380af800 con 0x7f7d380cb360 2023-07-18T03:38:43.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.335+0000 7f7d3f5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d38150380 con 0x7f7d380cab10 2023-07-18T03:38:43.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.335+0000 7fb4683ff700 1 -- start start 2023-07-18T03:38:43.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.336+0000 7fb4683ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb46006cd60 0x7fb460147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.336+0000 7fb4683ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4600cd940 0x7fb4601623c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.336+0000 7fb4683ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb460160d50 0x7fb460162900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.336+0000 7fb4683ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb460147db0 con 0x7fb4600cd940 2023-07-18T03:38:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.336+0000 7fb4683ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600c5070 con 0x7fb460160d50 2023-07-18T03:38:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.336+0000 7fb4683ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600c5500 con 0x7fb46006cd60 2023-07-18T03:38:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7f7d3cb9a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d380cb360 0x7f7d3814f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7f7d3d39b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d380cab10 0x7f7d380b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7f7d3d39b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d380cab10 0x7f7d380b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46598/0 (socket says 172.21.15.49:46598) 2023-07-18T03:38:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7f7d3d39b700 1 -- 172.21.15.49:0/202921649 learned_addr learned my addr 172.21.15.49:0/202921649 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7f7d3db9c700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d38157700 0x7f7d3814fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7fb46619b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb46006cd60 0x7fb460147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7fb46699c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb460160d50 0x7fb460162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7fb46699c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb460160d50 0x7fb460162900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45142/0 (socket says 172.21.15.49:45142) 2023-07-18T03:38:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7fb46699c700 1 -- 172.21.15.49:0/4158576686 learned_addr learned my addr 172.21.15.49:0/4158576686 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7fb46599a700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4600cd940 0x7fb4601623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.336+0000 7f993e5ff700 1 -- start start 2023-07-18T03:38:43.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f993e5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99380cdbe0 0x7f993815ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f993e5ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9938160c30 0x7f99381636e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f993e5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9938164710 0x7f9938163c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f993e5ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9938161f20 con 0x7f9938160c30 2023-07-18T03:38:43.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f993e5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9938162200 con 0x7f99380cdbe0 2023-07-18T03:38:43.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f993e5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99381624e0 con 0x7f9938164710 2023-07-18T03:38:43.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f7d3d39b700 1 -- 172.21.15.49:0/202921649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d380cb360 msgr2=0x7f7d3814f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f7d3d39b700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d380cb360 0x7f7d3814f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f7d3d39b700 1 -- 172.21.15.49:0/202921649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d38157700 msgr2=0x7f7d3814fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f7d3d39b700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d38157700 0x7f7d3814fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f7d3d39b700 1 -- 172.21.15.49:0/202921649 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d30009a70 con 0x7f7d380cab10 2023-07-18T03:38:43.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7fb46619b700 1 -- 172.21.15.49:0/4158576686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb460160d50 msgr2=0x7fb460162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f9937fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99380cdbe0 0x7f993815ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7f713e9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f713806cd60 0x7f7138143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7f713e9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71380cd940 0x7f71380c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7f713e9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138160d50 0x7f7138150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7f713e9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138144510 con 0x7f713806cd60 2023-07-18T03:38:43.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7f713e9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71380c4a80 con 0x7f7138160d50 2023-07-18T03:38:43.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.337+0000 7f713e9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71380c3e70 con 0x7f71380cd940 2023-07-18T03:38:43.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f713cf9c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138160d50 0x7f7138150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f713cf9c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138160d50 0x7f7138150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45150/0 (socket says 172.21.15.49:45150) 2023-07-18T03:38:43.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f713cf9c700 1 -- 172.21.15.49:0/925385100 learned_addr learned my addr 172.21.15.49:0/925385100 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.338+0000 7f71377fe700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71380cd940 0x7f71380c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f71377fe700 1 -- 172.21.15.49:0/925385100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138160d50 msgr2=0x7f7138150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f71377fe700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138160d50 0x7f7138150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f71377fe700 1 -- 172.21.15.49:0/925385100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f713806cd60 msgr2=0x7f7138143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f71377fe700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f713806cd60 0x7f7138143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f71377fe700 1 -- 172.21.15.49:0/925385100 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c009550 con 0x7f71380cd940 2023-07-18T03:38:43.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7fb46619b700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb460160d50 0x7fb460162900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7fb46619b700 1 -- 172.21.15.49:0/4158576686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4600cd940 msgr2=0x7fb4601623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7fb46619b700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4600cd940 0x7fb4601623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7fb46619b700 1 -- 172.21.15.49:0/4158576686 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458009b30 con 0x7fb46006cd60 2023-07-18T03:38:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7fb46699c700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb460160d50 0x7fb460162900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:43.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f9937fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99380cdbe0 0x7f993815ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45154/0 (socket says 172.21.15.49:45154) 2023-07-18T03:38:43.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f9937fff700 1 -- 172.21.15.49:0/1434363766 learned_addr learned my addr 172.21.15.49:0/1434363766 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f7d3d39b700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d380cab10 0x7f7d380b4410 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400c550 tx=0x7f7d3400c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f7d2dffb700 1 -- 172.21.15.49:0/202921649 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34004c80 con 0x7f7d380cab10 2023-07-18T03:38:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f7d2dffb700 1 -- 172.21.15.49:0/202921649 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d34004de0 con 0x7f7d380cab10 2023-07-18T03:38:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f7d2dffb700 1 -- 172.21.15.49:0/202921649 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34007310 con 0x7f7d380cab10 2023-07-18T03:38:43.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f99377fe700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9938160c30 0x7f99381636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.336+0000 7fc828dff700 1 -- 172.21.15.49:0/1854362433 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc808000fc0 con 0x7fc8240ae0c0 2023-07-18T03:38:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/202921649 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d380aeed0 con 0x7f7d380cab10 2023-07-18T03:38:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f993cb9c700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9938164710 0x7f9938163c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.339+0000 7f9937fff700 1 -- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9938164710 msgr2=0x7f9938163c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7f9937fff700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9938164710 0x7f9938163c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7f9937fff700 1 -- 172.21.15.49:0/1434363766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9938160c30 msgr2=0x7f99381636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7f9937fff700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9938160c30 0x7f99381636e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7f9937fff700 1 -- 172.21.15.49:0/1434363766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c009550 con 0x7f99380cdbe0 2023-07-18T03:38:43.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/202921649 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d380af490 con 0x7f7d380cab10 2023-07-18T03:38:43.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7f71377fe700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71380cd940 0x7f71380c4f10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f712c015000 tx=0x7f712c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7f7134ff9700 1 -- 172.21.15.49:0/925385100 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c011630 con 0x7f71380cd940 2023-07-18T03:38:43.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7f7134ff9700 1 -- 172.21.15.49:0/925385100 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c011790 con 0x7f71380cd940 2023-07-18T03:38:43.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7f7134ff9700 1 -- 172.21.15.49:0/925385100 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c011b00 con 0x7f71380cd940 2023-07-18T03:38:43.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7f713e9ff700 1 -- 172.21.15.49:0/925385100 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71380c4150 con 0x7f71380cd940 2023-07-18T03:38:43.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7fb46599a700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4600cd940 0x7fb4601623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:43.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7fb46619b700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb46006cd60 0x7fb460147870 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00c820 tx=0x7fb45c00cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.340+0000 7f713e9ff700 1 -- 172.21.15.49:0/925385100 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71380c4710 con 0x7f71380cd940 2023-07-18T03:38:43.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.341+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/202921649 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d3806c460 con 0x7f7d380cab10 2023-07-18T03:38:43.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.342+0000 7f9937fff700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99380cdbe0 0x7f993815ffe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f992c000c00 tx=0x7f992c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.342+0000 7fc812ffd700 1 -- 172.21.15.49:0/1854362433 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc818018050 con 0x7fc8240ae0c0 2023-07-18T03:38:43.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.344+0000 7f713e9ff700 1 -- 172.21.15.49:0/925385100 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7138161fa0 con 0x7f71380cd940 2023-07-18T03:38:43.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.344+0000 7f7d2dffb700 1 -- 172.21.15.49:0/202921649 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d34007470 con 0x7f7d380cab10 2023-07-18T03:38:43.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.344+0000 7f7134ff9700 1 -- 172.21.15.49:0/925385100 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f712c021070 con 0x7f71380cd940 2023-07-18T03:38:43.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.344+0000 7fc812ffd700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc80406adb0 0x7fc80406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.344+0000 7fc812ffd700 1 -- 172.21.15.49:0/1854362433 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc818089d60 con 0x7fc8240ae0c0 2023-07-18T03:38:43.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7fc812ffd700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] conn(0x7fc804073660 0x7fc804075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7fc812ffd700 1 -- 172.21.15.49:0/1854362433 --> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc804076120 con 0x7fc804073660 2023-07-18T03:38:43.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7fc812ffd700 1 -- 172.21.15.49:0/1854362433 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc81808c380 con 0x7fc8240ae0c0 2023-07-18T03:38:43.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7fc821d9b700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc80406adb0 0x7fc80406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7fc822d9d700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] conn(0x7fc804073660 0x7fc804075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.346+0000 7fc822d9d700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] conn(0x7fc804073660 0x7fc804075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7f7d2dffb700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7d1c06ad30 0x7f7d1c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7f7d2dffb700 1 -- 172.21.15.49:0/202921649 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d340895f0 con 0x7f7d380cab10 2023-07-18T03:38:43.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.346+0000 7f7d2dffb700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] conn(0x7f7d1c0735e0 0x7f7d1c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.346+0000 7f7d2dffb700 1 -- 172.21.15.49:0/202921649 --> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d1c0760a0 con 0x7f7d1c0735e0 2023-07-18T03:38:43.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.346+0000 7f7d2dffb700 1 -- 172.21.15.49:0/202921649 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d34089960 con 0x7f7d380cab10 2023-07-18T03:38:43.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7fb456ffd700 1 -- 172.21.15.49:0/4158576686 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c004220 con 0x7fb46006cd60 2023-07-18T03:38:43.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7fb456ffd700 1 -- 172.21.15.49:0/4158576686 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c004380 con 0x7fb46006cd60 2023-07-18T03:38:43.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7fb456ffd700 1 -- 172.21.15.49:0/4158576686 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0036a0 con 0x7fb46006cd60 2023-07-18T03:38:43.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7fb4683ff700 1 -- 172.21.15.49:0/4158576686 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4600c4740 con 0x7fb46006cd60 2023-07-18T03:38:43.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7fb4683ff700 1 -- 172.21.15.49:0/4158576686 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4600c4d00 con 0x7fb46006cd60 2023-07-18T03:38:43.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7f9934ff9700 1 -- 172.21.15.49:0/1434363766 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c003db0 con 0x7f99380cdbe0 2023-07-18T03:38:43.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7f9934ff9700 1 -- 172.21.15.49:0/1434363766 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c01f070 con 0x7f99380cdbe0 2023-07-18T03:38:43.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.345+0000 7f9934ff9700 1 -- 172.21.15.49:0/1434363766 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c00b880 con 0x7f99380cdbe0 2023-07-18T03:38:43.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.346+0000 7fed64dff700 1 Processor -- start 2023-07-18T03:38:43.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.346+0000 7f993e5ff700 1 -- 172.21.15.49:0/1434363766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99381627c0 con 0x7f99380cdbe0 2023-07-18T03:38:43.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.346+0000 7f993e5ff700 1 -- 172.21.15.49:0/1434363766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9938160520 con 0x7f99380cdbe0 2023-07-18T03:38:43.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.347+0000 7f7d3db9c700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] conn(0x7f7d1c0735e0 0x7f7d1c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.346+0000 7f7134ff9700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f712406ad30 0x7f712406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.346+0000 7f7134ff9700 1 -- 172.21.15.49:0/925385100 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f712c08a7e0 con 0x7f71380cd940 2023-07-18T03:38:43.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.347+0000 7f7134ff9700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] conn(0x7f71240735e0 0x7f71240759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.347+0000 7f7134ff9700 1 -- 172.21.15.49:0/925385100 --> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71240760a0 con 0x7f71240735e0 2023-07-18T03:38:43.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.347+0000 7f7134ff9700 1 -- 172.21.15.49:0/925385100 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f712c08f2c0 con 0x7f71380cd940 2023-07-18T03:38:43.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.347+0000 7f7137fff700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f712406ad30 0x7f712406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.347+0000 7f713cf9c700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] conn(0x7f71240735e0 0x7f71240759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.348+0000 7fc812ffd700 1 -- 172.21.15.49:0/1854362433 <== osd.0 v2:172.21.15.49:6800/990566862 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc804076120 con 0x7fc804073660 2023-07-18T03:38:43.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.348+0000 7fb4683ff700 1 -- 172.21.15.49:0/4158576686 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb460161260 con 0x7fb46006cd60 2023-07-18T03:38:43.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.347+0000 7f993e5ff700 1 -- 172.21.15.49:0/1434363766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9938162370 con 0x7f99380cdbe0 2023-07-18T03:38:43.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.348+0000 7fc821d9b700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc80406adb0 0x7fc80406d260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8240afff0 tx=0x7fc814009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.348+0000 7f7d3db9c700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] conn(0x7f7d1c0735e0 0x7f7d1c0759f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.349+0000 7f7137fff700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f712406ad30 0x7f712406d1e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f71380cbc60 tx=0x7f7128006cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.349+0000 7f713cf9c700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] conn(0x7f71240735e0 0x7f71240759f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.346+0000 7fed64dff700 1 -- start start 2023-07-18T03:38:43.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.349+0000 7fed64dff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fed6006cd60 0x7fed600cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.349+0000 7fed64dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed600cd530 con 0x7fed600cd940 2023-07-18T03:38:43.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.349+0000 7fed64dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed601615d0 con 0x7fed60160d50 2023-07-18T03:38:43.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.349+0000 7fed64dff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed600ccf70 con 0x7fed6006cd60 2023-07-18T03:38:43.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.349+0000 7f7d3cb9a700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7d1c06ad30 0x7f7d1c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.349+0000 7fed5e59c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fed600cd940 0x7fed600cdd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41582/0 (socket says 172.21.15.49:41582) 2023-07-18T03:38:43.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.350+0000 7fed5e59c700 1 -- 172.21.15.49:0/2480827555 learned_addr learned my addr 172.21.15.49:0/2480827555 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:43.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.350+0000 7fb456ffd700 1 -- 172.21.15.49:0/4158576686 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb45c003820 con 0x7fb46006cd60 2023-07-18T03:38:43.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.350+0000 7f7d2dffb700 1 -- 172.21.15.49:0/202921649 <== osd.1 v2:172.21.15.49:6815/2328014854 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7d1c0760a0 con 0x7f7d1c0735e0 2023-07-18T03:38:43.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.350+0000 7fed5ed9d700 1 --2- 172.21.15.49:0/2480827555 >> v2:172.21.15.49:3300/0 conn(0x7fed6006cd60 0x7fed600cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.350+0000 7fed5d59a700 1 -- 172.21.15.49:0/2480827555 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327378637 0 0) 0x7fed600cd530 con 0x7fed600cd940 2023-07-18T03:38:43.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.351+0000 7fed5d59a700 1 -- 172.21.15.49:0/2480827555 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed480035b0 con 0x7fed600cd940 2023-07-18T03:38:43.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.351+0000 7fed5ed9d700 1 -- 172.21.15.49:0/2480827555 >> v1:172.21.15.107:6789/0 conn(0x7fed60160d50 legacy=0x7fed60161a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.351+0000 7fed5ed9d700 1 -- 172.21.15.49:0/2480827555 >> v1:172.21.15.49:6789/0 conn(0x7fed600cd940 legacy=0x7fed600cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.351+0000 7fed5ed9d700 1 -- 172.21.15.49:0/2480827555 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed600cd250 con 0x7fed6006cd60 2023-07-18T03:38:43.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.352+0000 7fb456ffd700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb44406adb0 0x7fb44406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.352+0000 7fb456ffd700 1 -- 172.21.15.49:0/4158576686 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb45c088850 con 0x7fb46006cd60 2023-07-18T03:38:43.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.352+0000 7fb456ffd700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] conn(0x7fb444073660 0x7fb444075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.352+0000 7fb456ffd700 1 -- 172.21.15.49:0/4158576686 --> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb444076120 con 0x7fb444073660 2023-07-18T03:38:43.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.352+0000 7fb456ffd700 1 -- 172.21.15.49:0/4158576686 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb45c08d2a0 con 0x7fb46006cd60 2023-07-18T03:38:43.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.352+0000 7fed5ed9d700 1 --2- 172.21.15.49:0/2480827555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6006cd60 0x7fed600cc690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fed54009840 tx=0x7fed54009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5226ec02aa06db16 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.352+0000 7fed5d59a700 1 -- 172.21.15.49:0/2480827555 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54011830 con 0x7fed6006cd60 2023-07-18T03:38:43.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.352+0000 7fed5d59a700 1 -- 172.21.15.49:0/2480827555 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed54011990 con 0x7fed6006cd60 2023-07-18T03:38:43.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.352+0000 7fed5d59a700 1 -- 172.21.15.49:0/2480827555 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54011d00 con 0x7fed6006cd60 2023-07-18T03:38:43.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.351+0000 7f9934ff9700 1 -- 172.21.15.49:0/1434363766 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f992c021070 con 0x7f99380cdbe0 2023-07-18T03:38:43.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.353+0000 7fb46699c700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] conn(0x7fb444073660 0x7fb444075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.353+0000 7fb46599a700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb44406adb0 0x7fb44406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.353+0000 7f9934ff9700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f992406adb0 0x7f992406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.353+0000 7f9934ff9700 1 -- 172.21.15.49:0/1434363766 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f992c08ae30 con 0x7f99380cdbe0 2023-07-18T03:38:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.353+0000 7fed64dff700 1 -- 172.21.15.49:0/2480827555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6006cd60 msgr2=0x7fed600cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.353+0000 7fed64dff700 1 --2- 172.21.15.49:0/2480827555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6006cd60 0x7fed600cc690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fed54009840 tx=0x7fed54009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.354+0000 7fb46699c700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] conn(0x7fb444073660 0x7fb444075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.353+0000 7f9934ff9700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] conn(0x7f9924073660 0x7f9924075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.354+0000 7f9934ff9700 1 -- 172.21.15.49:0/1434363766 --> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9924076120 con 0x7f9924073660 2023-07-18T03:38:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.354+0000 7f9934ff9700 1 -- 172.21.15.49:0/1434363766 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f992c08b1e0 con 0x7f99380cdbe0 2023-07-18T03:38:43.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.354+0000 7fed64dff700 1 -- 172.21.15.49:0/2480827555 shutdown_connections 2023-07-18T03:38:43.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.354+0000 7fed64dff700 1 --2- 172.21.15.49:0/2480827555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6006cd60 0x7fed600cc690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.354+0000 7fed64dff700 1 -- 172.21.15.49:0/2480827555 >> 172.21.15.49:0/2480827555 conn(0x7fed600b05d0 msgr2=0x7fed600b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.354+0000 7f77159ff700 1 Processor -- start 2023-07-18T03:38:43.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.354+0000 7fed64dff700 1 -- 172.21.15.49:0/2480827555 shutdown_connections 2023-07-18T03:38:43.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.355+0000 7f99377fe700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f992406adb0 0x7f992406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.355+0000 7f77159ff700 1 -- start start 2023-07-18T03:38:43.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.355+0000 7f993cb9c700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] conn(0x7f9924073660 0x7f9924075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.355+0000 7f7134ff9700 1 -- 172.21.15.49:0/925385100 <== osd.2 v2:172.21.15.49:6824/46330117 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f71240760a0 con 0x7f71240735e0 2023-07-18T03:38:43.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.355+0000 7fb456ffd700 1 -- 172.21.15.49:0/4158576686 <== osd.3 v2:172.21.15.49:6808/3802584914 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb444076120 con 0x7fb444073660 2023-07-18T03:38:43.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.355+0000 7fed64dff700 1 -- 172.21.15.49:0/2480827555 wait complete. 2023-07-18T03:38:43.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.356+0000 7fb46599a700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb44406adb0 0x7fb44406d260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb46015ee50 tx=0x7fb450007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.357+0000 7f77159ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f77100ae0c0 0x7f771006d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.357+0000 7f77159ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100cc850 0x7f77100cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.357+0000 7f77159ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77100cbe70 con 0x7f77100cba10 2023-07-18T03:38:43.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.357+0000 7f77159ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771006bfc0 con 0x7f77100ae0c0 2023-07-18T03:38:43.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.357+0000 7f77159ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77100ca120 con 0x7f77100cc850 2023-07-18T03:38:43.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.358+0000 7f7d3cb9a700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7d1c06ad30 0x7f7d1c06d1e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d381f9b20 tx=0x7f7d28009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.357+0000 7fed64dff700 1 Processor -- start 2023-07-18T03:38:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.358+0000 7f770effd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f77100ae0c0 0x7f771006d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.358+0000 7f770effd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f77100ae0c0 0x7f771006d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40370/0 (socket says 172.21.15.49:40370) 2023-07-18T03:38:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.358+0000 7f770e7fc700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100cc850 0x7f77100cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.358+0000 7f770effd700 1 -- 172.21.15.49:0/3282513646 learned_addr learned my addr 172.21.15.49:0/3282513646 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.359+0000 7f770effd700 1 -- 172.21.15.49:0/3282513646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100cc850 msgr2=0x7f77100cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.359+0000 7f770effd700 1 --2- 172.21.15.49:0/3282513646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100cc850 0x7f77100cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.359+0000 7f770effd700 1 -- 172.21.15.49:0/3282513646 >> v1:172.21.15.107:6789/0 conn(0x7f77100cba10 legacy=0x7f7710160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.359+0000 7f770effd700 1 -- 172.21.15.49:0/3282513646 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771015cb10 con 0x7f77100ae0c0 2023-07-18T03:38:43.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.359+0000 7fed64dff700 1 -- start start 2023-07-18T03:38:43.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.359+0000 7f770effd700 1 --2- 172.21.15.49:0/3282513646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100ae0c0 0x7f771006d040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f77000098f0 tx=0x7f7700009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cd47276ec32247 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.358+0000 7f993cb9c700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] conn(0x7f9924073660 0x7f9924075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.360+0000 7f770dffb700 1 -- 172.21.15.49:0/3282513646 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7700011830 con 0x7f77100ae0c0 2023-07-18T03:38:43.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.360+0000 7f770dffb700 1 -- 172.21.15.49:0/3282513646 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7700011990 con 0x7f77100ae0c0 2023-07-18T03:38:43.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.360+0000 7f770dffb700 1 -- 172.21.15.49:0/3282513646 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7700011d20 con 0x7f77100ae0c0 2023-07-18T03:38:43.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.360+0000 7f9934ff9700 1 -- 172.21.15.49:0/1434363766 <== osd.4 v2:172.21.15.107:6800/4156054429 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9924076120 con 0x7f9924073660 2023-07-18T03:38:43.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.360+0000 7fed64dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed6006cd60 0x7fed6015de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.360+0000 7fed64dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed600cd940 0x7fed600b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.360+0000 7fed64dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed60160d50 0x7fed6015d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.360+0000 7fed64dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed600c5220 con 0x7fed600cd940 2023-07-18T03:38:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.360+0000 7fed64dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed600c3740 con 0x7fed6006cd60 2023-07-18T03:38:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.360+0000 7fed64dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed600b4860 con 0x7fed60160d50 2023-07-18T03:38:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.361+0000 7f77159ff700 1 -- 172.21.15.49:0/3282513646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100ae0c0 msgr2=0x7f771006d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.361+0000 7f77159ff700 1 --2- 172.21.15.49:0/3282513646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100ae0c0 0x7f771006d040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f77000098f0 tx=0x7f7700009c00 comp rx=0 tx=0).stop 2023-07-18T03:38:43.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.362+0000 7fed5dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed600cd940 0x7fed600b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.362+0000 7fed5dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed600cd940 0x7fed600b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40372/0 (socket says 172.21.15.49:40372) 2023-07-18T03:38:43.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.362+0000 7fed5dd9b700 1 -- 172.21.15.49:0/242443787 learned_addr learned my addr 172.21.15.49:0/242443787 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.362+0000 7fed5ed9d700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed60160d50 0x7fed6015d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.363+0000 7f77159ff700 1 -- 172.21.15.49:0/3282513646 shutdown_connections 2023-07-18T03:38:43.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.363+0000 7f77159ff700 1 --2- 172.21.15.49:0/3282513646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100ae0c0 0x7f771006d040 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.363+0000 7f77159ff700 1 --2- 172.21.15.49:0/3282513646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100cc850 0x7f77100cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.363+0000 7f77159ff700 1 -- 172.21.15.49:0/3282513646 >> 172.21.15.49:0/3282513646 conn(0x7f77100b05d0 msgr2=0x7f77100b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.363+0000 7fed5dd9b700 1 -- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed60160d50 msgr2=0x7fed6015d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.363+0000 7fed5dd9b700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed60160d50 0x7fed6015d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.363+0000 7fed5dd9b700 1 -- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed6006cd60 msgr2=0x7fed6015de30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:38:43.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.363+0000 7fed5dd9b700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed6006cd60 0x7fed6015de30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.363+0000 7fed5dd9b700 1 -- 172.21.15.49:0/242443787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed540094a0 con 0x7fed600cd940 2023-07-18T03:38:43.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.363+0000 7f77159ff700 1 -- 172.21.15.49:0/3282513646 shutdown_connections 2023-07-18T03:38:43.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.364+0000 7fed5dd9b700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed600cd940 0x7fed600b40a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fed5800cc50 tx=0x7fed5800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.364+0000 7fed4effd700 1 -- 172.21.15.49:0/242443787 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5800ad80 con 0x7fed600cd940 2023-07-18T03:38:43.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.365+0000 7fed4effd700 1 -- 172.21.15.49:0/242443787 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5800ce50 con 0x7fed600cd940 2023-07-18T03:38:43.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.365+0000 7fed4effd700 1 -- 172.21.15.49:0/242443787 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58003c20 con 0x7fed600cd940 2023-07-18T03:38:43.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.363+0000 7f77159ff700 1 -- 172.21.15.49:0/3282513646 wait complete. 2023-07-18T03:38:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.367+0000 7f99377fe700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f992406adb0 0x7f992406d260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f99380c3620 tx=0x7f9928008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.366+0000 7f77159ff700 1 Processor -- start 2023-07-18T03:38:43.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.366+0000 7f77159ff700 1 -- start start 2023-07-18T03:38:43.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.366+0000 7fed64dff700 1 -- 172.21.15.49:0/242443787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed600b45e0 con 0x7fed600cd940 2023-07-18T03:38:43.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.366+0000 7fed64dff700 1 -- 172.21.15.49:0/242443787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed600c4880 con 0x7fed600cd940 2023-07-18T03:38:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.368+0000 7f77159ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100ae0c0 0x7f771015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.368+0000 7f77159ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100cba10 0x7f77100a9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.368+0000 7f77159ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100cc850 0x7f771015f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.368+0000 7f77159ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77100c3a50 con 0x7f77100cc850 2023-07-18T03:38:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.368+0000 7f77159ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77100c4010 con 0x7f77100ae0c0 2023-07-18T03:38:43.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.368+0000 7f77159ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77100c3200 con 0x7f77100cba10 2023-07-18T03:38:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.369+0000 7f770e7fc700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100cba10 0x7f77100a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.369+0000 7f770e7fc700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100cba10 0x7f77100a9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46648/0 (socket says 172.21.15.49:46648) 2023-07-18T03:38:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.369+0000 7f770e7fc700 1 -- 172.21.15.49:0/1779420583 learned_addr learned my addr 172.21.15.49:0/1779420583 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.369+0000 7f770f7fe700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100cc850 0x7f771015f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.369+0000 7f770effd700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100ae0c0 0x7f771015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.370+0000 7f770e7fc700 1 -- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100ae0c0 msgr2=0x7f771015afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.370+0000 7f770e7fc700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77100ae0c0 0x7f771015afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.370+0000 7f770e7fc700 1 -- 172.21.15.49:0/1779420583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100cc850 msgr2=0x7f771015f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.370+0000 7f770e7fc700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77100cc850 0x7f771015f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.370+0000 7f770e7fc700 1 -- 172.21.15.49:0/1779420583 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7700009550 con 0x7f77100cba10 2023-07-18T03:38:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.370+0000 7f770e7fc700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100cba10 0x7f77100a9110 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7708009cd0 tx=0x7f770800c0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.370+0000 7f76f7fff700 1 -- 172.21.15.49:0/1779420583 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800cc00 con 0x7f77100cba10 2023-07-18T03:38:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.371+0000 7f76f7fff700 1 -- 172.21.15.49:0/1779420583 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770800f040 con 0x7f77100cba10 2023-07-18T03:38:43.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.371+0000 7f76f7fff700 1 -- 172.21.15.49:0/1779420583 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708014480 con 0x7f77100cba10 2023-07-18T03:38:43.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.371+0000 7f77159ff700 1 -- 172.21.15.49:0/1779420583 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77100c34a0 con 0x7f77100cba10 2023-07-18T03:38:43.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.371+0000 7f77159ff700 1 -- 172.21.15.49:0/1779420583 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771015b710 con 0x7f77100cba10 2023-07-18T03:38:43.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.371+0000 7fed4effd700 1 -- 172.21.15.49:0/242443787 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed580050d0 con 0x7fed600cd940 2023-07-18T03:38:43.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.373+0000 7fed4effd700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fed4006adb0 0x7fed4006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.373+0000 7fed4effd700 1 -- 172.21.15.49:0/242443787 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fed58089510 con 0x7fed600cd940 2023-07-18T03:38:43.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.374+0000 7fed64dff700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] conn(0x7fed600cdfc0 0x7fed600c1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.374+0000 7fed64dff700 1 -- 172.21.15.49:0/242443787 --> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed600c4fb0 con 0x7fed600cdfc0 2023-07-18T03:38:43.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.374+0000 7fed5e59c700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fed4006adb0 0x7fed4006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.374+0000 7f77159ff700 1 -- 172.21.15.49:0/1779420583 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f76f0000fc0 con 0x7f77100cba10 2023-07-18T03:38:43.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.375+0000 7f251e7ff700 1 Processor -- start 2023-07-18T03:38:43.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.375+0000 7fed5ed9d700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] conn(0x7fed600cdfc0 0x7fed600c1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.376+0000 7f251e7ff700 1 -- start start 2023-07-18T03:38:43.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.377+0000 7fed5e59c700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fed4006adb0 0x7fed4006d260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fed50007fd0 tx=0x7fed50007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.377+0000 7fed5ed9d700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] conn(0x7fed600cdfc0 0x7fed600c1850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.377+0000 7f76f7fff700 1 -- 172.21.15.49:0/1779420583 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f77080145e0 con 0x7f77100cba10 2023-07-18T03:38:43.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.378+0000 7f251e7ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25180cd940 0x7f25180cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.378+0000 7f251e7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2518160d50 0x7f2518161a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.378+0000 7f251e7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f251806cd60 0x7f25180cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.378+0000 7f251e7ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25180cd530 con 0x7f25180cd940 2023-07-18T03:38:43.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.378+0000 7f251e7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25181615d0 con 0x7f251806cd60 2023-07-18T03:38:43.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.378+0000 7f251e7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25180ce250 con 0x7f2518160d50 2023-07-18T03:38:43.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.379+0000 7f76f7fff700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f76f806af10 0x7f76f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.379+0000 7f76f7fff700 1 -- 172.21.15.49:0/1779420583 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7708089ec0 con 0x7f77100cba10 2023-07-18T03:38:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.379+0000 7f76f7fff700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] conn(0x7f76f8073740 0x7f76f8075b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.379+0000 7f76f7fff700 1 -- 172.21.15.49:0/1779420583 --> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76f8076200 con 0x7f76f8073740 2023-07-18T03:38:43.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.380+0000 7f76f7fff700 1 -- 172.21.15.49:0/1779420583 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7708054fe0 con 0x7f77100cba10 2023-07-18T03:38:43.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.380+0000 7fed4effd700 1 -- 172.21.15.49:0/242443787 <== osd.5 v2:172.21.15.107:6818/1123907778 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fed600c4fb0 con 0x7fed600cdfc0 2023-07-18T03:38:43.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.380+0000 7f770effd700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f76f806af10 0x7f76f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.380+0000 7f251cd9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f251806cd60 0x7f25180cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.380+0000 7f2517fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25180cd940 0x7f25180cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.380+0000 7f251cd9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f251806cd60 0x7f25180cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45190/0 (socket says 172.21.15.49:45190) 2023-07-18T03:38:43.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.380+0000 7f251cd9c700 1 -- 172.21.15.49:0/2781738189 learned_addr learned my addr 172.21.15.49:0/2781738189 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.380+0000 7f250f7fe700 1 --2- 172.21.15.49:0/2781738189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2518160d50 0x7f2518161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.381+0000 7f251cd9c700 1 -- 172.21.15.49:0/2781738189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2518160d50 msgr2=0x7f2518161a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.381+0000 7f251cd9c700 1 --2- 172.21.15.49:0/2781738189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2518160d50 0x7f2518161a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.381+0000 7f251cd9c700 1 -- 172.21.15.49:0/2781738189 >> v2:172.21.15.49:3300/0 conn(0x7f25180cd940 msgr2=0x7f25180cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.381+0000 7f251cd9c700 1 --2- 172.21.15.49:0/2781738189 >> v2:172.21.15.49:3300/0 conn(0x7f25180cd940 0x7f25180cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.381+0000 7f251cd9c700 1 -- 172.21.15.49:0/2781738189 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25180ccf70 con 0x7f251806cd60 2023-07-18T03:38:43.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.381+0000 7f251cd9c700 1 --2- 172.21.15.49:0/2781738189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f251806cd60 0x7f25180cc690 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f25180cc2d0 tx=0x7f251000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=275e2ed2132faa62 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.382+0000 7f25177fe700 1 -- 172.21.15.49:0/2781738189 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251000e070 con 0x7f251806cd60 2023-07-18T03:38:43.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.382+0000 7f25177fe700 1 -- 172.21.15.49:0/2781738189 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251000b910 con 0x7f251806cd60 2023-07-18T03:38:43.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.382+0000 7f25177fe700 1 -- 172.21.15.49:0/2781738189 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251000bcc0 con 0x7f251806cd60 2023-07-18T03:38:43.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.383+0000 7f251e7ff700 1 -- 172.21.15.49:0/2781738189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f251806cd60 msgr2=0x7f25180cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.383+0000 7f251e7ff700 1 --2- 172.21.15.49:0/2781738189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f251806cd60 0x7f25180cc690 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f25180cc2d0 tx=0x7f251000b330 comp rx=0 tx=0).stop 2023-07-18T03:38:43.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.380+0000 7f770f7fe700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] conn(0x7f76f8073740 0x7f76f8075b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.382+0000 7f770f7fe700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] conn(0x7f76f8073740 0x7f76f8075b50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.383+0000 7f770effd700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f76f806af10 0x7f76f806d3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7700009c40 tx=0x7f7700008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.384+0000 7f76f7fff700 1 -- 172.21.15.49:0/1779420583 <== osd.6 v2:172.21.15.107:6808/447642181 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f76f8076200 con 0x7f76f8073740 2023-07-18T03:38:43.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.384+0000 7f251e7ff700 1 -- 172.21.15.49:0/2781738189 shutdown_connections 2023-07-18T03:38:43.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.384+0000 7f251e7ff700 1 --2- 172.21.15.49:0/2781738189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f251806cd60 0x7f25180cc690 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.384+0000 7f251e7ff700 1 --2- 172.21.15.49:0/2781738189 >> v2:172.21.15.49:3300/0 conn(0x7f25180cd940 0x7f25180cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.384+0000 7f251e7ff700 1 --2- 172.21.15.49:0/2781738189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2518160d50 0x7f2518161a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.384+0000 7f251e7ff700 1 -- 172.21.15.49:0/2781738189 >> 172.21.15.49:0/2781738189 conn(0x7f25180b05d0 msgr2=0x7f25180b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.384+0000 7f251e7ff700 1 -- 172.21.15.49:0/2781738189 shutdown_connections 2023-07-18T03:38:43.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.386+0000 7f713e9ff700 1 -- 172.21.15.49:0/925385100 --> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7138161260 con 0x7f71240735e0 2023-07-18T03:38:43.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.387+0000 7f7134ff9700 1 -- 172.21.15.49:0/925385100 <== osd.2 v2:172.21.15.49:6824/46330117 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f713000f040 con 0x7f71240735e0 2023-07-18T03:38:43.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.385+0000 7f251e7ff700 1 -- 172.21.15.49:0/2781738189 wait complete. 2023-07-18T03:38:43.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.388+0000 7f251e7ff700 1 Processor -- start 2023-07-18T03:38:43.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.388+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/202921649 --> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7d3806c180 con 0x7f7d1c0735e0 2023-07-18T03:38:43.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.388+0000 7fc828dff700 1 -- 172.21.15.49:0/1854362433 --> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc808002d40 con 0x7fc804073660 2023-07-18T03:38:43.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.388+0000 7f251e7ff700 1 -- start start 2023-07-18T03:38:43.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.389+0000 7f7d2dffb700 1 -- 172.21.15.49:0/202921649 <== osd.1 v2:172.21.15.49:6815/2328014854 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7d300098f0 con 0x7f7d1c0735e0 2023-07-18T03:38:43.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.390+0000 7f713e9ff700 1 -- 172.21.15.49:0/925385100 >> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] conn(0x7f71240735e0 msgr2=0x7f71240759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.390+0000 7f713e9ff700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] conn(0x7f71240735e0 0x7f71240759f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.390+0000 7f251e7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251806cd60 0x7f25181fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.390+0000 7f251e7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25180cd940 0x7f25181643b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.390+0000 7f251e7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2518160d50 0x7f25181648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.390+0000 7f251e7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25180b2320 con 0x7f251806cd60 2023-07-18T03:38:43.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.390+0000 7f251e7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25181ff270 con 0x7f2518160d50 2023-07-18T03:38:43.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.390+0000 7f251e7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25180b1d60 con 0x7f25180cd940 2023-07-18T03:38:43.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f713e9ff700 1 -- 172.21.15.49:0/925385100 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f712406ad30 msgr2=0x7f712406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f713e9ff700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f712406ad30 0x7f712406d1e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f71380cbc60 tx=0x7f7128006cd0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f713e9ff700 1 -- 172.21.15.49:0/925385100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71380cd940 msgr2=0x7f71380c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f713e9ff700 1 --2- 172.21.15.49:0/925385100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71380cd940 0x7f71380c4f10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f712c015000 tx=0x7f712c004750 comp rx=0 tx=0).stop 2023-07-18T03:38:43.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f251cd9c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2518160d50 0x7f25181648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f251cd9c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2518160d50 0x7f25181648f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45194/0 (socket says 172.21.15.49:45194) 2023-07-18T03:38:43.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f251cd9c700 1 -- 172.21.15.49:0/2636022017 learned_addr learned my addr 172.21.15.49:0/2636022017 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/202921649 >> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] conn(0x7f7d1c0735e0 msgr2=0x7f7d1c0759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f7d3f5ff700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] conn(0x7f7d1c0735e0 0x7f7d1c0759f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f713cf9c700 1 -- 172.21.15.49:0/925385100 reap_dead start 2023-07-18T03:38:43.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f250f7fe700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25180cd940 0x7f25181643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/202921649 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7d1c06ad30 msgr2=0x7f7d1c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f7d3f5ff700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7d1c06ad30 0x7f7d1c06d1e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d381f9b20 tx=0x7f7d28009210 comp rx=0 tx=0).stop 2023-07-18T03:38:43.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/202921649 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d380cab10 msgr2=0x7f7d380b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.391+0000 7f7d3f5ff700 1 --2- 172.21.15.49:0/202921649 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d380cab10 0x7f7d380b4410 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400c550 tx=0x7f7d3400c860 comp rx=0 tx=0).stop 2023-07-18T03:38:43.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.392+0000 7f7d3db9c700 1 -- 172.21.15.49:0/202921649 reap_dead start 2023-07-18T03:38:43.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.392+0000 7f251cd9c700 1 -- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25180cd940 msgr2=0x7f25181643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.392+0000 7f251cd9c700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25180cd940 0x7f25181643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.392+0000 7f2517fff700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251806cd60 0x7f25181fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.392+0000 7f251cd9c700 1 -- 172.21.15.49:0/2636022017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251806cd60 msgr2=0x7f25181fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.392+0000 7f251cd9c700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251806cd60 0x7f25181fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.392+0000 7f251cd9c700 1 -- 172.21.15.49:0/2636022017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2510009b30 con 0x7f2518160d50 2023-07-18T03:38:43.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.392+0000 7f2517fff700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251806cd60 0x7f25181fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:43.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.392+0000 7f713e9ff700 1 -- 172.21.15.49:0/925385100 shutdown_connections 2023-07-18T03:38:43.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.392+0000 7f713e9ff700 1 -- 172.21.15.49:0/925385100 >> 172.21.15.49:0/925385100 conn(0x7f71380b05d0 msgr2=0x7f7138162120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.392+0000 7f251cd9c700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2518160d50 0x7f25181648f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2510012000 tx=0x7f2510004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.393+0000 7f25157fa700 1 -- 172.21.15.49:0/2636022017 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251000ce80 con 0x7f2518160d50 2023-07-18T03:38:43.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.393+0000 7f25157fa700 1 -- 172.21.15.49:0/2636022017 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2510004250 con 0x7f2518160d50 2023-07-18T03:38:43.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.393+0000 7f25157fa700 1 -- 172.21.15.49:0/2636022017 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510004600 con 0x7f2518160d50 2023-07-18T03:38:43.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.393+0000 7f251e7ff700 1 -- 172.21.15.49:0/2636022017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25180b2040 con 0x7f2518160d50 2023-07-18T03:38:43.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.393+0000 7f713e9ff700 1 -- 172.21.15.49:0/925385100 shutdown_connections 2023-07-18T03:38:43.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.393+0000 7f251e7ff700 1 -- 172.21.15.49:0/2636022017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25180c6a20 con 0x7f2518160d50 2023-07-18T03:38:43.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.393+0000 7f713e9ff700 1 -- 172.21.15.49:0/925385100 wait complete. 2023-07-18T03:38:43.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.394+0000 7fb4683ff700 1 -- 172.21.15.49:0/4158576686 --> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb4600cd250 con 0x7fb444073660 2023-07-18T03:38:43.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.394+0000 7f250e7fc700 1 -- 172.21.15.49:0/2636022017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f24f0000fc0 con 0x7f2518160d50 2023-07-18T03:38:43.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.395+0000 7fb456ffd700 1 -- 172.21.15.49:0/4158576686 <== osd.3 v2:172.21.15.49:6808/3802584914 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb458003c00 con 0x7fb444073660 2023-07-18T03:38:43.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.395+0000 7fb4683ff700 1 -- 172.21.15.49:0/4158576686 >> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] conn(0x7fb444073660 msgr2=0x7fb444075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.395+0000 7fb4683ff700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] conn(0x7fb444073660 0x7fb444075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.396+0000 7f993e5ff700 1 -- 172.21.15.49:0/1434363766 --> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f99380ccdd0 con 0x7f9924073660 2023-07-18T03:38:43.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.396+0000 7fc812ffd700 1 -- 172.21.15.49:0/1854362433 <== osd.0 v2:172.21.15.49:6800/990566862 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc808002d40 con 0x7fc804073660 2023-07-18T03:38:43.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.396+0000 7f9934ff9700 1 -- 172.21.15.49:0/1434363766 <== osd.4 v2:172.21.15.107:6800/4156054429 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9930009a70 con 0x7f9924073660 2023-07-18T03:38:43.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.397+0000 7fb4683ff700 1 -- 172.21.15.49:0/4158576686 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb44406adb0 msgr2=0x7fb44406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.397+0000 7fb4683ff700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb44406adb0 0x7fb44406d260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb46015ee50 tx=0x7fb450007ed0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.397+0000 7fb4683ff700 1 -- 172.21.15.49:0/4158576686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb46006cd60 msgr2=0x7fb460147870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.397+0000 7fb4683ff700 1 --2- 172.21.15.49:0/4158576686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb46006cd60 0x7fb460147870 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00c820 tx=0x7fb45c00cb30 comp rx=0 tx=0).stop 2023-07-18T03:38:43.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.397+0000 7fb46699c700 1 -- 172.21.15.49:0/4158576686 reap_dead start 2023-07-18T03:38:43.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.397+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/202921649 shutdown_connections 2023-07-18T03:38:43.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.397+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/202921649 >> 172.21.15.49:0/202921649 conn(0x7f7d380b05d0 msgr2=0x7f7d3806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.397+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/202921649 shutdown_connections 2023-07-18T03:38:43.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.397+0000 7f7d3f5ff700 1 -- 172.21.15.49:0/202921649 wait complete. 2023-07-18T03:38:43.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.398+0000 7f25157fa700 1 -- 172.21.15.49:0/2636022017 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2510004760 con 0x7f2518160d50 2023-07-18T03:38:43.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.399+0000 7fc828dff700 1 -- 172.21.15.49:0/1854362433 >> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] conn(0x7fc804073660 msgr2=0x7fc804075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.399+0000 7fc828dff700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] conn(0x7fc804073660 0x7fc804075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.399+0000 7f25157fa700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f24f806ace0 0x7f24f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.399+0000 7f25157fa700 1 -- 172.21.15.49:0/2636022017 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f251008ad50 con 0x7f2518160d50 2023-07-18T03:38:43.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.399+0000 7f25157fa700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] conn(0x7f24f8073510 0x7f24f8075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.399+0000 7f25157fa700 1 -- 172.21.15.49:0/2636022017 --> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f24f8075fd0 con 0x7f24f8073510 2023-07-18T03:38:43.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.399+0000 7f25157fa700 1 -- 172.21.15.49:0/2636022017 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f251008b1a0 con 0x7f2518160d50 2023-07-18T03:38:43.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.399+0000 7fc828dff700 1 -- 172.21.15.49:0/1854362433 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc80406adb0 msgr2=0x7fc80406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.399+0000 7fc828dff700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc80406adb0 0x7fc80406d260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8240afff0 tx=0x7fc814009210 comp rx=0 tx=0).stop 2023-07-18T03:38:43.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.399+0000 7fc828dff700 1 -- 172.21.15.49:0/1854362433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8240ae0c0 msgr2=0x7fc82414dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.399+0000 7fc828dff700 1 --2- 172.21.15.49:0/1854362433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8240ae0c0 0x7fc82414dd50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc8180090b0 tx=0x7fc818004750 comp rx=0 tx=0).stop 2023-07-18T03:38:43.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.400+0000 7fc822d9d700 1 -- 172.21.15.49:0/1854362433 reap_dead start 2023-07-18T03:38:43.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.400+0000 7f250f7fe700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] conn(0x7f24f8073510 0x7f24f8075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.400+0000 7f2517fff700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f24f806ace0 0x7f24f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.400+0000 7fb4683ff700 1 -- 172.21.15.49:0/4158576686 shutdown_connections 2023-07-18T03:38:43.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.400+0000 7fb4683ff700 1 -- 172.21.15.49:0/4158576686 >> 172.21.15.49:0/4158576686 conn(0x7fb4600b05d0 msgr2=0x7fb460164590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.400+0000 7fb4683ff700 1 -- 172.21.15.49:0/4158576686 shutdown_connections 2023-07-18T03:38:43.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.400+0000 7fb4683ff700 1 -- 172.21.15.49:0/4158576686 wait complete. 2023-07-18T03:38:43.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.401+0000 7f250f7fe700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] conn(0x7f24f8073510 0x7f24f8075920 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.401+0000 7f2517fff700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f24f806ace0 0x7f24f806d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2500009ae0 tx=0x7f2500008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.402+0000 7fc828dff700 1 -- 172.21.15.49:0/1854362433 shutdown_connections 2023-07-18T03:38:43.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.402+0000 7fc828dff700 1 -- 172.21.15.49:0/1854362433 >> 172.21.15.49:0/1854362433 conn(0x7fc8240b05d0 msgr2=0x7fc82415c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.402+0000 7f25157fa700 1 -- 172.21.15.49:0/2636022017 <== osd.7 v2:172.21.15.107:6816/2327910262 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f24f8075fd0 con 0x7f24f8073510 2023-07-18T03:38:43.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.402+0000 7f993e5ff700 1 -- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] conn(0x7f9924073660 msgr2=0x7f9924075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.402+0000 7f993e5ff700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] conn(0x7f9924073660 0x7f9924075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.402+0000 7f993e5ff700 1 -- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f992406adb0 msgr2=0x7f992406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.402+0000 7f993e5ff700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f992406adb0 0x7f992406d260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f99380c3620 tx=0x7f9928008040 comp rx=0 tx=0).stop 2023-07-18T03:38:43.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.402+0000 7f993e5ff700 1 -- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99380cdbe0 msgr2=0x7f993815ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.402+0000 7f993e5ff700 1 --2- 172.21.15.49:0/1434363766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99380cdbe0 0x7f993815ffe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f992c000c00 tx=0x7f992c004750 comp rx=0 tx=0).stop 2023-07-18T03:38:43.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.403+0000 7f993cb9c700 1 -- 172.21.15.49:0/1434363766 reap_dead start 2023-07-18T03:38:43.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.403+0000 7f993e5ff700 1 -- 172.21.15.49:0/1434363766 shutdown_connections 2023-07-18T03:38:43.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.403+0000 7f993e5ff700 1 -- 172.21.15.49:0/1434363766 >> 172.21.15.49:0/1434363766 conn(0x7f99380b05d0 msgr2=0x7f993806f730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.403+0000 7f993e5ff700 1 -- 172.21.15.49:0/1434363766 shutdown_connections 2023-07-18T03:38:43.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.403+0000 7f993e5ff700 1 -- 172.21.15.49:0/1434363766 wait complete. 2023-07-18T03:38:43.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.405+0000 7fc828dff700 1 -- 172.21.15.49:0/1854362433 shutdown_connections 2023-07-18T03:38:43.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.405+0000 7fc828dff700 1 -- 172.21.15.49:0/1854362433 wait complete. 2023-07-18T03:38:43.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.415+0000 7fed64dff700 1 -- 172.21.15.49:0/242443787 --> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed600c5360 con 0x7fed600cdfc0 2023-07-18T03:38:43.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.416+0000 7fed4effd700 1 -- 172.21.15.49:0/242443787 <== osd.5 v2:172.21.15.107:6818/1123907778 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fed600c5360 con 0x7fed600cdfc0 2023-07-18T03:38:43.455 INFO:teuthology.orchestra.run.smithi049.stdout:64424509445 2023-07-18T03:38:43.455 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-18T03:38:43.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.418+0000 7fed64dff700 1 -- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] conn(0x7fed600cdfc0 msgr2=0x7fed600c1850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.418+0000 7fed64dff700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] conn(0x7fed600cdfc0 0x7fed600c1850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.418+0000 7fed64dff700 1 -- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fed4006adb0 msgr2=0x7fed4006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.418+0000 7fed64dff700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fed4006adb0 0x7fed4006d260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fed50007fd0 tx=0x7fed50007980 comp rx=0 tx=0).stop 2023-07-18T03:38:43.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.418+0000 7fed64dff700 1 -- 172.21.15.49:0/242443787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed600cd940 msgr2=0x7fed600b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.418+0000 7fed64dff700 1 --2- 172.21.15.49:0/242443787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed600cd940 0x7fed600b40a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fed5800cc50 tx=0x7fed5800a340 comp rx=0 tx=0).stop 2023-07-18T03:38:43.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.418+0000 7fed5ed9d700 1 -- 172.21.15.49:0/242443787 reap_dead start 2023-07-18T03:38:43.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.418+0000 7fed64dff700 1 -- 172.21.15.49:0/242443787 shutdown_connections 2023-07-18T03:38:43.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.418+0000 7fed64dff700 1 -- 172.21.15.49:0/242443787 >> 172.21.15.49:0/242443787 conn(0x7fed600b05d0 msgr2=0x7fed6015f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.419+0000 7fed64dff700 1 -- 172.21.15.49:0/242443787 shutdown_connections 2023-07-18T03:38:43.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.419+0000 7fed64dff700 1 -- 172.21.15.49:0/242443787 wait complete. 2023-07-18T03:38:43.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.420+0000 7f77159ff700 1 -- 172.21.15.49:0/1779420583 --> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f76f0002d40 con 0x7f76f8073740 2023-07-18T03:38:43.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.421+0000 7f76f7fff700 1 -- 172.21.15.49:0/1779420583 <== osd.6 v2:172.21.15.107:6808/447642181 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f76f0002d40 con 0x7f76f8073740 2023-07-18T03:38:43.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.424+0000 7f77159ff700 1 -- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] conn(0x7f76f8073740 msgr2=0x7f76f8075b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.424+0000 7f77159ff700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] conn(0x7f76f8073740 0x7f76f8075b50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.424+0000 7f77159ff700 1 -- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f76f806af10 msgr2=0x7f76f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.424+0000 7f77159ff700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f76f806af10 0x7f76f806d3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7700009c40 tx=0x7f7700008030 comp rx=0 tx=0).stop 2023-07-18T03:38:43.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.424+0000 7f77159ff700 1 -- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100cba10 msgr2=0x7f77100a9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.424+0000 7f77159ff700 1 --2- 172.21.15.49:0/1779420583 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77100cba10 0x7f77100a9110 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7708009cd0 tx=0x7f770800c0c0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.424+0000 7f770f7fe700 1 -- 172.21.15.49:0/1779420583 reap_dead start 2023-07-18T03:38:43.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.424+0000 7f77159ff700 1 -- 172.21.15.49:0/1779420583 shutdown_connections 2023-07-18T03:38:43.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.424+0000 7f77159ff700 1 -- 172.21.15.49:0/1779420583 >> 172.21.15.49:0/1779420583 conn(0x7f77100b05d0 msgr2=0x7f77100af360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.424+0000 7f77159ff700 1 -- 172.21.15.49:0/1779420583 shutdown_connections 2023-07-18T03:38:43.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.424+0000 7f77159ff700 1 -- 172.21.15.49:0/1779420583 wait complete. 2023-07-18T03:38:43.461 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2023-07-18T03:38:43.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-18T03:38:43.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.433+0000 7f250e7fc700 1 -- 172.21.15.49:0/2636022017 --> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f24f0002d40 con 0x7f24f8073510 2023-07-18T03:38:43.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.434+0000 7f25157fa700 1 -- 172.21.15.49:0/2636022017 <== osd.7 v2:172.21.15.107:6816/2327910262 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f24f0002d40 con 0x7f24f8073510 2023-07-18T03:38:43.463 INFO:teuthology.orchestra.run.smithi049.stdout:60129542151 2023-07-18T03:38:43.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.434+0000 7f250e7fc700 1 -- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] conn(0x7f24f8073510 msgr2=0x7f24f8075920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.434+0000 7f250e7fc700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] conn(0x7f24f8073510 0x7f24f8075920 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.434+0000 7f250e7fc700 1 -- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f24f806ace0 msgr2=0x7f24f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.434+0000 7f250e7fc700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f24f806ace0 0x7f24f806d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2500009ae0 tx=0x7f2500008040 comp rx=0 tx=0).stop 2023-07-18T03:38:43.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.434+0000 7f250e7fc700 1 -- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2518160d50 msgr2=0x7f25181648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.434+0000 7f250e7fc700 1 --2- 172.21.15.49:0/2636022017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2518160d50 0x7f25181648f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2510012000 tx=0x7f2510004ad0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.434+0000 7f251cd9c700 1 -- 172.21.15.49:0/2636022017 reap_dead start 2023-07-18T03:38:43.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.435+0000 7f250e7fc700 1 -- 172.21.15.49:0/2636022017 shutdown_connections 2023-07-18T03:38:43.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.435+0000 7f250e7fc700 1 -- 172.21.15.49:0/2636022017 >> 172.21.15.49:0/2636022017 conn(0x7f25180b05d0 msgr2=0x7f25180cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.465 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-18T03:38:43.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.435+0000 7f250e7fc700 1 -- 172.21.15.49:0/2636022017 shutdown_connections 2023-07-18T03:38:43.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.435+0000 7f250e7fc700 1 -- 172.21.15.49:0/2636022017 wait complete. 2023-07-18T03:38:43.467 INFO:teuthology.orchestra.run.smithi049.stdout:60129542151 2023-07-18T03:38:43.467 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-18T03:38:43.468 INFO:teuthology.orchestra.run.smithi049.stdout:64424509448 2023-07-18T03:38:43.468 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-18T03:38:43.469 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2023-07-18T03:38:43.470 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-18T03:38:43.471 INFO:teuthology.orchestra.run.smithi049.stdout:64424509446 2023-07-18T03:38:43.471 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-18T03:38:43.472 INFO:teuthology.orchestra.run.smithi049.stdout:64424509447 2023-07-18T03:38:43.473 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-18T03:38:43.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.871+0000 7f204cfff700 1 Processor -- start 2023-07-18T03:38:43.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.872+0000 7f204cfff700 1 -- start start 2023-07-18T03:38:43.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.874+0000 7f204cfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f20480ce3b0 0x7f20480cb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.874+0000 7f204cfff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20480cab60 con 0x7f20480cdf40 2023-07-18T03:38:43.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.874+0000 7f204cfff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204806bcc0 con 0x7f20481587d0 2023-07-18T03:38:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.874+0000 7f204cfff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2048158ba0 con 0x7f20480ce3b0 2023-07-18T03:38:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.876+0000 7f203dd9b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f20480cdf40 0x7f20480caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41596/0 (socket says 172.21.15.49:41596) 2023-07-18T03:38:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.876+0000 7f203dd9b700 1 -- 172.21.15.49:0/3742732788 learned_addr learned my addr 172.21.15.49:0/3742732788 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:43.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.876+0000 7f2046d9d700 1 --2- 172.21.15.49:0/3742732788 >> v2:172.21.15.49:3300/0 conn(0x7f20480ce3b0 0x7f20480cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.877+0000 7f2046d9d700 1 -- 172.21.15.49:0/3742732788 >> v1:172.21.15.107:6789/0 conn(0x7f20481587d0 legacy=0x7f204806b5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:38:43.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.877+0000 7f2046d9d700 1 -- 172.21.15.49:0/3742732788 >> v1:172.21.15.49:6789/0 conn(0x7f20480cdf40 legacy=0x7f20480caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.877+0000 7f2046d9d700 1 -- 172.21.15.49:0/3742732788 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20480ce850 con 0x7f20480ce3b0 2023-07-18T03:38:43.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.877+0000 7f2046d9d700 1 --2- 172.21.15.49:0/3742732788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20480ce3b0 0x7f20480cb700 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f204000aea0 tx=0x7f204000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a4a64c96b39eb4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.878+0000 7f2045d9b700 1 -- 172.21.15.49:0/3742732788 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204000f070 con 0x7f20480ce3b0 2023-07-18T03:38:43.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.878+0000 7f2045d9b700 1 -- 172.21.15.49:0/3742732788 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2040003a20 con 0x7f20480ce3b0 2023-07-18T03:38:43.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.878+0000 7f2045d9b700 1 -- 172.21.15.49:0/3742732788 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040003db0 con 0x7f20480ce3b0 2023-07-18T03:38:43.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.879+0000 7f204cfff700 1 -- 172.21.15.49:0/3742732788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20480ce3b0 msgr2=0x7f20480cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.879+0000 7f204cfff700 1 --2- 172.21.15.49:0/3742732788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20480ce3b0 0x7f20480cb700 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f204000aea0 tx=0x7f204000c330 comp rx=0 tx=0).stop 2023-07-18T03:38:43.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.880+0000 7f204cfff700 1 -- 172.21.15.49:0/3742732788 shutdown_connections 2023-07-18T03:38:43.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.880+0000 7f204cfff700 1 --2- 172.21.15.49:0/3742732788 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20480ce3b0 0x7f20480cb700 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.880+0000 7f204cfff700 1 -- 172.21.15.49:0/3742732788 >> 172.21.15.49:0/3742732788 conn(0x7f20480b05d0 msgr2=0x7f20480b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.880+0000 7f204cfff700 1 -- 172.21.15.49:0/3742732788 shutdown_connections 2023-07-18T03:38:43.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.880+0000 7f204cfff700 1 -- 172.21.15.49:0/3742732788 wait complete. 2023-07-18T03:38:43.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.880+0000 7f4ec7fff700 1 Processor -- start 2023-07-18T03:38:43.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.880+0000 7f4ec7fff700 1 -- start start 2023-07-18T03:38:43.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.882+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ec0160c30 0x7f4ec00cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.882+0000 7f4ec7fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4ec00cdbe0 0x7f4ec00ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.882+0000 7f4ec7fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4ec0164710 0x7f4ec0164b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.882+0000 7f4ec7fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec00ce570 con 0x7f4ec0164710 2023-07-18T03:38:43.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.882+0000 7f4ec7fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec0161000 con 0x7f4ec00cdbe0 2023-07-18T03:38:43.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.882+0000 7f4ec7fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0165400 con 0x7f4ec0160c30 2023-07-18T03:38:43.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.884+0000 7f4ec559a700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4ec00cdbe0 0x7f4ec00ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.884+0000 7f4ec659c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4ec0164710 0x7f4ec0164b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.884+0000 7f4ec659c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4ec0164710 0x7f4ec0164b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40416/0 (socket says 172.21.15.49:40416) 2023-07-18T03:38:43.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.884+0000 7f4ec659c700 1 -- 172.21.15.49:0/2519442533 learned_addr learned my addr 172.21.15.49:0/2519442533 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.884+0000 7f204cfff700 1 Processor -- start 2023-07-18T03:38:43.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.885+0000 7f4ec659c700 1 -- 172.21.15.49:0/2519442533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ec0160c30 msgr2=0x7f4ec00cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.885+0000 7f4ec5d9b700 1 --2- 172.21.15.49:0/2519442533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ec0160c30 0x7f4ec00cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.885+0000 7f4ec659c700 1 --2- 172.21.15.49:0/2519442533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ec0160c30 0x7f4ec00cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.885+0000 7f4ec659c700 1 -- 172.21.15.49:0/2519442533 >> v2:172.21.15.107:3300/0 conn(0x7f4ec00cdbe0 msgr2=0x7f4ec00ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.885+0000 7f4ec659c700 1 --2- 172.21.15.49:0/2519442533 >> v2:172.21.15.107:3300/0 conn(0x7f4ec00cdbe0 0x7f4ec00ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.885+0000 7f4ec659c700 1 -- 172.21.15.49:0/2519442533 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec00cc810 con 0x7f4ec0164710 2023-07-18T03:38:43.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.886+0000 7f4ec659c700 1 --2- 172.21.15.49:0/2519442533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ec0164710 0x7f4ec0164b90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0067540 tx=0x7f4eb0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a2523c7f02a81 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.885+0000 7f204cfff700 1 -- start start 2023-07-18T03:38:43.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.886+0000 7f204cfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20480cdf40 0x7f20480af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.886+0000 7f204cfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20480ce3b0 0x7f20481541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.886+0000 7f204cfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20481587d0 0x7f2048158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.886+0000 7f204cfff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20481e3f80 con 0x7f20480cdf40 2023-07-18T03:38:43.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.886+0000 7f204cfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20480c5d30 con 0x7f20481587d0 2023-07-18T03:38:43.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.886+0000 7f204cfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20481e36e0 con 0x7f20480ce3b0 2023-07-18T03:38:43.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.887+0000 7f4ec4d99700 1 -- 172.21.15.49:0/2519442533 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000b870 con 0x7f4ec0164710 2023-07-18T03:38:43.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.887+0000 7f4ec4d99700 1 -- 172.21.15.49:0/2519442533 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb000b9d0 con 0x7f4ec0164710 2023-07-18T03:38:43.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.887+0000 7f4ec4d99700 1 -- 172.21.15.49:0/2519442533 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000bd40 con 0x7f4ec0164710 2023-07-18T03:38:43.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.887+0000 7f2046d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20481587d0 0x7f2048158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.887+0000 7f2046d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20481587d0 0x7f2048158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45208/0 (socket says 172.21.15.49:45208) 2023-07-18T03:38:43.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.887+0000 7f2046d9d700 1 -- 172.21.15.49:0/261590174 learned_addr learned my addr 172.21.15.49:0/261590174 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f203dd9b700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20480ce3b0 0x7f20481541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f204659c700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20480cdf40 0x7f20480af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f4ec7fff700 1 -- 172.21.15.49:0/2519442533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ec0164710 msgr2=0x7f4ec0164b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f4ec7fff700 1 --2- 172.21.15.49:0/2519442533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ec0164710 0x7f4ec0164b90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0067540 tx=0x7f4eb0009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f2046d9d700 1 -- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20480ce3b0 msgr2=0x7f20481541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f2046d9d700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20480ce3b0 0x7f20481541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f2046d9d700 1 -- 172.21.15.49:0/261590174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20480cdf40 msgr2=0x7f20480af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f2046d9d700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20480cdf40 0x7f20480af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f2046d9d700 1 -- 172.21.15.49:0/261590174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204000ab80 con 0x7f20481587d0 2023-07-18T03:38:43.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f204659c700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20480cdf40 0x7f20480af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:43.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f4ec7fff700 1 -- 172.21.15.49:0/2519442533 shutdown_connections 2023-07-18T03:38:43.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f4ec7fff700 1 --2- 172.21.15.49:0/2519442533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ec0164710 0x7f4ec0164b90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f4ec7fff700 1 --2- 172.21.15.49:0/2519442533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ec0160c30 0x7f4ec00cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f4ec7fff700 1 --2- 172.21.15.49:0/2519442533 >> v2:172.21.15.107:3300/0 conn(0x7f4ec00cdbe0 0x7f4ec00ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f4ec7fff700 1 -- 172.21.15.49:0/2519442533 >> 172.21.15.49:0/2519442533 conn(0x7f4ec00b05d0 msgr2=0x7f4ec00b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.888+0000 7f203dd9b700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20480ce3b0 0x7f20481541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:43.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.889+0000 7f2046d9d700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20481587d0 0x7f2048158ce0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f204000c370 tx=0x7f2040004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.889+0000 7f203f7fe700 1 -- 172.21.15.49:0/261590174 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20400045f0 con 0x7f20481587d0 2023-07-18T03:38:43.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.889+0000 7f203f7fe700 1 -- 172.21.15.49:0/261590174 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2040004750 con 0x7f20481587d0 2023-07-18T03:38:43.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.889+0000 7f203f7fe700 1 -- 172.21.15.49:0/261590174 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040008440 con 0x7f20481587d0 2023-07-18T03:38:43.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.889+0000 7f204cfff700 1 -- 172.21.15.49:0/261590174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20481e39c0 con 0x7f20481587d0 2023-07-18T03:38:43.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.889+0000 7f204cfff700 1 -- 172.21.15.49:0/261590174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2048154710 con 0x7f20481587d0 2023-07-18T03:38:43.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.890+0000 7fba1e1ff700 1 Processor -- start 2023-07-18T03:38:43.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.890+0000 7fba1e1ff700 1 -- start start 2023-07-18T03:38:43.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.894+0000 7f3db4dff700 1 Processor -- start 2023-07-18T03:38:43.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.894+0000 7f204cfff700 1 -- 172.21.15.49:0/261590174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20480c5b90 con 0x7f20481587d0 2023-07-18T03:38:43.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.892+0000 7f4ec7fff700 1 -- 172.21.15.49:0/2519442533 shutdown_connections 2023-07-18T03:38:43.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.893+0000 7f4ec7fff700 1 -- 172.21.15.49:0/2519442533 wait complete. 2023-07-18T03:38:43.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.892+0000 7fba1e1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fba180cdbe0 0x7fba180ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.892+0000 7fba1e1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba18164710 0x7fba18164b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.892+0000 7fba1e1ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba180cda00 con 0x7fba18160c30 2023-07-18T03:38:43.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.892+0000 7fba1e1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba180ce5a0 con 0x7fba180cdbe0 2023-07-18T03:38:43.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.892+0000 7fba1e1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba18161000 con 0x7fba18164710 2023-07-18T03:38:43.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.893+0000 7fba177fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fba18160c30 0x7fba180cd2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41598/0 (socket says 172.21.15.49:41598) 2023-07-18T03:38:43.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.893+0000 7fba177fe700 1 -- 172.21.15.49:0/1766030711 learned_addr learned my addr 172.21.15.49:0/1766030711 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:43.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.893+0000 7fba0effd700 1 --2- 172.21.15.49:0/1766030711 >> v2:172.21.15.107:3300/0 conn(0x7fba180cdbe0 0x7fba180ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.894+0000 7fba0effd700 1 -- 172.21.15.49:0/1766030711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba18164710 msgr2=0x7fba18164b90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:38:43.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.894+0000 7fba0effd700 1 --2- 172.21.15.49:0/1766030711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba18164710 0x7fba18164b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.894+0000 7fba0effd700 1 -- 172.21.15.49:0/1766030711 >> v1:172.21.15.49:6789/0 conn(0x7fba18160c30 legacy=0x7fba180cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.894+0000 7fba0effd700 1 -- 172.21.15.49:0/1766030711 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba18165400 con 0x7fba180cdbe0 2023-07-18T03:38:43.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.894+0000 7fba0effd700 1 --2- 172.21.15.49:0/1766030711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba180cdbe0 0x7fba180ce060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba00009840 tx=0x7fba00009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f105b4811707f9a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.895+0000 7fba16ffd700 1 -- 172.21.15.49:0/1766030711 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00004230 con 0x7fba180cdbe0 2023-07-18T03:38:43.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.895+0000 7fba16ffd700 1 -- 172.21.15.49:0/1766030711 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba00004390 con 0x7fba180cdbe0 2023-07-18T03:38:43.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.895+0000 7fba16ffd700 1 -- 172.21.15.49:0/1766030711 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba000036a0 con 0x7fba180cdbe0 2023-07-18T03:38:43.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.896+0000 7fba1e1ff700 1 -- 172.21.15.49:0/1766030711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba180cdbe0 msgr2=0x7fba180ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.896+0000 7fba1e1ff700 1 --2- 172.21.15.49:0/1766030711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba180cdbe0 0x7fba180ce060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba00009840 tx=0x7fba00009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:43.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.896+0000 7fba1e1ff700 1 -- 172.21.15.49:0/1766030711 shutdown_connections 2023-07-18T03:38:43.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.896+0000 7fba1e1ff700 1 --2- 172.21.15.49:0/1766030711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba18164710 0x7fba18164b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.896+0000 7fba1e1ff700 1 --2- 172.21.15.49:0/1766030711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba180cdbe0 0x7fba180ce060 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.896+0000 7fba1e1ff700 1 -- 172.21.15.49:0/1766030711 >> 172.21.15.49:0/1766030711 conn(0x7fba180b05d0 msgr2=0x7fba180b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.896+0000 7fba1e1ff700 1 -- 172.21.15.49:0/1766030711 shutdown_connections 2023-07-18T03:38:43.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.895+0000 7f3db4dff700 1 -- start start 2023-07-18T03:38:43.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.897+0000 7f3db4dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3db01587d0 0x7f3db006b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.897+0000 7f3db4dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3db006baf0 0x7f3db00cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.897+0000 7f3db4dff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3db00ce480 0x7f3db00cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.897+0000 7f3db4dff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db00ce8d0 con 0x7f3db00ce480 2023-07-18T03:38:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.897+0000 7f3db4dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db00cab60 con 0x7f3db006baf0 2023-07-18T03:38:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.897+0000 7f3db4dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db0158ba0 con 0x7f3db01587d0 2023-07-18T03:38:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.896+0000 7f4ec7fff700 1 Processor -- start 2023-07-18T03:38:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.896+0000 7fba1e1ff700 1 -- 172.21.15.49:0/1766030711 wait complete. 2023-07-18T03:38:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.898+0000 7f3dadd9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3db006baf0 0x7f3db00cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.898+0000 7f3dadd9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3db006baf0 0x7f3db00cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45220/0 (socket says 172.21.15.49:45220) 2023-07-18T03:38:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.898+0000 7f3dadd9b700 1 -- 172.21.15.49:0/3678112602 learned_addr learned my addr 172.21.15.49:0/3678112602 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.898+0000 7f3dae59c700 1 --2- 172.21.15.49:0/3678112602 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3db01587d0 0x7f3db006b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.896+0000 7f203f7fe700 1 -- 172.21.15.49:0/261590174 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20400085a0 con 0x7f20481587d0 2023-07-18T03:38:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.897+0000 7f203f7fe700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f202806ace0 0x7f202806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.897+0000 7f203f7fe700 1 -- 172.21.15.49:0/261590174 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f204008af10 con 0x7f20481587d0 2023-07-18T03:38:43.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.898+0000 7fbb313ff700 1 Processor -- start 2023-07-18T03:38:43.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.898+0000 7f3dadd9b700 1 -- 172.21.15.49:0/3678112602 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3db01587d0 msgr2=0x7f3db006b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.898+0000 7f3dadd9b700 1 --2- 172.21.15.49:0/3678112602 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3db01587d0 0x7f3db006b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.898+0000 7f3dadd9b700 1 -- 172.21.15.49:0/3678112602 >> v2:172.21.15.49:3300/0 conn(0x7f3db00ce480 msgr2=0x7f3db00cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.898+0000 7f3dadd9b700 1 --2- 172.21.15.49:0/3678112602 >> v2:172.21.15.49:3300/0 conn(0x7f3db00ce480 0x7f3db00cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.898+0000 7f3dadd9b700 1 -- 172.21.15.49:0/3678112602 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db006bf70 con 0x7f3db006baf0 2023-07-18T03:38:43.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.899+0000 7fbb313ff700 1 -- start start 2023-07-18T03:38:43.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.899+0000 7fba1e1ff700 1 Processor -- start 2023-07-18T03:38:43.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.899+0000 7f3dadd9b700 1 --2- 172.21.15.49:0/3678112602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db006baf0 0x7f3db00cdf40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3db0067540 tx=0x7f3da000a2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb16c103b95caad9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.900+0000 7f204659c700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f202806ace0 0x7f202806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.900+0000 7fba1e1ff700 1 -- start start 2023-07-18T03:38:43.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7f204659c700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f202806ace0 0x7f202806d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f20480b2630 tx=0x7f2030008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7fba1e1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba180cdbe0 0x7fba181eb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7fba1e1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba18160c30 0x7fba18160270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7fba1e1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba18164710 0x7fba18163210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7fba1e1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba181607b0 con 0x7fba18160c30 2023-07-18T03:38:43.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7fba1e1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba181ea3f0 con 0x7fba180cdbe0 2023-07-18T03:38:43.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7fba1e1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba181ea690 con 0x7fba18164710 2023-07-18T03:38:43.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7fbb313ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbb2c0cf890 0x7fbb2c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7fbb313ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbb2c0cf030 0x7fbb2c0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7fbb313ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2c0cd8e0 con 0x7fbb2c1601b0 2023-07-18T03:38:43.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7fbb313ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2c0d2760 con 0x7fbb2c0cf030 2023-07-18T03:38:43.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7fbb313ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2c0d01a0 con 0x7fbb2c0cf890 2023-07-18T03:38:43.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.902+0000 7fba177fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba180cdbe0 0x7fba181eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.902+0000 7fbb2b7fe700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fbb2c1601b0 0x7fbb2c1605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:53484/0 (socket says 172.21.15.49:53484) 2023-07-18T03:38:43.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.902+0000 7fbb2b7fe700 1 -- 172.21.15.49:0/4042546513 learned_addr learned my addr 172.21.15.49:0/4042546513 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:43.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.902+0000 7fbb2affd700 1 --2- 172.21.15.49:0/4042546513 >> v2:172.21.15.107:3300/0 conn(0x7fbb2c0cf890 0x7fbb2c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.902+0000 7fba17fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba18164710 0x7fba18163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.902+0000 7fba17fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba18164710 0x7fba18163210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46726/0 (socket says 172.21.15.49:46726) 2023-07-18T03:38:43.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.902+0000 7fba17fff700 1 -- 172.21.15.49:0/3989695371 learned_addr learned my addr 172.21.15.49:0/3989695371 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.901+0000 7f4ec7fff700 1 -- start start 2023-07-18T03:38:43.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.903+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ec00cdbe0 0x7f4ec01f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.903+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ec0160c30 0x7f4ec015ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.903+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0164710 0x7f4ec015f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.903+0000 7f4ec7fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01f69a0 con 0x7f4ec0160c30 2023-07-18T03:38:43.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.903+0000 7f4ec7fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0071b60 con 0x7f4ec0164710 2023-07-18T03:38:43.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.903+0000 7f4ec7fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0070fe0 con 0x7f4ec00cdbe0 2023-07-18T03:38:43.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.903+0000 7f4ec5d9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ec00cdbe0 0x7f4ec01f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.903+0000 7fba0effd700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba18160c30 0x7fba18160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.903+0000 7f4ec5d9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ec00cdbe0 0x7f4ec01f3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46742/0 (socket says 172.21.15.49:46742) 2023-07-18T03:38:43.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.903+0000 7f4ec5d9b700 1 -- 172.21.15.49:0/3409510591 learned_addr learned my addr 172.21.15.49:0/3409510591 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7f4ec5d9b700 1 -- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0164710 msgr2=0x7f4ec015f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7f4ec5d9b700 1 --2- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0164710 0x7f4ec015f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7f4ec5d9b700 1 -- 172.21.15.49:0/3409510591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ec0160c30 msgr2=0x7f4ec015ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7f4ec5d9b700 1 --2- 172.21.15.49:0/3409510591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ec0160c30 0x7f4ec015ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7f4ec5d9b700 1 -- 172.21.15.49:0/3409510591 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb0009550 con 0x7f4ec00cdbe0 2023-07-18T03:38:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7fba177fe700 1 -- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba18164710 msgr2=0x7fba18163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7fba177fe700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba18164710 0x7fba18163210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7fba177fe700 1 -- 172.21.15.49:0/3989695371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba18160c30 msgr2=0x7fba18160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7fba177fe700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba18160c30 0x7fba18160270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7fba177fe700 1 -- 172.21.15.49:0/3989695371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba000094a0 con 0x7fba180cdbe0 2023-07-18T03:38:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7fba17fff700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba18164710 0x7fba18163210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:43.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7f3dad59a700 1 -- 172.21.15.49:0/3678112602 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da000acc0 con 0x7f3db006baf0 2023-07-18T03:38:43.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7f4ec5d9b700 1 --2- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ec00cdbe0 0x7f4ec01f3940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00c7f0 tx=0x7f4ebc00cbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.905+0000 7f203f7fe700 1 -- 172.21.15.49:0/261590174 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f20480c5b90 con 0x7f20481587d0 2023-07-18T03:38:43.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.905+0000 7fba177fe700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba180cdbe0 0x7fba181eb450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fba08001fc0 tx=0x7fba0800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7f3dad59a700 1 -- 172.21.15.49:0/3678112602 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da000ae20 con 0x7f3db006baf0 2023-07-18T03:38:43.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7f3dad59a700 1 -- 172.21.15.49:0/3678112602 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da000cab0 con 0x7f3db006baf0 2023-07-18T03:38:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.905+0000 7f3db4dff700 1 -- 172.21.15.49:0/3678112602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db006baf0 msgr2=0x7f3db00cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.905+0000 7f3db4dff700 1 --2- 172.21.15.49:0/3678112602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db006baf0 0x7f3db00cdf40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3db0067540 tx=0x7f3da000a2d0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.905+0000 7fba14ff9700 1 -- 172.21.15.49:0/3989695371 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0800ac90 con 0x7fba180cdbe0 2023-07-18T03:38:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.906+0000 7fba14ff9700 1 -- 172.21.15.49:0/3989695371 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0800adf0 con 0x7fba180cdbe0 2023-07-18T03:38:43.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.906+0000 7fba14ff9700 1 -- 172.21.15.49:0/3989695371 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba08007310 con 0x7fba180cdbe0 2023-07-18T03:38:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.906+0000 7fba1e1ff700 1 -- 172.21.15.49:0/3989695371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba181ea970 con 0x7fba180cdbe0 2023-07-18T03:38:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.906+0000 7fba1e1ff700 1 -- 172.21.15.49:0/3989695371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba181eaf30 con 0x7fba180cdbe0 2023-07-18T03:38:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.904+0000 7fbb23fff700 1 --2- 172.21.15.49:0/4042546513 >> v2:172.21.15.49:3300/0 conn(0x7fbb2c0cf030 0x7fbb2c0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.905+0000 7fbb23fff700 1 -- 172.21.15.49:0/4042546513 >> v2:172.21.15.107:3300/0 conn(0x7fbb2c0cf890 msgr2=0x7fbb2c0b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.905+0000 7fbb23fff700 1 --2- 172.21.15.49:0/4042546513 >> v2:172.21.15.107:3300/0 conn(0x7fbb2c0cf890 0x7fbb2c0b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.905+0000 7fbb23fff700 1 -- 172.21.15.49:0/4042546513 >> v1:172.21.15.107:6789/0 conn(0x7fbb2c1601b0 legacy=0x7fbb2c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.905+0000 7fbb23fff700 1 -- 172.21.15.49:0/4042546513 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c0d1740 con 0x7fbb2c0cf030 2023-07-18T03:38:43.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.906+0000 7fbb23fff700 1 --2- 172.21.15.49:0/4042546513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb2c0cf030 0x7fbb2c0cfc60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0cf860 tx=0x7fbb14009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87dbf569542c0240 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.906+0000 7fbb2a7fc700 1 -- 172.21.15.49:0/4042546513 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14004230 con 0x7fbb2c0cf030 2023-07-18T03:38:43.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.906+0000 7fbb2a7fc700 1 -- 172.21.15.49:0/4042546513 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb14004390 con 0x7fbb2c0cf030 2023-07-18T03:38:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.906+0000 7fbb2a7fc700 1 -- 172.21.15.49:0/4042546513 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb140036a0 con 0x7fbb2c0cf030 2023-07-18T03:38:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.907+0000 7fbb313ff700 1 -- 172.21.15.49:0/4042546513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb2c0cf030 msgr2=0x7fbb2c0cfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.907+0000 7fbb313ff700 1 --2- 172.21.15.49:0/4042546513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb2c0cf030 0x7fbb2c0cfc60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0cf860 tx=0x7fbb14009b20 comp rx=0 tx=0).stop 2023-07-18T03:38:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.907+0000 7f4eb67fc700 1 -- 172.21.15.49:0/3409510591 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc011800 con 0x7f4ec00cdbe0 2023-07-18T03:38:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.907+0000 7f4eb67fc700 1 -- 172.21.15.49:0/3409510591 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc011960 con 0x7f4ec00cdbe0 2023-07-18T03:38:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.907+0000 7f4eb67fc700 1 -- 172.21.15.49:0/3409510591 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc011cd0 con 0x7f4ec00cdbe0 2023-07-18T03:38:43.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7f3db4dff700 1 -- 172.21.15.49:0/3678112602 shutdown_connections 2023-07-18T03:38:43.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7f3db4dff700 1 --2- 172.21.15.49:0/3678112602 >> v2:172.21.15.49:3300/0 conn(0x7f3db00ce480 0x7f3db00cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7f3db4dff700 1 --2- 172.21.15.49:0/3678112602 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3db01587d0 0x7f3db006b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7f3db4dff700 1 --2- 172.21.15.49:0/3678112602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db006baf0 0x7f3db00cdf40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7f3db4dff700 1 -- 172.21.15.49:0/3678112602 >> 172.21.15.49:0/3678112602 conn(0x7f3db00b05d0 msgr2=0x7f3db00b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7f4ec7fff700 1 -- 172.21.15.49:0/3409510591 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec00712c0 con 0x7f4ec00cdbe0 2023-07-18T03:38:43.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7f3db4dff700 1 -- 172.21.15.49:0/3678112602 shutdown_connections 2023-07-18T03:38:43.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7fbb313ff700 1 -- 172.21.15.49:0/4042546513 shutdown_connections 2023-07-18T03:38:43.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7fbb313ff700 1 --2- 172.21.15.49:0/4042546513 >> v2:172.21.15.107:3300/0 conn(0x7fbb2c0cf890 0x7fbb2c0b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7fbb313ff700 1 --2- 172.21.15.49:0/4042546513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb2c0cf030 0x7fbb2c0cfc60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7fbb313ff700 1 -- 172.21.15.49:0/4042546513 >> 172.21.15.49:0/4042546513 conn(0x7fbb2c06c170 msgr2=0x7fbb2c06c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7f3e75dff700 1 Processor -- start 2023-07-18T03:38:43.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7fba1e1ff700 1 -- 172.21.15.49:0/3989695371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1806c690 con 0x7fba180cdbe0 2023-07-18T03:38:43.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7f4ec7fff700 1 -- 172.21.15.49:0/3409510591 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec0071880 con 0x7f4ec00cdbe0 2023-07-18T03:38:43.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.909+0000 7f4ec7fff700 1 -- 172.21.15.49:0/3409510591 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec00b1b90 con 0x7f4ec00cdbe0 2023-07-18T03:38:43.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.909+0000 7f3e75dff700 1 -- start start 2023-07-18T03:38:43.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.909+0000 7fbb313ff700 1 -- 172.21.15.49:0/4042546513 shutdown_connections 2023-07-18T03:38:43.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.909+0000 7fbb313ff700 1 -- 172.21.15.49:0/4042546513 wait complete. 2023-07-18T03:38:43.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.910+0000 7fba14ff9700 1 -- 172.21.15.49:0/3989695371 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba08007470 con 0x7fba180cdbe0 2023-07-18T03:38:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.911+0000 7f3e75dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700b21f0 0x7f3e700cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.911+0000 7f3e75dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e700cd8e0 con 0x7f3e700cf890 2023-07-18T03:38:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.911+0000 7f3e75dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e700d2760 con 0x7f3e700d29c0 2023-07-18T03:38:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.911+0000 7f3e75dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700d01a0 con 0x7f3e700b21f0 2023-07-18T03:38:43.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.911+0000 7fba14ff9700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb9f806adb0 0x7fb9f806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.911+0000 7fba14ff9700 1 -- 172.21.15.49:0/3989695371 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba080893e0 con 0x7fba180cdbe0 2023-07-18T03:38:43.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.912+0000 7f3e6ffff700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f3e700d29c0 0x7f3e700d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:53492/0 (socket says 172.21.15.49:53492) 2023-07-18T03:38:43.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.912+0000 7f3e6ffff700 1 -- 172.21.15.49:0/3899149504 learned_addr learned my addr 172.21.15.49:0/3899149504 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:43.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.908+0000 7f3db4dff700 1 -- 172.21.15.49:0/3678112602 wait complete. 2023-07-18T03:38:43.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.911+0000 7f3db4dff700 1 Processor -- start 2023-07-18T03:38:43.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.912+0000 7fba0effd700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb9f806adb0 0x7fb9f806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.912+0000 7fbb313ff700 1 Processor -- start 2023-07-18T03:38:43.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.912+0000 7f3e6effd700 1 --2- 172.21.15.49:0/3899149504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700b21f0 0x7f3e700cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.913+0000 7f3e6effd700 1 -- 172.21.15.49:0/3899149504 >> v1:172.21.15.107:6789/0 conn(0x7f3e700d29c0 legacy=0x7f3e700d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.913+0000 7f3e6effd700 1 -- 172.21.15.49:0/3899149504 >> v1:172.21.15.49:6789/0 conn(0x7f3e700cf890 legacy=0x7f3e700cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.913+0000 7f3e6effd700 1 -- 172.21.15.49:0/3899149504 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e700d1740 con 0x7f3e700b21f0 2023-07-18T03:38:43.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.913+0000 7f3e6effd700 1 --2- 172.21.15.49:0/3899149504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700b21f0 0x7f3e700cfc60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009870 tx=0x7f3e64009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc973383631503be server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.913+0000 7f3db4dff700 1 -- start start 2023-07-18T03:38:43.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.914+0000 7fba0effd700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb9f806adb0 0x7fb9f806d260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fba00000c00 tx=0x7fba00008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.914+0000 7f3db4dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db00ce480 0x7f3db01f1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.914+0000 7f3db4dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db01587d0 0x7f3db01f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.914+0000 7f3db4dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3db01f5280 0x7f3db00c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.914+0000 7f3db4dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db015bb40 con 0x7f3db00ce480 2023-07-18T03:38:43.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.914+0000 7f3db4dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db01f5700 con 0x7f3db01587d0 2023-07-18T03:38:43.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.914+0000 7f3db4dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db00c32a0 con 0x7f3db01f5280 2023-07-18T03:38:43.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.913+0000 7fbb313ff700 1 -- start start 2023-07-18T03:38:43.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.914+0000 7f4eb67fc700 1 -- 172.21.15.49:0/3409510591 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ebc01a450 con 0x7f4ec00cdbe0 2023-07-18T03:38:43.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.915+0000 7f3daed9d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3db01f5280 0x7f3db00c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.915+0000 7f3daed9d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3db01f5280 0x7f3db00c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46756/0 (socket says 172.21.15.49:46756) 2023-07-18T03:38:43.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.915+0000 7f3daed9d700 1 -- 172.21.15.49:0/890313993 learned_addr learned my addr 172.21.15.49:0/890313993 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.916+0000 7fba14ff9700 1 -- 172.21.15.49:0/3989695371 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fba08054500 con 0x7fba180cdbe0 2023-07-18T03:38:43.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.915+0000 7fbb313ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb2c0cf030 0x7fbb2c156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.915+0000 7fbb313ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb2c0cf890 0x7fbb2c0b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.915+0000 7fbb313ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb2c1601b0 0x7fbb2c0b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.915+0000 7fbb313ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0cb4c0 con 0x7fbb2c0cf890 2023-07-18T03:38:43.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.915+0000 7fbb313ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c15b000 con 0x7fbb2c1601b0 2023-07-18T03:38:43.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.915+0000 7fbb313ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c15b2e0 con 0x7fbb2c0cf030 2023-07-18T03:38:43.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.915+0000 7f4eb67fc700 1 --2- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4ea406adb0 0x7f4ea406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.916+0000 7f4eb67fc700 1 -- 172.21.15.49:0/3409510591 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4ebc0893e0 con 0x7f4ec00cdbe0 2023-07-18T03:38:43.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.916+0000 7fbb2b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb2c1601b0 0x7fbb2c0b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.916+0000 7fbb2b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb2c1601b0 0x7fbb2c0b32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45254/0 (socket says 172.21.15.49:45254) 2023-07-18T03:38:43.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.916+0000 7fbb2b7fe700 1 -- 172.21.15.49:0/3004180385 learned_addr learned my addr 172.21.15.49:0/3004180385 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.916+0000 7fbb2affd700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb2c0cf030 0x7fbb2c156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.916+0000 7f4ec559a700 1 --2- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4ea406adb0 0x7f4ea406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7fad84dff700 1 Processor -- start 2023-07-18T03:38:43.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7fad84dff700 1 -- start start 2023-07-18T03:38:43.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7fbb23fff700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb2c0cf890 0x7fbb2c0b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7fbb2b7fe700 1 -- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb2c0cf030 msgr2=0x7fbb2c156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7fbb2b7fe700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb2c0cf030 0x7fbb2c156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7fbb2b7fe700 1 -- 172.21.15.49:0/3004180385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb2c0cf890 msgr2=0x7fbb2c0b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7fbb2b7fe700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb2c0cf890 0x7fbb2c0b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7fbb2b7fe700 1 -- 172.21.15.49:0/3004180385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb140094a0 con 0x7fbb2c1601b0 2023-07-18T03:38:43.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.918+0000 7f4ec559a700 1 --2- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4ea406adb0 0x7f4ea406d260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8008cd0 tx=0x7f4eb8008200 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.918+0000 7fbb2b7fe700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb2c1601b0 0x7fbb2c0b32b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400c780 tx=0x7fbb2400cb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.916+0000 7f3dae59c700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db00ce480 0x7f3db01f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.916+0000 7f3dadd9b700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db01587d0 0x7f3db01f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7f3dadd9b700 1 -- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3db01f5280 msgr2=0x7f3db00c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7f3dadd9b700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3db01f5280 0x7f3db00c4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7f3dadd9b700 1 -- 172.21.15.49:0/890313993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db00ce480 msgr2=0x7f3db01f1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7f3dadd9b700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db00ce480 0x7f3db01f1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.917+0000 7f3dadd9b700 1 -- 172.21.15.49:0/890313993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da000a5f0 con 0x7f3db01587d0 2023-07-18T03:38:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.918+0000 7f3dadd9b700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db01587d0 0x7f3db01f4d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3da0017000 tx=0x7f3da0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.919+0000 7f3d9effd700 1 -- 172.21.15.49:0/890313993 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da000acc0 con 0x7f3db01587d0 2023-07-18T03:38:43.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.919+0000 7f3d9effd700 1 -- 172.21.15.49:0/890313993 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da00038c0 con 0x7f3db01587d0 2023-07-18T03:38:43.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.919+0000 7f3d9effd700 1 -- 172.21.15.49:0/890313993 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da0003c70 con 0x7f3db01587d0 2023-07-18T03:38:43.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.919+0000 7f3e6e7fc700 1 -- 172.21.15.49:0/3899149504 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64011830 con 0x7f3e700b21f0 2023-07-18T03:38:43.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.919+0000 7f3e6e7fc700 1 -- 172.21.15.49:0/3899149504 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e64011990 con 0x7f3e700b21f0 2023-07-18T03:38:43.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.919+0000 7f3e6e7fc700 1 -- 172.21.15.49:0/3899149504 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64011d20 con 0x7f3e700b21f0 2023-07-18T03:38:43.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.919+0000 7f4eb67fc700 1 -- 172.21.15.49:0/3409510591 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4ebc057f90 con 0x7f4ec00cdbe0 2023-07-18T03:38:43.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.918+0000 7fbb237fe700 1 -- 172.21.15.49:0/3004180385 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24004c80 con 0x7fbb2c1601b0 2023-07-18T03:38:43.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.918+0000 7fbb237fe700 1 -- 172.21.15.49:0/3004180385 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb24004de0 con 0x7fbb2c1601b0 2023-07-18T03:38:43.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.918+0000 7fbb237fe700 1 -- 172.21.15.49:0/3004180385 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24007310 con 0x7fbb2c1601b0 2023-07-18T03:38:43.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.920+0000 7fbb313ff700 1 -- 172.21.15.49:0/3004180385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c0cb710 con 0x7fbb2c1601b0 2023-07-18T03:38:43.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.920+0000 7fbb313ff700 1 -- 172.21.15.49:0/3004180385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2c156dc0 con 0x7fbb2c1601b0 2023-07-18T03:38:43.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.919+0000 7fad84dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fad800b21f0 0x7fad800cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.919+0000 7fad84dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d29c0 0x7fad800d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.920+0000 7fad84dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad800cd8e0 con 0x7fad800cf890 2023-07-18T03:38:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.920+0000 7fad84dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad800d2760 con 0x7fad800b21f0 2023-07-18T03:38:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.920+0000 7fad84dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad800d01a0 con 0x7fad800d29c0 2023-07-18T03:38:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.920+0000 7fad7e59c700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fad800cf890 0x7fad800cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:53504/0 (socket says 172.21.15.49:53504) 2023-07-18T03:38:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.920+0000 7fad7e59c700 1 -- 172.21.15.49:0/3483354728 learned_addr learned my addr 172.21.15.49:0/3483354728 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.920+0000 7fad7ed9d700 1 --2- 172.21.15.49:0/3483354728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d29c0 0x7fad800d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.921+0000 7fad7dd9b700 1 --2- 172.21.15.49:0/3483354728 >> v2:172.21.15.107:3300/0 conn(0x7fad800b21f0 0x7fad800cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.921+0000 7fad7d59a700 1 -- 172.21.15.49:0/3483354728 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055940953 0 0) 0x7fad800cd8e0 con 0x7fad800cf890 2023-07-18T03:38:43.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.921+0000 7fad7d59a700 1 -- 172.21.15.49:0/3483354728 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad6c0035b0 con 0x7fad800cf890 2023-07-18T03:38:43.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.921+0000 7fad7ed9d700 1 -- 172.21.15.49:0/3483354728 >> v2:172.21.15.107:3300/0 conn(0x7fad800b21f0 msgr2=0x7fad800cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.921+0000 7fad7ed9d700 1 --2- 172.21.15.49:0/3483354728 >> v2:172.21.15.107:3300/0 conn(0x7fad800b21f0 0x7fad800cfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.921+0000 7fad7ed9d700 1 -- 172.21.15.49:0/3483354728 >> v1:172.21.15.107:6789/0 conn(0x7fad800cf890 legacy=0x7fad800cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.921+0000 7fad7ed9d700 1 -- 172.21.15.49:0/3483354728 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad800d1740 con 0x7fad800d29c0 2023-07-18T03:38:43.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.920+0000 7f3e75dff700 1 -- 172.21.15.49:0/3899149504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700b21f0 msgr2=0x7f3e700cfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.920+0000 7f3e75dff700 1 --2- 172.21.15.49:0/3899149504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700b21f0 0x7f3e700cfc60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009870 tx=0x7f3e64009b80 comp rx=0 tx=0).stop 2023-07-18T03:38:43.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.921+0000 7f3e75dff700 1 -- 172.21.15.49:0/3899149504 shutdown_connections 2023-07-18T03:38:43.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.921+0000 7f3e75dff700 1 --2- 172.21.15.49:0/3899149504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700b21f0 0x7f3e700cfc60 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009870 tx=0x7f3e64009b80 comp rx=0 tx=0).stop 2023-07-18T03:38:43.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.921+0000 7f3e75dff700 1 -- 172.21.15.49:0/3899149504 >> 172.21.15.49:0/3899149504 conn(0x7f3e7006c170 msgr2=0x7f3e7006c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.922+0000 7f3db4dff700 1 -- 172.21.15.49:0/890313993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db00c2690 con 0x7f3db01587d0 2023-07-18T03:38:43.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.922+0000 7f3db4dff700 1 -- 172.21.15.49:0/890313993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db00c2c50 con 0x7f3db01587d0 2023-07-18T03:38:43.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.922+0000 7f3e75dff700 1 -- 172.21.15.49:0/3899149504 shutdown_connections 2023-07-18T03:38:43.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.922+0000 7fad7ed9d700 1 --2- 172.21.15.49:0/3483354728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d29c0 0x7fad800d2e40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad70009840 tx=0x7fad70009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47616dbe7d538462 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.921+0000 7fbb313ff700 1 -- 172.21.15.49:0/3004180385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2c0cbc30 con 0x7fbb2c1601b0 2023-07-18T03:38:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.923+0000 7fad7d59a700 1 -- 172.21.15.49:0/3483354728 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70011830 con 0x7fad800d29c0 2023-07-18T03:38:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.923+0000 7fad7d59a700 1 -- 172.21.15.49:0/3483354728 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad70011990 con 0x7fad800d29c0 2023-07-18T03:38:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.923+0000 7fad7d59a700 1 -- 172.21.15.49:0/3483354728 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70011d20 con 0x7fad800d29c0 2023-07-18T03:38:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.924+0000 7fad84dff700 1 -- 172.21.15.49:0/3483354728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d29c0 msgr2=0x7fad800d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.924+0000 7fad84dff700 1 --2- 172.21.15.49:0/3483354728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d29c0 0x7fad800d2e40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad70009840 tx=0x7fad70009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.924+0000 7fbb237fe700 1 -- 172.21.15.49:0/3004180385 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbb24007470 con 0x7fbb2c1601b0 2023-07-18T03:38:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.924+0000 7fad84dff700 1 -- 172.21.15.49:0/3483354728 shutdown_connections 2023-07-18T03:38:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.924+0000 7fad84dff700 1 --2- 172.21.15.49:0/3483354728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d29c0 0x7fad800d2e40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.924+0000 7fad84dff700 1 --2- 172.21.15.49:0/3483354728 >> v2:172.21.15.107:3300/0 conn(0x7fad800b21f0 0x7fad800cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.924+0000 7fad84dff700 1 -- 172.21.15.49:0/3483354728 >> 172.21.15.49:0/3483354728 conn(0x7fad8006c170 msgr2=0x7fad8006c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.922+0000 7f3e75dff700 1 -- 172.21.15.49:0/3899149504 wait complete. 2023-07-18T03:38:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.925+0000 7fad84dff700 1 -- 172.21.15.49:0/3483354728 shutdown_connections 2023-07-18T03:38:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.925+0000 7f3e75dff700 1 Processor -- start 2023-07-18T03:38:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.925+0000 7fbb237fe700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb0c06ad30 0x7fbb0c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.925+0000 7fbb237fe700 1 -- 172.21.15.49:0/3004180385 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbb24088a20 con 0x7fbb2c1601b0 2023-07-18T03:38:43.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.925+0000 7fad84dff700 1 -- 172.21.15.49:0/3483354728 wait complete. 2023-07-18T03:38:43.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.925+0000 7f3e75dff700 1 -- start start 2023-07-18T03:38:43.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.926+0000 7fbb2affd700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb0c06ad30 0x7fbb0c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.927+0000 7f3e75dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e700cf890 0x7f3e700ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.927+0000 7f3e75dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700d29c0 0x7f3e700c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.927+0000 7f3e75dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7015f230 0x7f3e7006f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.927+0000 7f3e75dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7006eac0 con 0x7f3e7015f230 2023-07-18T03:38:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.927+0000 7f3e75dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7006dc60 con 0x7f3e700cf890 2023-07-18T03:38:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.927+0000 7f3e75dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7006df40 con 0x7f3e700d29c0 2023-07-18T03:38:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.928+0000 7f3e6effd700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700d29c0 0x7f3e700c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.928+0000 7f3e6effd700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700d29c0 0x7f3e700c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:46780/0 (socket says 172.21.15.49:46780) 2023-07-18T03:38:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.928+0000 7f3e6effd700 1 -- 172.21.15.49:0/358283703 learned_addr learned my addr 172.21.15.49:0/358283703 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.928+0000 7f3e6f7fe700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e700cf890 0x7f3e700ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.928+0000 7fad84dff700 1 Processor -- start 2023-07-18T03:38:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.928+0000 7f3e6ffff700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7015f230 0x7f3e7006f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.928+0000 7f3e6effd700 1 -- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e700cf890 msgr2=0x7f3e700ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.928+0000 7f3e6effd700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e700cf890 0x7f3e700ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.928+0000 7f3e6effd700 1 -- 172.21.15.49:0/358283703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7015f230 msgr2=0x7f3e7006f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.928+0000 7f3e6effd700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7015f230 0x7f3e7006f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.929+0000 7f3e6effd700 1 -- 172.21.15.49:0/358283703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e640094a0 con 0x7f3e700d29c0 2023-07-18T03:38:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.929+0000 7f3d9effd700 1 -- 172.21.15.49:0/890313993 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3da001c070 con 0x7f3db01587d0 2023-07-18T03:38:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.929+0000 7f3db4dff700 1 -- 172.21.15.49:0/890313993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db006ca40 con 0x7f3db01587d0 2023-07-18T03:38:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.929+0000 7fad84dff700 1 -- start start 2023-07-18T03:38:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.929+0000 7f3e6ffff700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7015f230 0x7f3e7006f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.930+0000 7fbb237fe700 1 -- 172.21.15.49:0/3004180385 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbb2c0cbc30 con 0x7fbb2c1601b0 2023-07-18T03:38:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.930+0000 7fbb2affd700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb0c06ad30 0x7fbb0c06d1e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0074a0 tx=0x7fbb1c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.931+0000 7fad84dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800cf890 0x7fad8015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.931+0000 7fad84dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad800d29c0 0x7fad801531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.931+0000 7fad84dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8006a6d0 0x7fad8006ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.931+0000 7fad84dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8006b0b0 con 0x7fad800d29c0 2023-07-18T03:38:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.931+0000 7fad84dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8015b5a0 con 0x7fad8006a6d0 2023-07-18T03:38:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.931+0000 7fad84dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8015b880 con 0x7fad800cf890 2023-07-18T03:38:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.931+0000 7f3d9effd700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d9006ad30 0x7f3d9006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.931+0000 7f3d9effd700 1 -- 172.21.15.49:0/890313993 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3da008a970 con 0x7f3db01587d0 2023-07-18T03:38:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.931+0000 7f3dae59c700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d9006ad30 0x7f3d9006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.931+0000 7f3e6effd700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700d29c0 0x7f3e700c8d10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000c00 tx=0x7f3e64004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.932+0000 7fad7ed9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8006a6d0 0x7fad8006ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.932+0000 7f3dae59c700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d9006ad30 0x7f3d9006d1e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3da400dde0 tx=0x7f3da400f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.932+0000 7f3e57fff700 1 -- 172.21.15.49:0/358283703 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e640117c0 con 0x7f3e700d29c0 2023-07-18T03:38:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.932+0000 7f3e57fff700 1 -- 172.21.15.49:0/358283703 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6401f070 con 0x7f3e700d29c0 2023-07-18T03:38:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.932+0000 7f3e57fff700 1 -- 172.21.15.49:0/358283703 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e640038a0 con 0x7f3e700d29c0 2023-07-18T03:38:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.932+0000 7fad7ed9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8006a6d0 0x7fad8006ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45288/0 (socket says 172.21.15.49:45288) 2023-07-18T03:38:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.932+0000 7fad7ed9d700 1 -- 172.21.15.49:0/2750486489 learned_addr learned my addr 172.21.15.49:0/2750486489 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.932+0000 7fad7ed9d700 1 -- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800cf890 msgr2=0x7fad8015ce00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:38:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.932+0000 7fad7ed9d700 1 --2- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800cf890 0x7fad8015ce00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.932+0000 7fad7ed9d700 1 -- 172.21.15.49:0/2750486489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad800d29c0 msgr2=0x7fad801531d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:38:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.932+0000 7fad7ed9d700 1 --2- 172.21.15.49:0/2750486489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad800d29c0 0x7fad801531d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.932+0000 7fad7ed9d700 1 -- 172.21.15.49:0/2750486489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad700094a0 con 0x7fad8006a6d0 2023-07-18T03:38:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.933+0000 7fad7ed9d700 1 --2- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8006a6d0 0x7fad8006ab70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad70009b90 tx=0x7fad70004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.933+0000 7f3e75dff700 1 -- 172.21.15.49:0/358283703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7006e220 con 0x7f3e700d29c0 2023-07-18T03:38:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.933+0000 7f3e75dff700 1 -- 172.21.15.49:0/358283703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7006e7e0 con 0x7f3e700d29c0 2023-07-18T03:38:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.931+0000 7f255afff700 1 Processor -- start 2023-07-18T03:38:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.931+0000 7f255afff700 1 -- start start 2023-07-18T03:38:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.933+0000 7f255afff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2554164e20 0x7f25540ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.933+0000 7f255afff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25540cda00 con 0x7f25540cdbe0 2023-07-18T03:38:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.933+0000 7f255afff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2554161000 con 0x7f2554160c30 2023-07-18T03:38:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.933+0000 7f255afff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25540ce5a0 con 0x7f2554164e20 2023-07-18T03:38:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.933+0000 7f2558d9b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f2554160c30 0x7f25540cd2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:53510/0 (socket says 172.21.15.49:53510) 2023-07-18T03:38:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.933+0000 7f2558d9b700 1 -- 172.21.15.49:0/2042148581 learned_addr learned my addr 172.21.15.49:0/2042148581 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.935+0000 7f3e75dff700 1 -- 172.21.15.49:0/358283703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e700ca0b0 con 0x7f3e700d29c0 2023-07-18T03:38:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.935+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308710525 0 0) 0x7f25540cda00 con 0x7f25540cdbe0 2023-07-18T03:38:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.935+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25440035b0 con 0x7f25540cdbe0 2023-07-18T03:38:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f3d9effd700 1 -- 172.21.15.49:0/890313993 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3da0055b90 con 0x7f3db01587d0 2023-07-18T03:38:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f255959c700 1 --2- 172.21.15.49:0/2042148581 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2554164e20 0x7f25540ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1625145240 0 0) 0x7f25440035b0 con 0x7f25540cdbe0 2023-07-18T03:38:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2554164e20 msgr2=0x7f25540ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 --2- 172.21.15.49:0/2042148581 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2554164e20 0x7f25540ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 >> v1:172.21.15.107:6789/0 conn(0x7f2554160c30 legacy=0x7f25540cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2554165400 con 0x7f25540cdbe0 2023-07-18T03:38:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f2540001670 con 0x7f25540cdbe0 2023-07-18T03:38:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 >> v1:172.21.15.49:6789/0 conn(0x7f25540cdbe0 legacy=0x7f2554164710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 --2- 172.21.15.49:0/2042148581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544003a30 0x7f2544003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 --2- 172.21.15.49:0/2042148581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2544004850 0x7f2544004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 --2- 172.21.15.49:0/2042148581 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2544005200 0x7f254400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25540ce5a0 con 0x7f2544003a30 2023-07-18T03:38:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254400c080 con 0x7f2544004850 2023-07-18T03:38:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.936+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254400c240 con 0x7f2544005200 2023-07-18T03:38:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.937+0000 7f2553fff700 1 --2- 172.21.15.49:0/2042148581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2544004850 0x7f2544004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.938+0000 7f255959c700 1 --2- 172.21.15.49:0/2042148581 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2544005200 0x7f254400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.938+0000 7fad76ffd700 1 -- 172.21.15.49:0/2750486489 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad700044d0 con 0x7fad8006a6d0 2023-07-18T03:38:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.938+0000 7fad76ffd700 1 -- 172.21.15.49:0/2750486489 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7001f070 con 0x7fad8006a6d0 2023-07-18T03:38:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.938+0000 7fad76ffd700 1 -- 172.21.15.49:0/2750486489 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad700038a0 con 0x7fad8006a6d0 2023-07-18T03:38:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.938+0000 7fad84dff700 1 -- 172.21.15.49:0/2750486489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8015bb60 con 0x7fad8006a6d0 2023-07-18T03:38:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.938+0000 7fad84dff700 1 -- 172.21.15.49:0/2750486489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad80153710 con 0x7fad8006a6d0 2023-07-18T03:38:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.938+0000 7f3e57fff700 1 -- 172.21.15.49:0/358283703 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e64003a00 con 0x7f3e700d29c0 2023-07-18T03:38:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.939+0000 7f2553fff700 1 -- 172.21.15.49:0/2042148581 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2544005200 msgr2=0x7f254400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.939+0000 7f2553fff700 1 --2- 172.21.15.49:0/2042148581 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2544005200 0x7f254400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.939+0000 7f2553fff700 1 -- 172.21.15.49:0/2042148581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544003a30 msgr2=0x7f2544003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.939+0000 7f2553fff700 1 --2- 172.21.15.49:0/2042148581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544003a30 0x7f2544003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.939+0000 7f2553fff700 1 -- 172.21.15.49:0/2042148581 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2554165400 con 0x7f2544004850 2023-07-18T03:38:43.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.939+0000 7f3e57fff700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3e5c06adb0 0x7f3e5c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.939+0000 7f3e57fff700 1 -- 172.21.15.49:0/358283703 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e64021070 con 0x7f3e700d29c0 2023-07-18T03:38:43.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.940+0000 7f3e6f7fe700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3e5c06adb0 0x7f3e5c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.942+0000 7f2553fff700 1 --2- 172.21.15.49:0/2042148581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2544004850 0x7f2544004cc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f254000c190 tx=0x7f2544005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b557da22600d62a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.942+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000cd30 con 0x7f2544004850 2023-07-18T03:38:43.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.942+0000 7f3e6f7fe700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3e5c06adb0 0x7f3e5c06d260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800f290 tx=0x7f3e68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.943+0000 7f3e57fff700 1 -- 172.21.15.49:0/358283703 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3e640561b0 con 0x7f3e700d29c0 2023-07-18T03:38:43.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.943+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254000ce90 con 0x7f2544004850 2023-07-18T03:38:43.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.943+0000 7f25537fe700 1 -- 172.21.15.49:0/2042148581 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540012660 con 0x7f2544004850 2023-07-18T03:38:43.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.944+0000 7f255afff700 1 -- 172.21.15.49:0/2042148581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2544004850 msgr2=0x7f2544004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.944+0000 7f255afff700 1 --2- 172.21.15.49:0/2042148581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2544004850 0x7f2544004cc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f254000c190 tx=0x7f2544005880 comp rx=0 tx=0).stop 2023-07-18T03:38:43.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.944+0000 7f255959c700 1 -- 172.21.15.49:0/2042148581 reap_dead start 2023-07-18T03:38:43.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.943+0000 7fad76ffd700 1 -- 172.21.15.49:0/2750486489 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad70003a00 con 0x7fad8006a6d0 2023-07-18T03:38:43.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.945+0000 7fad76ffd700 1 --2- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad6006ad30 0x7fad6006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.945+0000 7fad76ffd700 1 -- 172.21.15.49:0/2750486489 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad70021070 con 0x7fad8006a6d0 2023-07-18T03:38:43.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.946+0000 7f255afff700 1 -- 172.21.15.49:0/2042148581 shutdown_connections 2023-07-18T03:38:43.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.946+0000 7f255afff700 1 -- 172.21.15.49:0/2042148581 >> 172.21.15.49:0/2042148581 conn(0x7f25540b05d0 msgr2=0x7f25540b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:43.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.946+0000 7fad7e59c700 1 --2- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad6006ad30 0x7fad6006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.946+0000 7fad84dff700 1 -- 172.21.15.49:0/2750486489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad640052f0 con 0x7fad8006a6d0 2023-07-18T03:38:43.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.947+0000 7fad7e59c700 1 --2- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad6006ad30 0x7fad6006d1e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fad78007fd0 tx=0x7fad78007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.952+0000 7fad76ffd700 1 -- 172.21.15.49:0/2750486489 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fad70055da0 con 0x7fad8006a6d0 2023-07-18T03:38:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.952+0000 7f255afff700 1 -- 172.21.15.49:0/2042148581 shutdown_connections 2023-07-18T03:38:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.953+0000 7f255afff700 1 -- 172.21.15.49:0/2042148581 wait complete. 2023-07-18T03:38:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.956+0000 7f255afff700 1 Processor -- start 2023-07-18T03:38:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.956+0000 7f255afff700 1 -- start start 2023-07-18T03:38:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.958+0000 7f255afff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544004850 0x7f2554163800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.958+0000 7f255afff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2554161190 0x7f255408a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.958+0000 7f255afff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f255406b5e0 0x7f255406ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.958+0000 7f255afff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25541640c0 con 0x7f2544004850 2023-07-18T03:38:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.958+0000 7f255afff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255406bf90 con 0x7f2554161190 2023-07-18T03:38:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.958+0000 7f255afff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25540c3300 con 0x7f255406b5e0 2023-07-18T03:38:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.959+0000 7f2553fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2554161190 0x7f255408a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.959+0000 7f255959c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f255406b5e0 0x7f255406ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.960+0000 7f2558d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544004850 0x7f2554163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.960+0000 7f2558d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544004850 0x7f2554163800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40506/0 (socket says 172.21.15.49:40506) 2023-07-18T03:38:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.960+0000 7f2558d9b700 1 -- 172.21.15.49:0/2014684130 learned_addr learned my addr 172.21.15.49:0/2014684130 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.961+0000 7f2558d9b700 1 -- 172.21.15.49:0/2014684130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f255406b5e0 msgr2=0x7f255406ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.961+0000 7f2558d9b700 1 --2- 172.21.15.49:0/2014684130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f255406b5e0 0x7f255406ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.961+0000 7f2558d9b700 1 -- 172.21.15.49:0/2014684130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2554161190 msgr2=0x7f255408a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.961+0000 7f2558d9b700 1 --2- 172.21.15.49:0/2014684130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2554161190 0x7f255408a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.961+0000 7f2558d9b700 1 -- 172.21.15.49:0/2014684130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255408aa40 con 0x7f2544004850 2023-07-18T03:38:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.961+0000 7f2558d9b700 1 --2- 172.21.15.49:0/2014684130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544004850 0x7f2554163800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f254800cc80 tx=0x7f254800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.965+0000 7f25517fa700 1 -- 172.21.15.49:0/2014684130 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254800ada0 con 0x7f2544004850 2023-07-18T03:38:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.965+0000 7f25517fa700 1 -- 172.21.15.49:0/2014684130 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254800ce80 con 0x7f2544004850 2023-07-18T03:38:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.965+0000 7f25517fa700 1 -- 172.21.15.49:0/2014684130 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548012620 con 0x7f2544004850 2023-07-18T03:38:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.965+0000 7f255afff700 1 -- 172.21.15.49:0/2014684130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25540c2cf0 con 0x7f2544004850 2023-07-18T03:38:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.965+0000 7f255afff700 1 -- 172.21.15.49:0/2014684130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25540c2f90 con 0x7f2544004850 2023-07-18T03:38:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.970+0000 7f25517fa700 1 -- 172.21.15.49:0/2014684130 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2548012780 con 0x7f2544004850 2023-07-18T03:38:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.972+0000 7f25517fa700 1 --2- 172.21.15.49:0/2014684130 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f253806adb0 0x7f253806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.972+0000 7f25517fa700 1 -- 172.21.15.49:0/2014684130 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f25480892b0 con 0x7f2544004850 2023-07-18T03:38:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.973+0000 7f255afff700 1 -- 172.21.15.49:0/2014684130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25340052f0 con 0x7f2544004850 2023-07-18T03:38:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.975+0000 7f2553fff700 1 --2- 172.21.15.49:0/2014684130 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f253806adb0 0x7f253806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.984+0000 7f25517fa700 1 -- 172.21.15.49:0/2014684130 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2548054480 con 0x7f2544004850 2023-07-18T03:38:43.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:43.991+0000 7f2553fff700 1 --2- 172.21.15.49:0/2014684130 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f253806adb0 0x7f253806d260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2540014040 tx=0x7f2540005040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:44.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.359+0000 7fba1e1ff700 1 -- 172.21.15.49:0/3989695371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fba180c5950 con 0x7fba180cdbe0 2023-07-18T03:38:44.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.360+0000 7fba14ff9700 1 -- 172.21.15.49:0/3989695371 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba08018090 con 0x7fba180cdbe0 2023-07-18T03:38:44.358 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2023-07-18T03:38:44.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.364+0000 7fba1e1ff700 1 -- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb9f806adb0 msgr2=0x7fb9f806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.364+0000 7fba1e1ff700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb9f806adb0 0x7fb9f806d260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fba00000c00 tx=0x7fba00008030 comp rx=0 tx=0).stop 2023-07-18T03:38:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.364+0000 7fba1e1ff700 1 -- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba180cdbe0 msgr2=0x7fba181eb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.364+0000 7fba1e1ff700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba180cdbe0 0x7fba181eb450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fba08001fc0 tx=0x7fba0800a340 comp rx=0 tx=0).stop 2023-07-18T03:38:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.365+0000 7fba1e1ff700 1 -- 172.21.15.49:0/3989695371 shutdown_connections 2023-07-18T03:38:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.365+0000 7fba1e1ff700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba18164710 0x7fba18163210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.365+0000 7fba1e1ff700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba180cdbe0 0x7fba181eb450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.365+0000 7fba1e1ff700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb9f806adb0 0x7fb9f806d260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.365+0000 7fba1e1ff700 1 --2- 172.21.15.49:0/3989695371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba18160c30 0x7fba18160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.365+0000 7fba1e1ff700 1 -- 172.21.15.49:0/3989695371 >> 172.21.15.49:0/3989695371 conn(0x7fba180b05d0 msgr2=0x7fba180afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.368+0000 7fba1e1ff700 1 -- 172.21.15.49:0/3989695371 shutdown_connections 2023-07-18T03:38:44.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.368+0000 7fba1e1ff700 1 -- 172.21.15.49:0/3989695371 wait complete. 2023-07-18T03:38:44.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.376+0000 7f4ec7fff700 1 -- 172.21.15.49:0/3409510591 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4ec00b1b90 con 0x7f4ec00cdbe0 2023-07-18T03:38:44.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.377+0000 7f4eb67fc700 1 -- 172.21.15.49:0/3409510591 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4ec00b1b90 con 0x7f4ec00cdbe0 2023-07-18T03:38:44.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.378+0000 7f204cfff700 1 -- 172.21.15.49:0/261590174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f20480c5b90 con 0x7f20481587d0 2023-07-18T03:38:44.373 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2023-07-18T03:38:44.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.379+0000 7f203f7fe700 1 -- 172.21.15.49:0/261590174 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f20480c5b90 con 0x7f20481587d0 2023-07-18T03:38:44.376 INFO:teuthology.orchestra.run.smithi049.stdout:64424509443 2023-07-18T03:38:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f204cfff700 1 -- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f202806ace0 msgr2=0x7f202806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f204cfff700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f202806ace0 0x7f202806d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f20480b2630 tx=0x7f2030008040 comp rx=0 tx=0).stop 2023-07-18T03:38:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f204cfff700 1 -- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20481587d0 msgr2=0x7f2048158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f204cfff700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20481587d0 0x7f2048158ce0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f204000c370 tx=0x7f2040004ad0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f204cfff700 1 -- 172.21.15.49:0/261590174 shutdown_connections 2023-07-18T03:38:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f204cfff700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20481587d0 0x7f2048158ce0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f204cfff700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f202806ace0 0x7f202806d190 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f20480b2630 tx=0x7f2030008040 comp rx=0 tx=0).stop 2023-07-18T03:38:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f204cfff700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20480cdf40 0x7f20480af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f204cfff700 1 --2- 172.21.15.49:0/261590174 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20480ce3b0 0x7f20481541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f4ec7fff700 1 -- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4ea406adb0 msgr2=0x7f4ea406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f4ec7fff700 1 --2- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4ea406adb0 0x7f4ea406d260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8008cd0 tx=0x7f4eb8008200 comp rx=0 tx=0).stop 2023-07-18T03:38:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f4ec7fff700 1 -- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ec00cdbe0 msgr2=0x7f4ec01f3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f4ec7fff700 1 --2- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ec00cdbe0 0x7f4ec01f3940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00c7f0 tx=0x7f4ebc00cbb0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f204cfff700 1 -- 172.21.15.49:0/261590174 >> 172.21.15.49:0/261590174 conn(0x7f20480b05d0 msgr2=0x7f20480cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.388+0000 7f204cfff700 1 -- 172.21.15.49:0/261590174 shutdown_connections 2023-07-18T03:38:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f4ec7fff700 1 -- 172.21.15.49:0/3409510591 shutdown_connections 2023-07-18T03:38:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f4ec7fff700 1 --2- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0164710 0x7f4ec015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f4ec7fff700 1 --2- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ec00cdbe0 0x7f4ec01f3940 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f4ec7fff700 1 --2- 172.21.15.49:0/3409510591 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4ea406adb0 0x7f4ea406d260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f4ec7fff700 1 --2- 172.21.15.49:0/3409510591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ec0160c30 0x7f4ec015ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.387+0000 7f4ec7fff700 1 -- 172.21.15.49:0/3409510591 >> 172.21.15.49:0/3409510591 conn(0x7f4ec00b05d0 msgr2=0x7f4ec00afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.388+0000 7f204cfff700 1 -- 172.21.15.49:0/261590174 wait complete. 2023-07-18T03:38:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.388+0000 7f4ec7fff700 1 -- 172.21.15.49:0/3409510591 shutdown_connections 2023-07-18T03:38:44.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.388+0000 7f4ec7fff700 1 -- 172.21.15.49:0/3409510591 wait complete. 2023-07-18T03:38:44.399 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.4 2023-07-18T03:38:44.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.405+0000 7fbb313ff700 1 -- 172.21.15.49:0/3004180385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbb2c0cbc30 con 0x7fbb2c1601b0 2023-07-18T03:38:44.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.406+0000 7fbb237fe700 1 -- 172.21.15.49:0/3004180385 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb24018020 con 0x7fbb2c1601b0 2023-07-18T03:38:44.401 INFO:teuthology.orchestra.run.smithi049.stdout:64424509446 2023-07-18T03:38:44.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.409+0000 7fbb313ff700 1 -- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb0c06ad30 msgr2=0x7fbb0c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.409+0000 7fbb313ff700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb0c06ad30 0x7fbb0c06d1e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0074a0 tx=0x7fbb1c00b040 comp rx=0 tx=0).stop 2023-07-18T03:38:44.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.409+0000 7fbb313ff700 1 -- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb2c1601b0 msgr2=0x7fbb2c0b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.409+0000 7fbb313ff700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb2c1601b0 0x7fbb2c0b32b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400c780 tx=0x7fbb2400cb40 comp rx=0 tx=0).stop 2023-07-18T03:38:44.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.410+0000 7fbb313ff700 1 -- 172.21.15.49:0/3004180385 shutdown_connections 2023-07-18T03:38:44.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.410+0000 7fbb313ff700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb2c1601b0 0x7fbb2c0b32b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.410+0000 7fbb313ff700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb2c0cf030 0x7fbb2c156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.410+0000 7fbb313ff700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb0c06ad30 0x7fbb0c06d1e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.410+0000 7fbb313ff700 1 --2- 172.21.15.49:0/3004180385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb2c0cf890 0x7fbb2c0b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.410+0000 7fbb313ff700 1 -- 172.21.15.49:0/3004180385 >> 172.21.15.49:0/3004180385 conn(0x7fbb2c06c170 msgr2=0x7fbb2c0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:44.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.410+0000 7fbb313ff700 1 -- 172.21.15.49:0/3004180385 shutdown_connections 2023-07-18T03:38:44.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.410+0000 7f3db4dff700 1 -- 172.21.15.49:0/890313993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3db01f1960 con 0x7f3db01587d0 2023-07-18T03:38:44.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.410+0000 7fbb313ff700 1 -- 172.21.15.49:0/3004180385 wait complete. 2023-07-18T03:38:44.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.411+0000 7f3d9effd700 1 -- 172.21.15.49:0/890313993 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3da00591b0 con 0x7f3db01587d0 2023-07-18T03:38:44.408 INFO:teuthology.orchestra.run.smithi049.stdout:60129542147 2023-07-18T03:38:44.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.411+0000 7fad84dff700 1 -- 172.21.15.49:0/2750486489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fad640055a0 con 0x7fad8006a6d0 2023-07-18T03:38:44.409 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.2 2023-07-18T03:38:44.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.412+0000 7fad76ffd700 1 -- 172.21.15.49:0/2750486489 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fad70015020 con 0x7fad8006a6d0 2023-07-18T03:38:44.410 INFO:teuthology.orchestra.run.smithi049.stdout:60129542148 2023-07-18T03:38:44.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.413+0000 7f3e75dff700 1 -- 172.21.15.49:0/358283703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3e700ca0b0 con 0x7f3e700d29c0 2023-07-18T03:38:44.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.413+0000 7f3e57fff700 1 -- 172.21.15.49:0/358283703 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e700ca0b0 con 0x7f3e700d29c0 2023-07-18T03:38:44.411 INFO:teuthology.orchestra.run.smithi049.stdout:64424509444 2023-07-18T03:38:44.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.415+0000 7f3db4dff700 1 -- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d9006ad30 msgr2=0x7f3d9006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.415+0000 7f3db4dff700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d9006ad30 0x7f3d9006d1e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3da400dde0 tx=0x7f3da400f040 comp rx=0 tx=0).stop 2023-07-18T03:38:44.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.415+0000 7f3db4dff700 1 -- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db01587d0 msgr2=0x7f3db01f4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.415+0000 7f3db4dff700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db01587d0 0x7f3db01f4d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3da0017000 tx=0x7f3da0004750 comp rx=0 tx=0).stop 2023-07-18T03:38:44.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.415+0000 7f3db4dff700 1 -- 172.21.15.49:0/890313993 shutdown_connections 2023-07-18T03:38:44.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.415+0000 7f3db4dff700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3db01f5280 0x7f3db00c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.415+0000 7f3db4dff700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d9006ad30 0x7f3d9006d1e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.415+0000 7f3db4dff700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3db00ce480 0x7f3db01f1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.415+0000 7f3db4dff700 1 --2- 172.21.15.49:0/890313993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db01587d0 0x7f3db01f4d40 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3da0017000 tx=0x7f3da0004750 comp rx=0 tx=0).stop 2023-07-18T03:38:44.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.415+0000 7f3db4dff700 1 -- 172.21.15.49:0/890313993 >> 172.21.15.49:0/890313993 conn(0x7f3db00b05d0 msgr2=0x7f3db0071420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:44.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.415+0000 7f3db4dff700 1 -- 172.21.15.49:0/890313993 shutdown_connections 2023-07-18T03:38:44.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.416+0000 7f3db4dff700 1 -- 172.21.15.49:0/890313993 wait complete. 2023-07-18T03:38:44.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.416+0000 7fad84dff700 1 -- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad6006ad30 msgr2=0x7fad6006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.416+0000 7fad84dff700 1 --2- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad6006ad30 0x7fad6006d1e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fad78007fd0 tx=0x7fad78007980 comp rx=0 tx=0).stop 2023-07-18T03:38:44.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.416+0000 7fad84dff700 1 -- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8006a6d0 msgr2=0x7fad8006ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.416+0000 7fad84dff700 1 --2- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8006a6d0 0x7fad8006ab70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad70009b90 tx=0x7fad70004750 comp rx=0 tx=0).stop 2023-07-18T03:38:44.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.416+0000 7fad84dff700 1 -- 172.21.15.49:0/2750486489 shutdown_connections 2023-07-18T03:38:44.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.416+0000 7fad84dff700 1 --2- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8006a6d0 0x7fad8006ab70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.416+0000 7fad84dff700 1 --2- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800cf890 0x7fad8015ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.416+0000 7fad84dff700 1 --2- 172.21.15.49:0/2750486489 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad6006ad30 0x7fad6006d1e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.416+0000 7fad84dff700 1 --2- 172.21.15.49:0/2750486489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad800d29c0 0x7fad801531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.416+0000 7fad84dff700 1 -- 172.21.15.49:0/2750486489 >> 172.21.15.49:0/2750486489 conn(0x7fad8006c170 msgr2=0x7fad800b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:44.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.417+0000 7fad84dff700 1 -- 172.21.15.49:0/2750486489 shutdown_connections 2023-07-18T03:38:44.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.417+0000 7fad84dff700 1 -- 172.21.15.49:0/2750486489 wait complete. 2023-07-18T03:38:44.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.417+0000 7f3e75dff700 1 -- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3e5c06adb0 msgr2=0x7f3e5c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.417+0000 7f3e75dff700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3e5c06adb0 0x7f3e5c06d260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800f290 tx=0x7f3e68009040 comp rx=0 tx=0).stop 2023-07-18T03:38:44.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.417+0000 7f3e75dff700 1 -- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700d29c0 msgr2=0x7f3e700c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.417+0000 7f3e75dff700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700d29c0 0x7f3e700c8d10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000c00 tx=0x7f3e64004750 comp rx=0 tx=0).stop 2023-07-18T03:38:44.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.417+0000 7f3e75dff700 1 -- 172.21.15.49:0/358283703 shutdown_connections 2023-07-18T03:38:44.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.417+0000 7f3e75dff700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3e5c06adb0 0x7f3e5c06d260 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800f290 tx=0x7f3e68009040 comp rx=0 tx=0).stop 2023-07-18T03:38:44.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.417+0000 7f3e75dff700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e7015f230 0x7f3e7006f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.417+0000 7f3e75dff700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e700cf890 0x7f3e700ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.417+0000 7f3e75dff700 1 --2- 172.21.15.49:0/358283703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e700d29c0 0x7f3e700c8d10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.417+0000 7f3e75dff700 1 -- 172.21.15.49:0/358283703 >> 172.21.15.49:0/358283703 conn(0x7f3e7006c170 msgr2=0x7f3e700b6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:44.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.3 2023-07-18T03:38:44.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.418+0000 7f3e75dff700 1 -- 172.21.15.49:0/358283703 shutdown_connections 2023-07-18T03:38:44.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.418+0000 7f3e75dff700 1 -- 172.21.15.49:0/358283703 wait complete. 2023-07-18T03:38:44.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509446 for osd.0 2023-07-18T03:38:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.441+0000 7f255afff700 1 -- 172.21.15.49:0/2014684130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f25340055a0 con 0x7f2544004850 2023-07-18T03:38:44.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.441+0000 7f25517fa700 1 -- 172.21.15.49:0/2014684130 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2548017020 con 0x7f2544004850 2023-07-18T03:38:44.436 INFO:teuthology.orchestra.run.smithi049.stdout:64424509445 2023-07-18T03:38:44.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.444+0000 7f255afff700 1 -- 172.21.15.49:0/2014684130 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f253806adb0 msgr2=0x7f253806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.444+0000 7f255afff700 1 --2- 172.21.15.49:0/2014684130 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f253806adb0 0x7f253806d260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2540014040 tx=0x7f2540005040 comp rx=0 tx=0).stop 2023-07-18T03:38:44.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.444+0000 7f255afff700 1 -- 172.21.15.49:0/2014684130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544004850 msgr2=0x7f2554163800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:44.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.444+0000 7f255afff700 1 --2- 172.21.15.49:0/2014684130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544004850 0x7f2554163800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f254800cc80 tx=0x7f254800a340 comp rx=0 tx=0).stop 2023-07-18T03:38:44.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.444+0000 7f255afff700 1 -- 172.21.15.49:0/2014684130 shutdown_connections 2023-07-18T03:38:44.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.444+0000 7f255afff700 1 --2- 172.21.15.49:0/2014684130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f255406b5e0 0x7f255406ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.444+0000 7f255afff700 1 --2- 172.21.15.49:0/2014684130 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f253806adb0 0x7f253806d260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.444+0000 7f255afff700 1 --2- 172.21.15.49:0/2014684130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2544004850 0x7f2554163800 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.444+0000 7f255afff700 1 --2- 172.21.15.49:0/2014684130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2554161190 0x7f255408a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:44.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.444+0000 7f255afff700 1 -- 172.21.15.49:0/2014684130 >> 172.21.15.49:0/2014684130 conn(0x7f25540b05d0 msgr2=0x7f25540af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:44.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.444+0000 7f255afff700 1 -- 172.21.15.49:0/2014684130 shutdown_connections 2023-07-18T03:38:44.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:44.444+0000 7f255afff700 1 -- 172.21.15.49:0/2014684130 wait complete. 2023-07-18T03:38:44.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.1 2023-07-18T03:38:44.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.7 2023-07-18T03:38:44.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.5 2023-07-18T03:38:44.458 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.6 2023-07-18T03:38:45.399 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-18T03:38:45.411 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-18T03:38:45.421 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-18T03:38:45.431 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-18T03:38:45.444 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-18T03:38:45.445 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-18T03:38:45.446 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-18T03:38:45.459 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-18T03:38:45.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.737+0000 7fbb441ff700 1 Processor -- start 2023-07-18T03:38:45.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.737+0000 7fbb441ff700 1 -- start start 2023-07-18T03:38:45.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.739+0000 7fbb441ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbb3c160c30 0x7fbb3c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.739+0000 7fbb441ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3c161000 con 0x7fbb3c0cdbe0 2023-07-18T03:38:45.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.739+0000 7fbb441ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3c165400 con 0x7fbb3c164e20 2023-07-18T03:38:45.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.739+0000 7fbb441ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3c0cc810 con 0x7fbb3c160c30 2023-07-18T03:38:45.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.740+0000 7fbb41f9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbb3c160c30 0x7fbb3c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.740+0000 7fbb41f9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbb3c160c30 0x7fbb3c0cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40514/0 (socket says 172.21.15.49:40514) 2023-07-18T03:38:45.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.740+0000 7fbb41f9b700 1 -- 172.21.15.49:0/1468876483 learned_addr learned my addr 172.21.15.49:0/1468876483 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.741+0000 7fbb41f9b700 1 -- 172.21.15.49:0/1468876483 >> v1:172.21.15.107:6789/0 conn(0x7fbb3c164e20 legacy=0x7fbb3c0ca250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:38:45.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.741+0000 7fbb41f9b700 1 -- 172.21.15.49:0/1468876483 >> v1:172.21.15.49:6789/0 conn(0x7fbb3c0cdbe0 legacy=0x7fbb3c164710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:38:45.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.741+0000 7fbb41f9b700 1 -- 172.21.15.49:0/1468876483 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c0ccaf0 con 0x7fbb3c160c30 2023-07-18T03:38:45.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.741+0000 7fbb41f9b700 1 --2- 172.21.15.49:0/1468876483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c160c30 0x7fbb3c0cd2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009e50 tx=0x7fbb3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2826ed4386e3fc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.744+0000 7fbb40f99700 1 -- 172.21.15.49:0/1468876483 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400e070 con 0x7fbb3c160c30 2023-07-18T03:38:45.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.744+0000 7fbb40f99700 1 -- 172.21.15.49:0/1468876483 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb34003a20 con 0x7fbb3c160c30 2023-07-18T03:38:45.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.744+0000 7fbb40f99700 1 -- 172.21.15.49:0/1468876483 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34003dd0 con 0x7fbb3c160c30 2023-07-18T03:38:45.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.745+0000 7fbb441ff700 1 -- 172.21.15.49:0/1468876483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c160c30 msgr2=0x7fbb3c0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.745+0000 7fbb441ff700 1 --2- 172.21.15.49:0/1468876483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c160c30 0x7fbb3c0cd2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009e50 tx=0x7fbb3400b330 comp rx=0 tx=0).stop 2023-07-18T03:38:45.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.745+0000 7fbb441ff700 1 -- 172.21.15.49:0/1468876483 shutdown_connections 2023-07-18T03:38:45.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.745+0000 7fbb441ff700 1 --2- 172.21.15.49:0/1468876483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c160c30 0x7fbb3c0cd2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.745+0000 7fbb441ff700 1 -- 172.21.15.49:0/1468876483 >> 172.21.15.49:0/1468876483 conn(0x7fbb3c0b05d0 msgr2=0x7fbb3c0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:45.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.746+0000 7fbb441ff700 1 -- 172.21.15.49:0/1468876483 shutdown_connections 2023-07-18T03:38:45.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.746+0000 7fbb441ff700 1 -- 172.21.15.49:0/1468876483 wait complete. 2023-07-18T03:38:45.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.749+0000 7fbb441ff700 1 Processor -- start 2023-07-18T03:38:45.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.750+0000 7fbb441ff700 1 -- start start 2023-07-18T03:38:45.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.751+0000 7fbb441ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb3c0cdbe0 0x7fbb3c149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.752+0000 7fbb441ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c160c30 0x7fbb3c0c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.752+0000 7fbb441ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb3c164e20 0x7fbb3c151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.752+0000 7fbb441ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0b2350 con 0x7fbb3c160c30 2023-07-18T03:38:45.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.752+0000 7fbb441ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0c44c0 con 0x7fbb3c0cdbe0 2023-07-18T03:38:45.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.752+0000 7fbb441ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c1607c0 con 0x7fbb3c164e20 2023-07-18T03:38:45.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.753+0000 7fbb4179a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c160c30 0x7fbb3c0c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.753+0000 7fbb4179a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c160c30 0x7fbb3c0c3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40516/0 (socket says 172.21.15.49:40516) 2023-07-18T03:38:45.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.753+0000 7fbb4179a700 1 -- 172.21.15.49:0/4107459748 learned_addr learned my addr 172.21.15.49:0/4107459748 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.754+0000 7fbb4179a700 1 -- 172.21.15.49:0/4107459748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb3c164e20 msgr2=0x7fbb3c151110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:38:45.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.754+0000 7fbb4179a700 1 --2- 172.21.15.49:0/4107459748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb3c164e20 0x7fbb3c151110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.754+0000 7fbb4179a700 1 -- 172.21.15.49:0/4107459748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb3c0cdbe0 msgr2=0x7fbb3c149bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:38:45.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.754+0000 7fbb4179a700 1 --2- 172.21.15.49:0/4107459748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb3c0cdbe0 0x7fbb3c149bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.754+0000 7fbb4179a700 1 -- 172.21.15.49:0/4107459748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb34009b30 con 0x7fbb3c160c30 2023-07-18T03:38:45.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.755+0000 7fbb4179a700 1 --2- 172.21.15.49:0/4107459748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c160c30 0x7fbb3c0c3f80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00b480 tx=0x7fbb2c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.760+0000 7fbb327fc700 1 -- 172.21.15.49:0/4107459748 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c011830 con 0x7fbb3c160c30 2023-07-18T03:38:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.760+0000 7fbb327fc700 1 -- 172.21.15.49:0/4107459748 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c0119b0 con 0x7fbb3c160c30 2023-07-18T03:38:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.760+0000 7fbb327fc700 1 -- 172.21.15.49:0/4107459748 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c011d60 con 0x7fbb3c160c30 2023-07-18T03:38:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.760+0000 7fbb441ff700 1 -- 172.21.15.49:0/4107459748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3c0c4740 con 0x7fbb3c160c30 2023-07-18T03:38:45.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.760+0000 7fbb441ff700 1 -- 172.21.15.49:0/4107459748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c0b1d00 con 0x7fbb3c160c30 2023-07-18T03:38:45.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.765+0000 7fbb327fc700 1 -- 172.21.15.49:0/4107459748 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbb2c01a400 con 0x7fbb3c160c30 2023-07-18T03:38:45.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.767+0000 7fbb327fc700 1 --2- 172.21.15.49:0/4107459748 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb2006adb0 0x7fbb2006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.767+0000 7fbb327fc700 1 -- 172.21.15.49:0/4107459748 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbb2c089450 con 0x7fbb3c160c30 2023-07-18T03:38:45.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.768+0000 7fbb41f9b700 1 --2- 172.21.15.49:0/4107459748 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb2006adb0 0x7fbb2006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.769+0000 7fbb41f9b700 1 --2- 172.21.15.49:0/4107459748 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb2006adb0 0x7fbb2006d260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb34000f80 tx=0x7fbb34009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.770+0000 7fbb441ff700 1 -- 172.21.15.49:0/4107459748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb240052f0 con 0x7fbb3c160c30 2023-07-18T03:38:45.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.776+0000 7fbb327fc700 1 -- 172.21.15.49:0/4107459748 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbb2c054570 con 0x7fbb3c160c30 2023-07-18T03:38:45.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.810+0000 7fe5921ff700 1 Processor -- start 2023-07-18T03:38:45.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.810+0000 7fe5921ff700 1 -- start start 2023-07-18T03:38:45.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.812+0000 7fe5921ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe58c0cdf40 0x7fe58c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.812+0000 7fe5921ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe58c0caff0 0x7fe58c0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.812+0000 7fe5921ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58c0cab60 con 0x7fe58c1587d0 2023-07-18T03:38:45.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.812+0000 7fe5921ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58c0ce8f0 con 0x7fe58c0cdf40 2023-07-18T03:38:45.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.812+0000 7fe5921ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58c06bcc0 con 0x7fe58c0caff0 2023-07-18T03:38:45.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.813+0000 7fe58bfff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe58c0caff0 0x7fe58c0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.814+0000 7fe58bfff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe58c0caff0 0x7fe58c0cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45328/0 (socket says 172.21.15.49:45328) 2023-07-18T03:38:45.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.814+0000 7fe58bfff700 1 -- 172.21.15.49:0/2521349878 learned_addr learned my addr 172.21.15.49:0/2521349878 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.814+0000 7fe58affd700 1 --2- 172.21.15.49:0/2521349878 >> v2:172.21.15.49:3300/0 conn(0x7fe58c0cdf40 0x7fe58c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.814+0000 7fe58bfff700 1 -- 172.21.15.49:0/2521349878 >> v2:172.21.15.49:3300/0 conn(0x7fe58c0cdf40 msgr2=0x7fe58c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:45.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.814+0000 7fe58bfff700 1 --2- 172.21.15.49:0/2521349878 >> v2:172.21.15.49:3300/0 conn(0x7fe58c0cdf40 0x7fe58c0ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.814+0000 7fe58bfff700 1 -- 172.21.15.49:0/2521349878 >> v1:172.21.15.49:6789/0 conn(0x7fe58c1587d0 legacy=0x7fe58c06b5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:38:45.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.814+0000 7fe58bfff700 1 -- 172.21.15.49:0/2521349878 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c158ba0 con 0x7fe58c0caff0 2023-07-18T03:38:45.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.815+0000 7fe58bfff700 1 --2- 172.21.15.49:0/2521349878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c0caff0 0x7fe58c0cb490 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe58c06f7c0 tx=0x7fe57c00d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a42a5f722319a289 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.815+0000 7fe58a7fc700 1 -- 172.21.15.49:0/2521349878 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c010070 con 0x7fe58c0caff0 2023-07-18T03:38:45.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.815+0000 7fe58a7fc700 1 -- 172.21.15.49:0/2521349878 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c00d990 con 0x7fe58c0caff0 2023-07-18T03:38:45.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.815+0000 7fe58a7fc700 1 -- 172.21.15.49:0/2521349878 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c00dd20 con 0x7fe58c0caff0 2023-07-18T03:38:45.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.816+0000 7fe5921ff700 1 -- 172.21.15.49:0/2521349878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c0caff0 msgr2=0x7fe58c0cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.816+0000 7fe5921ff700 1 --2- 172.21.15.49:0/2521349878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c0caff0 0x7fe58c0cb490 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe58c06f7c0 tx=0x7fe57c00d330 comp rx=0 tx=0).stop 2023-07-18T03:38:45.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.817+0000 7fd9df3ff700 1 Processor -- start 2023-07-18T03:38:45.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.817+0000 7fe5921ff700 1 -- 172.21.15.49:0/2521349878 shutdown_connections 2023-07-18T03:38:45.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.817+0000 7fe5921ff700 1 --2- 172.21.15.49:0/2521349878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c0caff0 0x7fe58c0cb490 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.817+0000 7fe5921ff700 1 --2- 172.21.15.49:0/2521349878 >> v2:172.21.15.49:3300/0 conn(0x7fe58c0cdf40 0x7fe58c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.817+0000 7fe5921ff700 1 -- 172.21.15.49:0/2521349878 >> 172.21.15.49:0/2521349878 conn(0x7fe58c0b05d0 msgr2=0x7fe58c0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:45.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.817+0000 7fe5921ff700 1 -- 172.21.15.49:0/2521349878 shutdown_connections 2023-07-18T03:38:45.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.817+0000 7fd9df3ff700 1 -- start start 2023-07-18T03:38:45.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.819+0000 7fd9df3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd9d80cdf40 0x7fd9d80ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.819+0000 7fd9df3ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d80cab60 con 0x7fd9d81587d0 2023-07-18T03:38:45.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.819+0000 7fd9df3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d806bcc0 con 0x7fd9d80caff0 2023-07-18T03:38:45.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.819+0000 7fd9df3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d80ce8f0 con 0x7fd9d80cdf40 2023-07-18T03:38:45.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.817+0000 7fe5921ff700 1 -- 172.21.15.49:0/2521349878 wait complete. 2023-07-18T03:38:45.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.820+0000 7fe5921ff700 1 Processor -- start 2023-07-18T03:38:45.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.820+0000 7fd9dd99c700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fd9d80caff0 0x7fd9d80cb490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:53516/0 (socket says 172.21.15.49:53516) 2023-07-18T03:38:45.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.820+0000 7fd9dd99c700 1 -- 172.21.15.49:0/3108698878 learned_addr learned my addr 172.21.15.49:0/3108698878 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:45.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.820+0000 7fd9dc99a700 1 --2- 172.21.15.49:0/3108698878 >> v2:172.21.15.107:3300/0 conn(0x7fd9d80cdf40 0x7fd9d80ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427127097 0 0) 0x7fd9d80cab60 con 0x7fd9d81587d0 2023-07-18T03:38:45.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9c40034d0 con 0x7fd9d81587d0 2023-07-18T03:38:45.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4001343241 0 0) 0x7fd9c40034d0 con 0x7fd9d81587d0 2023-07-18T03:38:45.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 >> v2:172.21.15.107:3300/0 conn(0x7fd9d80cdf40 msgr2=0x7fd9d80ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:45.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fd9cffff700 1 --2- 172.21.15.49:0/3108698878 >> v2:172.21.15.107:3300/0 conn(0x7fd9d80cdf40 0x7fd9d80ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 >> v1:172.21.15.107:6789/0 conn(0x7fd9d80caff0 legacy=0x7fd9d80cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d8158ba0 con 0x7fd9d81587d0 2023-07-18T03:38:45.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7fd9d4001670 con 0x7fd9d81587d0 2023-07-18T03:38:45.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 >> v1:172.21.15.49:6789/0 conn(0x7fd9d81587d0 legacy=0x7fd9d806b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fd9cffff700 1 --2- 172.21.15.49:0/3108698878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9c4003af0 0x7fd9c4003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fd9cffff700 1 --2- 172.21.15.49:0/3108698878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c4008930 0x7fd9c400adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.823+0000 7fd9cffff700 1 --2- 172.21.15.49:0/3108698878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9c400b5b0 0x7fd9c400da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.823+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80ce8f0 con 0x7fd9c4008930 2023-07-18T03:38:45.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.823+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c400e340 con 0x7fd9c4003af0 2023-07-18T03:38:45.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.823+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c400e500 con 0x7fd9c400b5b0 2023-07-18T03:38:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.821+0000 7fe5921ff700 1 -- start start 2023-07-18T03:38:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fe5921ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c0caff0 0x7fe58c0c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fe5921ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe58c0cdf40 0x7fe58c15b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fe5921ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe58c1587d0 0x7fe58c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.822+0000 7fe5921ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c15ae20 con 0x7fe58c0cdf40 2023-07-18T03:38:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.823+0000 7fe5921ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c15a4f0 con 0x7fe58c0caff0 2023-07-18T03:38:45.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.823+0000 7fe5921ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c15a7d0 con 0x7fe58c1587d0 2023-07-18T03:38:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.823+0000 7fe58b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c0caff0 0x7fe58c0c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.823+0000 7fd9dd19b700 1 --2- 172.21.15.49:0/3108698878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9c4003af0 0x7fd9c4003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.823+0000 7fe58b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c0caff0 0x7fe58c0c5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45340/0 (socket says 172.21.15.49:45340) 2023-07-18T03:38:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.823+0000 7fe58b7fe700 1 -- 172.21.15.49:0/3957624603 learned_addr learned my addr 172.21.15.49:0/3957624603 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fe58bfff700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe58c1587d0 0x7fe58c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fe58affd700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe58c0cdf40 0x7fe58c15b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fd9dc99a700 1 --2- 172.21.15.49:0/3108698878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c4008930 0x7fd9c400adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fe1cb3ff700 1 Processor -- start 2023-07-18T03:38:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fd9dd19b700 1 -- 172.21.15.49:0/3108698878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9c400b5b0 msgr2=0x7fd9c400da60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:38:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fd9dd19b700 1 --2- 172.21.15.49:0/3108698878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9c400b5b0 0x7fd9c400da60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fd9dd19b700 1 -- 172.21.15.49:0/3108698878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c4008930 msgr2=0x7fd9c400adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:45.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fd9dd19b700 1 --2- 172.21.15.49:0/3108698878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c4008930 0x7fd9c400adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fd9dd19b700 1 -- 172.21.15.49:0/3108698878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d8158ba0 con 0x7fd9c4003af0 2023-07-18T03:38:45.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.825+0000 7fd9dd99c700 1 -- 172.21.15.49:0/3108698878 reap_dead start 2023-07-18T03:38:45.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.825+0000 7fd9dd19b700 1 --2- 172.21.15.49:0/3108698878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9c4003af0 0x7fd9c4003f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400c160 tx=0x7fd9c4003780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ba157f74cdd2b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fe58bfff700 1 -- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c0caff0 msgr2=0x7fe58c0c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fe58bfff700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c0caff0 0x7fe58c0c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fe58bfff700 1 -- 172.21.15.49:0/3957624603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe58c0cdf40 msgr2=0x7fe58c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fe58bfff700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe58c0cdf40 0x7fe58c15b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fe58bfff700 1 -- 172.21.15.49:0/3957624603 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c0085d0 con 0x7fe58c1587d0 2023-07-18T03:38:45.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.825+0000 7fe58bfff700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe58c1587d0 0x7fe58c15b7f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe57c018000 tx=0x7fe57c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.825+0000 7fe573fff700 1 -- 172.21.15.49:0/3957624603 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c004440 con 0x7fe58c1587d0 2023-07-18T03:38:45.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.825+0000 7fe573fff700 1 -- 172.21.15.49:0/3957624603 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c0045a0 con 0x7fe58c1587d0 2023-07-18T03:38:45.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.825+0000 7fe573fff700 1 -- 172.21.15.49:0/3957624603 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0038e0 con 0x7fe58c1587d0 2023-07-18T03:38:45.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.824+0000 7fe1cb3ff700 1 -- start start 2023-07-18T03:38:45.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.827+0000 7fe1cb3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1c41587d0 0x7fe1c406b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.827+0000 7fe1cb3ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe1c406baf0 0x7fe1c40cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.827+0000 7fe1cb3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe1c40ce480 0x7fe1c40cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.827+0000 7fe1cb3ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c40ce8d0 con 0x7fe1c406baf0 2023-07-18T03:38:45.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.827+0000 7fe1cb3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c40cab60 con 0x7fe1c40ce480 2023-07-18T03:38:45.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.827+0000 7fe1cb3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c4158ba0 con 0x7fe1c41587d0 2023-07-18T03:38:45.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.827+0000 7fe5921ff700 1 -- 172.21.15.49:0/3957624603 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58c15aab0 con 0x7fe58c1587d0 2023-07-18T03:38:45.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.827+0000 7fe5921ff700 1 -- 172.21.15.49:0/3957624603 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58c0c60d0 con 0x7fe58c1587d0 2023-07-18T03:38:45.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.827+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400cd00 con 0x7fd9c4003af0 2023-07-18T03:38:45.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.827+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d400ce60 con 0x7fd9c4003af0 2023-07-18T03:38:45.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.827+0000 7fd9cffff700 1 -- 172.21.15.49:0/3108698878 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4012640 con 0x7fd9c4003af0 2023-07-18T03:38:45.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.828+0000 7fd9df3ff700 1 -- 172.21.15.49:0/3108698878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9c4003af0 msgr2=0x7fd9c4003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.828+0000 7fd9df3ff700 1 --2- 172.21.15.49:0/3108698878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9c4003af0 0x7fd9c4003f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400c160 tx=0x7fd9c4003780 comp rx=0 tx=0).stop 2023-07-18T03:38:45.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.828+0000 7fe1c899a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe1c406baf0 0x7fe1c40cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.828+0000 7fe1c899a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe1c406baf0 0x7fe1c40cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40550/0 (socket says 172.21.15.49:40550) 2023-07-18T03:38:45.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.828+0000 7fe1c899a700 1 -- 172.21.15.49:0/1350755439 learned_addr learned my addr 172.21.15.49:0/1350755439 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.828+0000 7fe1c899a700 1 -- 172.21.15.49:0/1350755439 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1c41587d0 msgr2=0x7fe1c406b5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:38:45.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.828+0000 7fe1c899a700 1 --2- 172.21.15.49:0/1350755439 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1c41587d0 0x7fe1c406b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.828+0000 7fe1c899a700 1 -- 172.21.15.49:0/1350755439 >> v2:172.21.15.107:3300/0 conn(0x7fe1c40ce480 msgr2=0x7fe1c40cc690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:38:45.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.828+0000 7fe1c899a700 1 --2- 172.21.15.49:0/1350755439 >> v2:172.21.15.107:3300/0 conn(0x7fe1c40ce480 0x7fe1c40cc690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.828+0000 7fe1c899a700 1 -- 172.21.15.49:0/1350755439 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c406bf70 con 0x7fe1c406baf0 2023-07-18T03:38:45.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.829+0000 7fd9df3ff700 1 -- 172.21.15.49:0/3108698878 shutdown_connections 2023-07-18T03:38:45.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.829+0000 7fd9df3ff700 1 --2- 172.21.15.49:0/3108698878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9c4003af0 0x7fd9c4003f00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.829+0000 7fd9df3ff700 1 -- 172.21.15.49:0/3108698878 >> 172.21.15.49:0/3108698878 conn(0x7fd9d80b05d0 msgr2=0x7fd9d80b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:45.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.829+0000 7fe1c899a700 1 --2- 172.21.15.49:0/1350755439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1c406baf0 0x7fe1c40cdf40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe1c415d380 tx=0x7fe1c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e8f9f2e3431489 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.830+0000 7fe1bbfff700 1 -- 172.21.15.49:0/1350755439 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000b860 con 0x7fe1c406baf0 2023-07-18T03:38:45.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.830+0000 7fe1bbfff700 1 -- 172.21.15.49:0/1350755439 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000b9c0 con 0x7fe1c406baf0 2023-07-18T03:38:45.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.830+0000 7fe1bbfff700 1 -- 172.21.15.49:0/1350755439 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000bd50 con 0x7fe1c406baf0 2023-07-18T03:38:45.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.830+0000 7fe5921ff700 1 -- 172.21.15.49:0/3957624603 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58c15ac40 con 0x7fe58c1587d0 2023-07-18T03:38:45.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.831+0000 7fd9df3ff700 1 -- 172.21.15.49:0/3108698878 shutdown_connections 2023-07-18T03:38:45.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.831+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/1350755439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1c406baf0 msgr2=0x7fe1c40cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.831+0000 7fe1cb3ff700 1 --2- 172.21.15.49:0/1350755439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1c406baf0 0x7fe1c40cdf40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe1c415d380 tx=0x7fe1c0009c00 comp rx=0 tx=0).stop 2023-07-18T03:38:45.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.831+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/1350755439 shutdown_connections 2023-07-18T03:38:45.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.831+0000 7fe1cb3ff700 1 --2- 172.21.15.49:0/1350755439 >> v2:172.21.15.107:3300/0 conn(0x7fe1c40ce480 0x7fe1c40cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.831+0000 7fe1cb3ff700 1 --2- 172.21.15.49:0/1350755439 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1c41587d0 0x7fe1c406b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.831+0000 7fe1cb3ff700 1 --2- 172.21.15.49:0/1350755439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1c406baf0 0x7fe1c40cdf40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.831+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/1350755439 >> 172.21.15.49:0/1350755439 conn(0x7fe1c40b05d0 msgr2=0x7fe1c40b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:45.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.831+0000 7fe573fff700 1 -- 172.21.15.49:0/3957624603 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe57c003a40 con 0x7fe58c1587d0 2023-07-18T03:38:45.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.832+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/1350755439 shutdown_connections 2023-07-18T03:38:45.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.833+0000 7fe573fff700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe57806ade0 0x7fe57806d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.833+0000 7fe573fff700 1 -- 172.21.15.49:0/3957624603 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe57c089db0 con 0x7fe58c1587d0 2023-07-18T03:38:45.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.832+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/1350755439 wait complete. 2023-07-18T03:38:45.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.834+0000 7fe58b7fe700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe57806ade0 0x7fe57806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.831+0000 7fd9df3ff700 1 -- 172.21.15.49:0/3108698878 wait complete. 2023-07-18T03:38:45.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.834+0000 7fd9df3ff700 1 Processor -- start 2023-07-18T03:38:45.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.834+0000 7fd9df3ff700 1 -- start start 2023-07-18T03:38:45.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.835+0000 7fe1cb3ff700 1 Processor -- start 2023-07-18T03:38:45.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.835+0000 7fe58b7fe700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe57806ade0 0x7fe57806d290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe58400e200 tx=0x7fe58400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.836+0000 7fd9df3ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c4003af0 0x7fd9d8159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.836+0000 7fd9df3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9d80c4c20 0x7fd9d81f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.836+0000 7fd9df3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9d80c5b90 0x7fd9d8070860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.836+0000 7fd9df3ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80b39e0 con 0x7fd9c4003af0 2023-07-18T03:38:45.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.836+0000 7fd9df3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80b2df0 con 0x7fd9d80c4c20 2023-07-18T03:38:45.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.836+0000 7fd9df3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81f6420 con 0x7fd9d80c5b90 2023-07-18T03:38:45.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fd9dc99a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9d80c4c20 0x7fd9d81f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fd9dc99a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9d80c4c20 0x7fd9d81f5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45372/0 (socket says 172.21.15.49:45372) 2023-07-18T03:38:45.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fd9dc99a700 1 -- 172.21.15.49:0/2548765708 learned_addr learned my addr 172.21.15.49:0/2548765708 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fd9dd19b700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c4003af0 0x7fd9d8159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.835+0000 7fe1cb3ff700 1 -- start start 2023-07-18T03:38:45.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.836+0000 7fe1cb3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1c40ce480 0x7fe1c415b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fe1cb3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1c41587d0 0x7fe1c40bf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fe1cb3ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1c4159ee0 0x7fe1c40cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fe1cb3ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40c7bb0 con 0x7fe1c4159ee0 2023-07-18T03:38:45.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fe1cb3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40c6fa0 con 0x7fe1c40ce480 2023-07-18T03:38:45.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fe1cb3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40c7280 con 0x7fe1c41587d0 2023-07-18T03:38:45.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fd9dd99c700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9d80c5b90 0x7fd9d8070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fe1c899a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1c41587d0 0x7fe1c40bf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fe1c999c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1c4159ee0 0x7fe1c40cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fe1c999c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1c4159ee0 0x7fe1c40cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40572/0 (socket says 172.21.15.49:40572) 2023-07-18T03:38:45.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fe1c999c700 1 -- 172.21.15.49:0/4014547519 learned_addr learned my addr 172.21.15.49:0/4014547519 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fd9dc99a700 1 -- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9d80c5b90 msgr2=0x7fd9d8070860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fd9dc99a700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9d80c5b90 0x7fd9d8070860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fd9dc99a700 1 -- 172.21.15.49:0/2548765708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c4003af0 msgr2=0x7fd9d8159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fd9dc99a700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c4003af0 0x7fd9d8159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.837+0000 7fd9dc99a700 1 -- 172.21.15.49:0/2548765708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d80b3040 con 0x7fd9d80c4c20 2023-07-18T03:38:45.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.838+0000 7fe573fff700 1 -- 172.21.15.49:0/3957624603 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe57c054dd0 con 0x7fe58c1587d0 2023-07-18T03:38:45.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.838+0000 7fe1c919b700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1c40ce480 0x7fe1c415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.839+0000 7ff3cfdff700 1 Processor -- start 2023-07-18T03:38:45.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.838+0000 7fe1c899a700 1 -- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1c40ce480 msgr2=0x7fe1c415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.839+0000 7fe1c899a700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1c40ce480 0x7fe1c415b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.839+0000 7fe1c899a700 1 -- 172.21.15.49:0/4014547519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1c4159ee0 msgr2=0x7fe1c40cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.839+0000 7fe1c899a700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1c4159ee0 0x7fe1c40cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.839+0000 7fe1c899a700 1 -- 172.21.15.49:0/4014547519 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c0009550 con 0x7fe1c41587d0 2023-07-18T03:38:45.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.839+0000 7fe1c999c700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1c4159ee0 0x7fe1c40cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:45.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.839+0000 7ff3cfdff700 1 -- start start 2023-07-18T03:38:45.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.841+0000 7ff3cfdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff3c80b10b0 0x7ff3c8158560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.841+0000 7ff3cfdff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff3c80ce650 0x7ff3c80caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.841+0000 7ff3cfdff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c80cab60 con 0x7ff3c806b5b0 2023-07-18T03:38:45.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.841+0000 7ff3cfdff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c806c250 con 0x7ff3c80ce650 2023-07-18T03:38:45.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.841+0000 7ff3cfdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c806c530 con 0x7ff3c80b10b0 2023-07-18T03:38:45.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.842+0000 7ff3cdb9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff3c80b10b0 0x7ff3c8158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.842+0000 7ff3cdb9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff3c80b10b0 0x7ff3c8158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45384/0 (socket says 172.21.15.49:45384) 2023-07-18T03:38:45.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.842+0000 7ff3cdb9b700 1 -- 172.21.15.49:0/3956001933 learned_addr learned my addr 172.21.15.49:0/3956001933 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.842+0000 7ff3cdb9b700 1 -- 172.21.15.49:0/3956001933 >> v2:172.21.15.49:3300/0 conn(0x7ff3c80ce650 msgr2=0x7ff3c80caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:45.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.842+0000 7ff3cdb9b700 1 --2- 172.21.15.49:0/3956001933 >> v2:172.21.15.49:3300/0 conn(0x7ff3c80ce650 0x7ff3c80caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.842+0000 7ff3cdb9b700 1 -- 172.21.15.49:0/3956001933 >> v1:172.21.15.49:6789/0 conn(0x7ff3c806b5b0 legacy=0x7ff3c80cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:45.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.842+0000 7ff3cdb9b700 1 -- 172.21.15.49:0/3956001933 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c80ca7f0 con 0x7ff3c80b10b0 2023-07-18T03:38:45.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.840+0000 7fe1c899a700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1c41587d0 0x7fe1c40bf9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0016000 tx=0x7fe1c0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.840+0000 7fe1b9ffb700 1 -- 172.21.15.49:0/4014547519 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0004510 con 0x7fe1c41587d0 2023-07-18T03:38:45.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.840+0000 7fe1b9ffb700 1 -- 172.21.15.49:0/4014547519 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c001f070 con 0x7fe1c41587d0 2023-07-18T03:38:45.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.840+0000 7fe1b9ffb700 1 -- 172.21.15.49:0/4014547519 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00038c0 con 0x7fe1c41587d0 2023-07-18T03:38:45.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.843+0000 7ff3cdb9b700 1 --2- 172.21.15.49:0/3956001933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3c80b10b0 0x7ff3c8158560 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff3c806d720 tx=0x7ff3c4009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2b56a1cddc0fc6b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.843+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/4014547519 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c40c7560 con 0x7fe1c41587d0 2023-07-18T03:38:45.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.843+0000 7ff3ccb99700 1 -- 172.21.15.49:0/3956001933 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4004230 con 0x7ff3c80b10b0 2023-07-18T03:38:45.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.843+0000 7ff3ccb99700 1 -- 172.21.15.49:0/3956001933 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c4004390 con 0x7ff3c80b10b0 2023-07-18T03:38:45.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.843+0000 7ff3ccb99700 1 -- 172.21.15.49:0/3956001933 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c40036a0 con 0x7ff3c80b10b0 2023-07-18T03:38:45.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.844+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/4014547519 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c40c2070 con 0x7fe1c41587d0 2023-07-18T03:38:45.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.845+0000 7ff3cfdff700 1 -- 172.21.15.49:0/3956001933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3c80b10b0 msgr2=0x7ff3c8158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.845+0000 7ff3cfdff700 1 --2- 172.21.15.49:0/3956001933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3c80b10b0 0x7ff3c8158560 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff3c806d720 tx=0x7ff3c4009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.845+0000 7ff3cfdff700 1 -- 172.21.15.49:0/3956001933 shutdown_connections 2023-07-18T03:38:45.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.845+0000 7ff3cfdff700 1 --2- 172.21.15.49:0/3956001933 >> v2:172.21.15.49:3300/0 conn(0x7ff3c80ce650 0x7ff3c80caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.845+0000 7ff3cfdff700 1 --2- 172.21.15.49:0/3956001933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3c80b10b0 0x7ff3c8158560 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.845+0000 7ff3cfdff700 1 -- 172.21.15.49:0/3956001933 >> 172.21.15.49:0/3956001933 conn(0x7ff3c80b05d0 msgr2=0x7ff3c80b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:45.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.846+0000 7ff3cfdff700 1 -- 172.21.15.49:0/3956001933 shutdown_connections 2023-07-18T03:38:45.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.847+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/4014547519 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c4074350 con 0x7fe1c41587d0 2023-07-18T03:38:45.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.848+0000 7fe1b9ffb700 1 -- 172.21.15.49:0/4014547519 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe1c0003a20 con 0x7fe1c41587d0 2023-07-18T03:38:45.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.846+0000 7fd9dc99a700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9d80c4c20 0x7fd9d81f5ee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8010860 tx=0x7fd9c8010c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.848+0000 7fbf36fff700 1 Processor -- start 2023-07-18T03:38:45.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.846+0000 7ff3cfdff700 1 -- 172.21.15.49:0/3956001933 wait complete. 2023-07-18T03:38:45.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.849+0000 7ff3cfdff700 1 Processor -- start 2023-07-18T03:38:45.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.849+0000 7fe1b9ffb700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1a806acb0 0x7fe1a806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.849+0000 7fe1b9ffb700 1 -- 172.21.15.49:0/4014547519 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1c0021070 con 0x7fe1c41587d0 2023-07-18T03:38:45.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.849+0000 7fd9cdffb700 1 -- 172.21.15.49:0/2548765708 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c800e860 con 0x7fd9d80c4c20 2023-07-18T03:38:45.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.849+0000 7fd9cdffb700 1 -- 172.21.15.49:0/2548765708 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c800c050 con 0x7fd9d80c4c20 2023-07-18T03:38:45.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.849+0000 7fd9cdffb700 1 -- 172.21.15.49:0/2548765708 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c800ebc0 con 0x7fd9d80c4c20 2023-07-18T03:38:45.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.849+0000 7ff3cfdff700 1 -- start start 2023-07-18T03:38:45.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7ff3cfdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3c806b5b0 0x7ff3c8171670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7ff3cfdff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3c80b10b0 0x7ff3c8171bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7ff3cfdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3c80ce650 0x7ff3c816c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7ff3cfdff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c8159800 con 0x7ff3c80b10b0 2023-07-18T03:38:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7ff3cfdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80c5560 con 0x7ff3c806b5b0 2023-07-18T03:38:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7ff3cfdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c8158ed0 con 0x7ff3c80ce650 2023-07-18T03:38:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7ff3cdb9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3c806b5b0 0x7ff3c8171670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7ff3cdb9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3c806b5b0 0x7ff3c8171670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45396/0 (socket says 172.21.15.49:45396) 2023-07-18T03:38:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7ff3cdb9b700 1 -- 172.21.15.49:0/260651661 learned_addr learned my addr 172.21.15.49:0/260651661 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.850+0000 7fd9df3ff700 1 -- 172.21.15.49:0/2548765708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d80b3320 con 0x7fd9d80c4c20 2023-07-18T03:38:45.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.850+0000 7fd9df3ff700 1 -- 172.21.15.49:0/2548765708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d8159770 con 0x7fd9d80c4c20 2023-07-18T03:38:45.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.852+0000 7ff3ce39c700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3c80ce650 0x7ff3c816c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.852+0000 7ff3cdb9b700 1 -- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3c80ce650 msgr2=0x7ff3c816c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.852+0000 7ff3cdb9b700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3c80ce650 0x7ff3c816c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.852+0000 7ff3cdb9b700 1 -- 172.21.15.49:0/260651661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3c80b10b0 msgr2=0x7ff3c8171bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.852+0000 7ff3cdb9b700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3c80b10b0 0x7ff3c8171bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.852+0000 7ff3cdb9b700 1 -- 172.21.15.49:0/260651661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c4009550 con 0x7ff3c806b5b0 2023-07-18T03:38:45.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.852+0000 7fe1c919b700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1a806acb0 0x7fe1a806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.852+0000 7ff3cdb9b700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3c806b5b0 0x7ff3c8171670 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3c806c670 tx=0x7ff3c4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.853+0000 7fd9df3ff700 1 -- 172.21.15.49:0/2548765708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d8074350 con 0x7fd9d80c4c20 2023-07-18T03:38:45.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.854+0000 7fe1c919b700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1a806acb0 0x7fe1a806d160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40b4910 tx=0x7fe1b4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.854+0000 7fe1b9ffb700 1 -- 172.21.15.49:0/4014547519 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe1c0055c80 con 0x7fe1c41587d0 2023-07-18T03:38:45.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.853+0000 7ff3be7fc700 1 -- 172.21.15.49:0/260651661 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4011630 con 0x7ff3c806b5b0 2023-07-18T03:38:45.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.853+0000 7ff3be7fc700 1 -- 172.21.15.49:0/260651661 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c40117b0 con 0x7ff3c806b5b0 2023-07-18T03:38:45.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.853+0000 7ff3be7fc700 1 -- 172.21.15.49:0/260651661 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4011b60 con 0x7ff3c806b5b0 2023-07-18T03:38:45.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.854+0000 7ff3cfdff700 1 -- 172.21.15.49:0/260651661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c81591b0 con 0x7ff3c806b5b0 2023-07-18T03:38:45.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.854+0000 7ff3cfdff700 1 -- 172.21.15.49:0/260651661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c8147890 con 0x7ff3c806b5b0 2023-07-18T03:38:45.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.855+0000 7fd9cdffb700 1 -- 172.21.15.49:0/2548765708 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd9c800ed20 con 0x7fd9d80c4c20 2023-07-18T03:38:45.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.856+0000 7ff3cfdff700 1 -- 172.21.15.49:0/260651661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c8074350 con 0x7ff3c806b5b0 2023-07-18T03:38:45.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.856+0000 7fd9cdffb700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd9bc06adb0 0x7fd9bc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.857+0000 7fd9cdffb700 1 -- 172.21.15.49:0/2548765708 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd9c808a240 con 0x7fd9d80c4c20 2023-07-18T03:38:45.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.849+0000 7fbf36fff700 1 -- start start 2023-07-18T03:38:45.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7fbf36fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf300cd940 0x7fbf300cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7fbf36fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbf3006cd60 0x7fbf300cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7fbf36fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf300cd530 con 0x7fbf30160d50 2023-07-18T03:38:45.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7fbf36fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf301615d0 con 0x7fbf3006cd60 2023-07-18T03:38:45.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.851+0000 7fbf36fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf300ce250 con 0x7fbf300cd940 2023-07-18T03:38:45.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.852+0000 7fbf34d9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf300cd940 0x7fbf300cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.852+0000 7fbf2ffff700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fbf30160d50 0x7fbf30161a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:53526/0 (socket says 172.21.15.49:53526) 2023-07-18T03:38:45.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.852+0000 7fbf2ffff700 1 -- 172.21.15.49:0/2625491014 learned_addr learned my addr 172.21.15.49:0/2625491014 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:45.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.852+0000 7fbf3559c700 1 --2- 172.21.15.49:0/2625491014 >> v2:172.21.15.107:3300/0 conn(0x7fbf3006cd60 0x7fbf300cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.853+0000 7fbf34d9b700 1 -- 172.21.15.49:0/2625491014 >> v2:172.21.15.107:3300/0 conn(0x7fbf3006cd60 msgr2=0x7fbf300cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:45.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.853+0000 7fbf34d9b700 1 --2- 172.21.15.49:0/2625491014 >> v2:172.21.15.107:3300/0 conn(0x7fbf3006cd60 0x7fbf300cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.853+0000 7fbf34d9b700 1 -- 172.21.15.49:0/2625491014 >> v1:172.21.15.107:6789/0 conn(0x7fbf30160d50 legacy=0x7fbf30161a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.853+0000 7fbf34d9b700 1 -- 172.21.15.49:0/2625491014 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf300ccf70 con 0x7fbf300cd940 2023-07-18T03:38:45.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.853+0000 7fbf34d9b700 1 --2- 172.21.15.49:0/2625491014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf300cd940 0x7fbf300cdd10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009d90 tx=0x7fbf2800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ced797a81cf03338 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.854+0000 7fbf2f7fe700 1 -- 172.21.15.49:0/2625491014 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2800e070 con 0x7fbf300cd940 2023-07-18T03:38:45.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.855+0000 7fbf2f7fe700 1 -- 172.21.15.49:0/2625491014 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2800bbd0 con 0x7fbf300cd940 2023-07-18T03:38:45.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.855+0000 7fbf2f7fe700 1 -- 172.21.15.49:0/2625491014 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28012430 con 0x7fbf300cd940 2023-07-18T03:38:45.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.857+0000 7fbf36fff700 1 -- 172.21.15.49:0/2625491014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf300cd940 msgr2=0x7fbf300cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.857+0000 7fbf36fff700 1 --2- 172.21.15.49:0/2625491014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf300cd940 0x7fbf300cdd10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009d90 tx=0x7fbf2800b330 comp rx=0 tx=0).stop 2023-07-18T03:38:45.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.857+0000 7fbf36fff700 1 -- 172.21.15.49:0/2625491014 shutdown_connections 2023-07-18T03:38:45.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.857+0000 7fbf36fff700 1 --2- 172.21.15.49:0/2625491014 >> v2:172.21.15.107:3300/0 conn(0x7fbf3006cd60 0x7fbf300cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.857+0000 7fbf36fff700 1 --2- 172.21.15.49:0/2625491014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf300cd940 0x7fbf300cdd10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.857+0000 7fbf36fff700 1 -- 172.21.15.49:0/2625491014 >> 172.21.15.49:0/2625491014 conn(0x7fbf300b05d0 msgr2=0x7fbf300b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:45.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.858+0000 7fd9dd19b700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd9bc06adb0 0x7fd9bc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.858+0000 7fbf36fff700 1 -- 172.21.15.49:0/2625491014 shutdown_connections 2023-07-18T03:38:45.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.858+0000 7fbf36fff700 1 -- 172.21.15.49:0/2625491014 wait complete. 2023-07-18T03:38:45.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.858+0000 7fd9dd19b700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd9bc06adb0 0x7fd9bc06d260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80c8c50 tx=0x7fd9d4005040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.859+0000 7fd9cdffb700 1 -- 172.21.15.49:0/2548765708 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd9c8055360 con 0x7fd9d80c4c20 2023-07-18T03:38:45.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.860+0000 7fbf36fff700 1 Processor -- start 2023-07-18T03:38:45.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.862+0000 7fbf36fff700 1 -- start start 2023-07-18T03:38:45.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.863+0000 7fbf36fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf3006cd60 0x7fbf300b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.864+0000 7fbf36fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf300cd940 0x7fbf300c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.864+0000 7fbf36fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf30160d50 0x7fbf300c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.864+0000 7fbf36fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3014c150 con 0x7fbf3006cd60 2023-07-18T03:38:45.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.864+0000 7fbf36fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf300c54d0 con 0x7fbf300cd940 2023-07-18T03:38:45.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.864+0000 7fbf36fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf30164590 con 0x7fbf30160d50 2023-07-18T03:38:45.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.864+0000 7fbf2ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf300cd940 0x7fbf300c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.865+0000 7fbf2ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf300cd940 0x7fbf300c44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45414/0 (socket says 172.21.15.49:45414) 2023-07-18T03:38:45.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.865+0000 7fbf2ffff700 1 -- 172.21.15.49:0/471490046 learned_addr learned my addr 172.21.15.49:0/471490046 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.865+0000 7fbf3559c700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf30160d50 0x7fbf300c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.865+0000 7ff3be7fc700 1 -- 172.21.15.49:0/260651661 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff3c4022070 con 0x7ff3c806b5b0 2023-07-18T03:38:45.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.866+0000 7fbf2ffff700 1 -- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf30160d50 msgr2=0x7fbf300c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.866+0000 7fbf2ffff700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf30160d50 0x7fbf300c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.866+0000 7fbf2ffff700 1 -- 172.21.15.49:0/471490046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf3006cd60 msgr2=0x7fbf300b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.866+0000 7fbf2ffff700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf3006cd60 0x7fbf300b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.866+0000 7fbf2ffff700 1 -- 172.21.15.49:0/471490046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf28009a70 con 0x7fbf300cd940 2023-07-18T03:38:45.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.867+0000 7fbf2ffff700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf300cd940 0x7fbf300c44c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400c480 tx=0x7fbf2400c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.867+0000 7ff3be7fc700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff3ac06ace0 0x7ff3ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.867+0000 7ff3be7fc700 1 -- 172.21.15.49:0/260651661 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff3c408b360 con 0x7ff3c806b5b0 2023-07-18T03:38:45.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.867+0000 7ff3be7fc700 1 -- 172.21.15.49:0/260651661 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff3c408b7e0 con 0x7ff3c806b5b0 2023-07-18T03:38:45.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.867+0000 7fbf2d7fa700 1 -- 172.21.15.49:0/471490046 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24005d70 con 0x7fbf300cd940 2023-07-18T03:38:45.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.867+0000 7fbf2d7fa700 1 -- 172.21.15.49:0/471490046 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf240070d0 con 0x7fbf300cd940 2023-07-18T03:38:45.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.867+0000 7fbf2d7fa700 1 -- 172.21.15.49:0/471490046 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24007440 con 0x7fbf300cd940 2023-07-18T03:38:45.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.868+0000 7fbf36fff700 1 -- 172.21.15.49:0/471490046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf30164870 con 0x7fbf300cd940 2023-07-18T03:38:45.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.868+0000 7fbf36fff700 1 -- 172.21.15.49:0/471490046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf30164e30 con 0x7fbf300cd940 2023-07-18T03:38:45.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.868+0000 7ff3cd39a700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff3ac06ace0 0x7ff3ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.871+0000 7fbf2d7fa700 1 -- 172.21.15.49:0/471490046 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf240075a0 con 0x7fbf300cd940 2023-07-18T03:38:45.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.873+0000 7fbf2d7fa700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbf1c06adb0 0x7fbf1c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.873+0000 7fbf2d7fa700 1 -- 172.21.15.49:0/471490046 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf240887e0 con 0x7fbf300cd940 2023-07-18T03:38:45.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.873+0000 7fbf34d9b700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbf1c06adb0 0x7fbf1c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.874+0000 7ff3cd39a700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff3ac06ace0 0x7ff3ac06d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3c814a190 tx=0x7ff3b8007a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.874+0000 7fbf36fff700 1 -- 172.21.15.49:0/471490046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf300c9b90 con 0x7fbf300cd940 2023-07-18T03:38:45.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.876+0000 7fbf34d9b700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbf1c06adb0 0x7fbf1c06d260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf28000f80 tx=0x7fbf28010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.883+0000 7fbf2d7fa700 1 -- 172.21.15.49:0/471490046 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbf240539e0 con 0x7fbf300cd940 2023-07-18T03:38:45.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.895+0000 7ffa96bff700 1 Processor -- start 2023-07-18T03:38:45.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.896+0000 7ffa96bff700 1 -- start start 2023-07-18T03:38:45.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.898+0000 7ffa96bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffa900ce2f0 0x7ffa900cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.898+0000 7ffa96bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ffa9016cfa0 0x7ffa900ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.898+0000 7ffa96bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa900cd910 con 0x7ffa9016da60 2023-07-18T03:38:45.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.898+0000 7ffa96bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa900ccd00 con 0x7ffa900ce2f0 2023-07-18T03:38:45.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.898+0000 7ffa96bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa900ccfe0 con 0x7ffa9016cfa0 2023-07-18T03:38:45.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.899+0000 7ffa9519c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ffa9016cfa0 0x7ffa900ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.899+0000 7ffa9519c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ffa9016cfa0 0x7ffa900ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45422/0 (socket says 172.21.15.49:45422) 2023-07-18T03:38:45.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.899+0000 7ffa9519c700 1 -- 172.21.15.49:0/3161688343 learned_addr learned my addr 172.21.15.49:0/3161688343 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.903+0000 7ffa9519c700 1 -- 172.21.15.49:0/3161688343 >> v2:172.21.15.49:3300/0 conn(0x7ffa900ce2f0 msgr2=0x7ffa900cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:45.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.903+0000 7ffa9519c700 1 --2- 172.21.15.49:0/3161688343 >> v2:172.21.15.49:3300/0 conn(0x7ffa900ce2f0 0x7ffa900cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.903+0000 7ffa9519c700 1 -- 172.21.15.49:0/3161688343 >> v1:172.21.15.49:6789/0 conn(0x7ffa9016da60 legacy=0x7ffa900cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:45.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.903+0000 7ffa9519c700 1 -- 172.21.15.49:0/3161688343 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa900cd2c0 con 0x7ffa9016cfa0 2023-07-18T03:38:45.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.904+0000 7ffa9519c700 1 --2- 172.21.15.49:0/3161688343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9016cfa0 0x7ffa900ca590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffa900cdb60 tx=0x7ffa88008190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5d34ce16f6f0981 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.905+0000 7ffa8f7fe700 1 -- 172.21.15.49:0/3161688343 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88008d10 con 0x7ffa9016cfa0 2023-07-18T03:38:45.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.905+0000 7ffa8f7fe700 1 -- 172.21.15.49:0/3161688343 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa88008e70 con 0x7ffa9016cfa0 2023-07-18T03:38:45.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.905+0000 7ffa8f7fe700 1 -- 172.21.15.49:0/3161688343 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88003c20 con 0x7ffa9016cfa0 2023-07-18T03:38:45.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.906+0000 7ffa96bff700 1 -- 172.21.15.49:0/3161688343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9016cfa0 msgr2=0x7ffa900ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.906+0000 7ffa96bff700 1 --2- 172.21.15.49:0/3161688343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9016cfa0 0x7ffa900ca590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffa900cdb60 tx=0x7ffa88008190 comp rx=0 tx=0).stop 2023-07-18T03:38:45.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.907+0000 7ffa96bff700 1 -- 172.21.15.49:0/3161688343 shutdown_connections 2023-07-18T03:38:45.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.907+0000 7ffa96bff700 1 --2- 172.21.15.49:0/3161688343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9016cfa0 0x7ffa900ca590 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.907+0000 7ffa96bff700 1 --2- 172.21.15.49:0/3161688343 >> v2:172.21.15.49:3300/0 conn(0x7ffa900ce2f0 0x7ffa900cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.907+0000 7ffa96bff700 1 -- 172.21.15.49:0/3161688343 >> 172.21.15.49:0/3161688343 conn(0x7ffa900b05d0 msgr2=0x7ffa900b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:45.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.907+0000 7ffa96bff700 1 -- 172.21.15.49:0/3161688343 shutdown_connections 2023-07-18T03:38:45.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.907+0000 7ffa96bff700 1 -- 172.21.15.49:0/3161688343 wait complete. 2023-07-18T03:38:45.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.910+0000 7ffa96bff700 1 Processor -- start 2023-07-18T03:38:45.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.911+0000 7ffa96bff700 1 -- start start 2023-07-18T03:38:45.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.912+0000 7ffa96bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa900ce2f0 0x7ffa900b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.912+0000 7ffa96bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9016cfa0 0x7ffa900c1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.912+0000 7ffa96bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9016da60 0x7ffa90167810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.912+0000 7ffa96bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa900b48d0 con 0x7ffa9016cfa0 2023-07-18T03:38:45.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.912+0000 7ffa96bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa900b4100 con 0x7ffa900ce2f0 2023-07-18T03:38:45.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.912+0000 7ffa96bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa900b4310 con 0x7ffa9016da60 2023-07-18T03:38:45.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.913+0000 7ffa8ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9016cfa0 0x7ffa900c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.913+0000 7ffa8ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9016cfa0 0x7ffa900c1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40632/0 (socket says 172.21.15.49:40632) 2023-07-18T03:38:45.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.913+0000 7ffa8ffff700 1 -- 172.21.15.49:0/6679904 learned_addr learned my addr 172.21.15.49:0/6679904 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.914+0000 7ffa9499b700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa900ce2f0 0x7ffa900b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.915+0000 7ffa9499b700 1 -- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9016da60 msgr2=0x7ffa90167810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.915+0000 7ffa9499b700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9016da60 0x7ffa90167810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.915+0000 7ffa9499b700 1 -- 172.21.15.49:0/6679904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9016cfa0 msgr2=0x7ffa900c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.915+0000 7ffa9499b700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9016cfa0 0x7ffa900c1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.915+0000 7ffa9499b700 1 -- 172.21.15.49:0/6679904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa88007b10 con 0x7ffa900ce2f0 2023-07-18T03:38:45.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.916+0000 7ffa9499b700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa900ce2f0 0x7ffa900b1cb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000b440 tx=0x7ffa80009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.916+0000 7ffa8d7fa700 1 -- 172.21.15.49:0/6679904 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa80011830 con 0x7ffa900ce2f0 2023-07-18T03:38:45.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.916+0000 7ffa8d7fa700 1 -- 172.21.15.49:0/6679904 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa800119b0 con 0x7ffa900ce2f0 2023-07-18T03:38:45.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.916+0000 7ffa8d7fa700 1 -- 172.21.15.49:0/6679904 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa80011d60 con 0x7ffa900ce2f0 2023-07-18T03:38:45.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.916+0000 7ffa96bff700 1 -- 172.21.15.49:0/6679904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa900b45f0 con 0x7ffa900ce2f0 2023-07-18T03:38:45.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.916+0000 7ffa96bff700 1 -- 172.21.15.49:0/6679904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa900b2c10 con 0x7ffa900ce2f0 2023-07-18T03:38:45.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.916+0000 7f6cba3ff700 1 Processor -- start 2023-07-18T03:38:45.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.916+0000 7f6cba3ff700 1 -- start start 2023-07-18T03:38:45.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.918+0000 7f6cba3ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6cb416da60 0x7f6cb40cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.918+0000 7f6cba3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cb40ce2f0 0x7f6cb40ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.918+0000 7f6cba3ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb40cd910 con 0x7f6cb416cfa0 2023-07-18T03:38:45.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.918+0000 7f6cba3ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb40ccd00 con 0x7f6cb416da60 2023-07-18T03:38:45.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.918+0000 7f6cba3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40ccfe0 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.919+0000 7f6cb899c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cb40ce2f0 0x7f6cb40ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.919+0000 7f6cb37fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6cb416cfa0 0x7f6cb40cdbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41654/0 (socket says 172.21.15.49:41654) 2023-07-18T03:38:45.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.919+0000 7f6cb37fe700 1 -- 172.21.15.49:0/2860860424 learned_addr learned my addr 172.21.15.49:0/2860860424 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:45.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.919+0000 7f6cb3fff700 1 --2- 172.21.15.49:0/2860860424 >> v2:172.21.15.49:3300/0 conn(0x7f6cb416da60 0x7f6cb40cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.921+0000 7f6cb899c700 1 -- 172.21.15.49:0/2860860424 >> v2:172.21.15.49:3300/0 conn(0x7f6cb416da60 msgr2=0x7f6cb40cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:45.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.921+0000 7f6cb899c700 1 --2- 172.21.15.49:0/2860860424 >> v2:172.21.15.49:3300/0 conn(0x7f6cb416da60 0x7f6cb40cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.921+0000 7f6cb899c700 1 -- 172.21.15.49:0/2860860424 >> v1:172.21.15.49:6789/0 conn(0x7f6cb416cfa0 legacy=0x7f6cb40cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.921+0000 7f6cb899c700 1 -- 172.21.15.49:0/2860860424 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb40cd2c0 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.924+0000 7f6cb899c700 1 --2- 172.21.15.49:0/2860860424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cb40ce2f0 0x7f6cb40ca590 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009840 tx=0x7f6ca8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f40ca4a883ed9ead server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.923+0000 7ffa8d7fa700 1 -- 172.21.15.49:0/6679904 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffa8001a400 con 0x7ffa900ce2f0 2023-07-18T03:38:45.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.925+0000 7ffa8d7fa700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa7c06adb0 0x7ffa7c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.925+0000 7ffa8d7fa700 1 -- 172.21.15.49:0/6679904 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffa80089270 con 0x7ffa900ce2f0 2023-07-18T03:38:45.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.924+0000 7f6cb2ffd700 1 -- 172.21.15.49:0/2860860424 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8004230 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.925+0000 7f6cb2ffd700 1 -- 172.21.15.49:0/2860860424 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca8004390 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.925+0000 7f6cb2ffd700 1 -- 172.21.15.49:0/2860860424 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80036a0 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.926+0000 7f6cba3ff700 1 -- 172.21.15.49:0/2860860424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cb40ce2f0 msgr2=0x7f6cb40ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.926+0000 7f6cba3ff700 1 --2- 172.21.15.49:0/2860860424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cb40ce2f0 0x7f6cb40ca590 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009840 tx=0x7f6ca8009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:45.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.926+0000 7f6cba3ff700 1 -- 172.21.15.49:0/2860860424 shutdown_connections 2023-07-18T03:38:45.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.926+0000 7f6cba3ff700 1 --2- 172.21.15.49:0/2860860424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cb40ce2f0 0x7f6cb40ca590 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.926+0000 7f6cba3ff700 1 --2- 172.21.15.49:0/2860860424 >> v2:172.21.15.49:3300/0 conn(0x7f6cb416da60 0x7f6cb40cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.926+0000 7f6cba3ff700 1 -- 172.21.15.49:0/2860860424 >> 172.21.15.49:0/2860860424 conn(0x7f6cb40b05d0 msgr2=0x7f6cb40b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:45.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.927+0000 7ffa8ffff700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa7c06adb0 0x7ffa7c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.926+0000 7f6cba3ff700 1 -- 172.21.15.49:0/2860860424 shutdown_connections 2023-07-18T03:38:45.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.927+0000 7f6cba3ff700 1 -- 172.21.15.49:0/2860860424 wait complete. 2023-07-18T03:38:45.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.929+0000 7ffa96bff700 1 -- 172.21.15.49:0/6679904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa700052f0 con 0x7ffa900ce2f0 2023-07-18T03:38:45.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.930+0000 7f6cba3ff700 1 Processor -- start 2023-07-18T03:38:45.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.930+0000 7f6cba3ff700 1 -- start start 2023-07-18T03:38:45.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.932+0000 7ffa8ffff700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa7c06adb0 0x7ffa7c06d260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffa900b21f0 tx=0x7ffa84009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.932+0000 7f6cba3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40ce2f0 0x7f6cb4089f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.932+0000 7f6cba3ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cb416cfa0 0x7f6cb4162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.932+0000 7f6cba3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cb416da60 0x7f6cb4165430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.932+0000 7f6cba3ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb416b720 con 0x7f6cb416cfa0 2023-07-18T03:38:45.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.932+0000 7f6cba3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40b3df0 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.932+0000 7f6cba3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb416ad80 con 0x7f6cb416da60 2023-07-18T03:38:45.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.932+0000 7f6cb3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40ce2f0 0x7f6cb4089f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.933+0000 7f6cb3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40ce2f0 0x7f6cb4089f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:45430/0 (socket says 172.21.15.49:45430) 2023-07-18T03:38:45.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.933+0000 7f6cb3fff700 1 -- 172.21.15.49:0/353740814 learned_addr learned my addr 172.21.15.49:0/353740814 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:45.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.933+0000 7f6cb37fe700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cb416cfa0 0x7f6cb4162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.933+0000 7f6cb899c700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cb416da60 0x7f6cb4165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.933+0000 7f6cb3fff700 1 -- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cb416da60 msgr2=0x7f6cb4165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.933+0000 7f6cb3fff700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cb416da60 0x7f6cb4165430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.933+0000 7f6cb3fff700 1 -- 172.21.15.49:0/353740814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cb416cfa0 msgr2=0x7f6cb4162cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:45.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.933+0000 7f6cb3fff700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cb416cfa0 0x7f6cb4162cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:45.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.933+0000 7f6cb3fff700 1 -- 172.21.15.49:0/353740814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca80094a0 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.934+0000 7f6cb3fff700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40ce2f0 0x7f6cb4089f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009ea0 tx=0x7f6cac00e430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:45.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.935+0000 7f6cb0ff9700 1 -- 172.21.15.49:0/353740814 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac007e70 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.935+0000 7f6cb0ff9700 1 -- 172.21.15.49:0/353740814 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac00ea40 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.935+0000 7f6cb0ff9700 1 -- 172.21.15.49:0/353740814 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00edd0 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.935+0000 7f6cba3ff700 1 -- 172.21.15.49:0/353740814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb416b060 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.935+0000 7f6cba3ff700 1 -- 172.21.15.49:0/353740814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb408a460 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.937+0000 7ffa8d7fa700 1 -- 172.21.15.49:0/6679904 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffa800543c0 con 0x7ffa900ce2f0 2023-07-18T03:38:45.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.938+0000 7f6cba3ff700 1 -- 172.21.15.49:0/353740814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c940052f0 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.940+0000 7f6cb0ff9700 1 -- 172.21.15.49:0/353740814 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6cac009070 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.942+0000 7f6cb0ff9700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6ca006adb0 0x7f6ca006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.942+0000 7f6cb0ff9700 1 -- 172.21.15.49:0/353740814 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6cac089cf0 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.944+0000 7f6cb0ff9700 1 -- 172.21.15.49:0/353740814 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6cac054e10 con 0x7f6cb40ce2f0 2023-07-18T03:38:45.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.944+0000 7f6cb37fe700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6ca006adb0 0x7f6ca006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:45.985+0000 7f6cb37fe700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6ca006adb0 0x7f6ca006d260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40012e0 tx=0x7f6ca4001270 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:46.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.208+0000 7fbb441ff700 1 -- 172.21.15.49:0/4107459748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbb240055a0 con 0x7fbb3c160c30 2023-07-18T03:38:46.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.209+0000 7fbb327fc700 1 -- 172.21.15.49:0/4107459748 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb2c015070 con 0x7fbb3c160c30 2023-07-18T03:38:46.205 INFO:teuthology.orchestra.run.smithi049.stdout:60129542151 2023-07-18T03:38:46.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.214+0000 7fbb441ff700 1 -- 172.21.15.49:0/4107459748 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb2006adb0 msgr2=0x7fbb2006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.214+0000 7fbb441ff700 1 --2- 172.21.15.49:0/4107459748 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb2006adb0 0x7fbb2006d260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb34000f80 tx=0x7fbb34009000 comp rx=0 tx=0).stop 2023-07-18T03:38:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.214+0000 7fbb441ff700 1 -- 172.21.15.49:0/4107459748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c160c30 msgr2=0x7fbb3c0c3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.214+0000 7fbb441ff700 1 --2- 172.21.15.49:0/4107459748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c160c30 0x7fbb3c0c3f80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00b480 tx=0x7fbb2c00b840 comp rx=0 tx=0).stop 2023-07-18T03:38:46.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.214+0000 7fbb441ff700 1 -- 172.21.15.49:0/4107459748 shutdown_connections 2023-07-18T03:38:46.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.214+0000 7fbb441ff700 1 --2- 172.21.15.49:0/4107459748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb3c164e20 0x7fbb3c151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.214+0000 7fbb441ff700 1 --2- 172.21.15.49:0/4107459748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb3c0cdbe0 0x7fbb3c149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.214+0000 7fbb441ff700 1 --2- 172.21.15.49:0/4107459748 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb2006adb0 0x7fbb2006d260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.214+0000 7fbb441ff700 1 --2- 172.21.15.49:0/4107459748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c160c30 0x7fbb3c0c3f80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.214+0000 7fbb441ff700 1 -- 172.21.15.49:0/4107459748 >> 172.21.15.49:0/4107459748 conn(0x7fbb3c0b05d0 msgr2=0x7fbb3c0ae780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:46.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.215+0000 7fbb441ff700 1 -- 172.21.15.49:0/4107459748 shutdown_connections 2023-07-18T03:38:46.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.215+0000 7fbb441ff700 1 -- 172.21.15.49:0/4107459748 wait complete. 2023-07-18T03:38:46.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.4 2023-07-18T03:38:46.247 DEBUG:teuthology.parallel:result is None 2023-07-18T03:38:46.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.286+0000 7fe5921ff700 1 -- 172.21.15.49:0/3957624603 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe58c15a630 con 0x7fe58c1587d0 2023-07-18T03:38:46.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.287+0000 7fe573fff700 1 -- 172.21.15.49:0/3957624603 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe58c15a630 con 0x7fe58c1587d0 2023-07-18T03:38:46.283 INFO:teuthology.orchestra.run.smithi049.stdout:64424509445 2023-07-18T03:38:46.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.290+0000 7fe5921ff700 1 -- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe57806ade0 msgr2=0x7fe57806d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.290+0000 7fe5921ff700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe57806ade0 0x7fe57806d290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe58400e200 tx=0x7fe58400b040 comp rx=0 tx=0).stop 2023-07-18T03:38:46.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.290+0000 7fe5921ff700 1 -- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe58c1587d0 msgr2=0x7fe58c15b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.290+0000 7fe5921ff700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe58c1587d0 0x7fe58c15b7f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe57c018000 tx=0x7fe57c004750 comp rx=0 tx=0).stop 2023-07-18T03:38:46.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.291+0000 7fe5921ff700 1 -- 172.21.15.49:0/3957624603 shutdown_connections 2023-07-18T03:38:46.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.291+0000 7fe5921ff700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe58c1587d0 0x7fe58c15b7f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.291+0000 7fe5921ff700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c0caff0 0x7fe58c0c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.291+0000 7fe5921ff700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe57806ade0 0x7fe57806d290 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.291+0000 7fe5921ff700 1 --2- 172.21.15.49:0/3957624603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe58c0cdf40 0x7fe58c15b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.291+0000 7fe5921ff700 1 -- 172.21.15.49:0/3957624603 >> 172.21.15.49:0/3957624603 conn(0x7fe58c0b05d0 msgr2=0x7fe58c0b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:46.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.291+0000 7fe5921ff700 1 -- 172.21.15.49:0/3957624603 shutdown_connections 2023-07-18T03:38:46.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.291+0000 7fe5921ff700 1 -- 172.21.15.49:0/3957624603 wait complete. 2023-07-18T03:38:46.314 INFO:teuthology.orchestra.run.smithi049.stdout:60129542150 2023-07-18T03:38:46.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.2 2023-07-18T03:38:46.317 DEBUG:teuthology.parallel:result is None 2023-07-18T03:38:46.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.321+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/4014547519 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe1c40c7cf0 con 0x7fe1c41587d0 2023-07-18T03:38:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.319+0000 7ffa96bff700 1 -- 172.21.15.49:0/6679904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ffa700055a0 con 0x7ffa900ce2f0 2023-07-18T03:38:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.319+0000 7ffa8d7fa700 1 -- 172.21.15.49:0/6679904 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffa80015030 con 0x7ffa900ce2f0 2023-07-18T03:38:46.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.322+0000 7ffa96bff700 1 -- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa7c06adb0 msgr2=0x7ffa7c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.322+0000 7ffa96bff700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa7c06adb0 0x7ffa7c06d260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffa900b21f0 tx=0x7ffa84009210 comp rx=0 tx=0).stop 2023-07-18T03:38:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.322+0000 7ffa96bff700 1 -- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa900ce2f0 msgr2=0x7ffa900b1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.322+0000 7ffa96bff700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa900ce2f0 0x7ffa900b1cb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000b440 tx=0x7ffa80009f90 comp rx=0 tx=0).stop 2023-07-18T03:38:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.323+0000 7ffa96bff700 1 -- 172.21.15.49:0/6679904 shutdown_connections 2023-07-18T03:38:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.323+0000 7ffa96bff700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9016da60 0x7ffa90167810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.323+0000 7ffa96bff700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa900ce2f0 0x7ffa900b1cb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.323+0000 7ffa96bff700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa7c06adb0 0x7ffa7c06d260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.323+0000 7ffa96bff700 1 --2- 172.21.15.49:0/6679904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9016cfa0 0x7ffa900c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.323+0000 7ffa96bff700 1 -- 172.21.15.49:0/6679904 >> 172.21.15.49:0/6679904 conn(0x7ffa900b05d0 msgr2=0x7ffa9006cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:46.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.323+0000 7ffa96bff700 1 -- 172.21.15.49:0/6679904 shutdown_connections 2023-07-18T03:38:46.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.323+0000 7ffa96bff700 1 -- 172.21.15.49:0/6679904 wait complete. 2023-07-18T03:38:46.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.324+0000 7fe1b9ffb700 1 -- 172.21.15.49:0/4014547519 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe1c002b020 con 0x7fe1c41587d0 2023-07-18T03:38:46.322 INFO:teuthology.orchestra.run.smithi049.stdout:64424509448 2023-07-18T03:38:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.327+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1a806acb0 msgr2=0x7fe1a806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.327+0000 7fe1cb3ff700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1a806acb0 0x7fe1a806d160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40b4910 tx=0x7fe1b4009190 comp rx=0 tx=0).stop 2023-07-18T03:38:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.327+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1c41587d0 msgr2=0x7fe1c40bf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.327+0000 7fe1cb3ff700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1c41587d0 0x7fe1c40bf9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0016000 tx=0x7fe1c0004750 comp rx=0 tx=0).stop 2023-07-18T03:38:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.327+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/4014547519 shutdown_connections 2023-07-18T03:38:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.327+0000 7fe1cb3ff700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1a806acb0 0x7fe1a806d160 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.327+0000 7fe1cb3ff700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1c4159ee0 0x7fe1c40cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.327+0000 7fe1cb3ff700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1c40ce480 0x7fe1c415b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.327+0000 7fe1cb3ff700 1 --2- 172.21.15.49:0/4014547519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1c41587d0 0x7fe1c40bf9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.327+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/4014547519 >> 172.21.15.49:0/4014547519 conn(0x7fe1c40b05d0 msgr2=0x7fe1c414dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:46.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.327+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/4014547519 shutdown_connections 2023-07-18T03:38:46.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.328+0000 7fe1cb3ff700 1 -- 172.21.15.49:0/4014547519 wait complete. 2023-07-18T03:38:46.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.332+0000 7fd9df3ff700 1 -- 172.21.15.49:0/2548765708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd9d806ca40 con 0x7fd9d80c4c20 2023-07-18T03:38:46.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.332+0000 7fd9cdffb700 1 -- 172.21.15.49:0/2548765708 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd9c8058980 con 0x7fd9d80c4c20 2023-07-18T03:38:46.327 INFO:teuthology.orchestra.run.smithi049.stdout:60129542151 2023-07-18T03:38:46.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.335+0000 7fd9df3ff700 1 -- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd9bc06adb0 msgr2=0x7fd9bc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.335+0000 7fd9df3ff700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd9bc06adb0 0x7fd9bc06d260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80c8c50 tx=0x7fd9d4005040 comp rx=0 tx=0).stop 2023-07-18T03:38:46.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.335+0000 7fd9df3ff700 1 -- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9d80c4c20 msgr2=0x7fd9d81f5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.335+0000 7fd9df3ff700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9d80c4c20 0x7fd9d81f5ee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8010860 tx=0x7fd9c8010c20 comp rx=0 tx=0).stop 2023-07-18T03:38:46.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.337+0000 7fd9df3ff700 1 -- 172.21.15.49:0/2548765708 shutdown_connections 2023-07-18T03:38:46.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.337+0000 7fd9df3ff700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9d80c5b90 0x7fd9d8070860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.337+0000 7fd9df3ff700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd9bc06adb0 0x7fd9bc06d260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.337+0000 7fd9df3ff700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c4003af0 0x7fd9d8159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.337+0000 7fd9df3ff700 1 --2- 172.21.15.49:0/2548765708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9d80c4c20 0x7fd9d81f5ee0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.337+0000 7fd9df3ff700 1 -- 172.21.15.49:0/2548765708 >> 172.21.15.49:0/2548765708 conn(0x7fd9d80b05d0 msgr2=0x7fd9d80b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:46.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.337+0000 7ff3cfdff700 1 -- 172.21.15.49:0/260651661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff3c806ca40 con 0x7ff3c806b5b0 2023-07-18T03:38:46.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.337+0000 7fd9df3ff700 1 -- 172.21.15.49:0/2548765708 shutdown_connections 2023-07-18T03:38:46.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.337+0000 7fd9df3ff700 1 -- 172.21.15.49:0/2548765708 wait complete. 2023-07-18T03:38:46.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.338+0000 7ff3be7fc700 1 -- 172.21.15.49:0/260651661 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff3c40565a0 con 0x7ff3c806b5b0 2023-07-18T03:38:46.334 INFO:teuthology.orchestra.run.smithi049.stdout:60129542149 2023-07-18T03:38:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.343+0000 7ff3cfdff700 1 -- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff3ac06ace0 msgr2=0x7ff3ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.343+0000 7ff3cfdff700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff3ac06ace0 0x7ff3ac06d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3c814a190 tx=0x7ff3b8007a20 comp rx=0 tx=0).stop 2023-07-18T03:38:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.343+0000 7ff3cfdff700 1 -- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3c806b5b0 msgr2=0x7ff3c8171670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.343+0000 7ff3cfdff700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3c806b5b0 0x7ff3c8171670 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3c806c670 tx=0x7ff3c4004750 comp rx=0 tx=0).stop 2023-07-18T03:38:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.344+0000 7ff3cfdff700 1 -- 172.21.15.49:0/260651661 shutdown_connections 2023-07-18T03:38:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.344+0000 7ff3cfdff700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3c80ce650 0x7ff3c816c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.344+0000 7ff3cfdff700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3c806b5b0 0x7ff3c8171670 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.344+0000 7ff3cfdff700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff3ac06ace0 0x7ff3ac06d190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.344+0000 7ff3cfdff700 1 --2- 172.21.15.49:0/260651661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3c80b10b0 0x7ff3c8171bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.344+0000 7ff3cfdff700 1 -- 172.21.15.49:0/260651661 >> 172.21.15.49:0/260651661 conn(0x7ff3c80b05d0 msgr2=0x7ff3c80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.344+0000 7ff3cfdff700 1 -- 172.21.15.49:0/260651661 shutdown_connections 2023-07-18T03:38:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.344+0000 7ff3cfdff700 1 -- 172.21.15.49:0/260651661 wait complete. 2023-07-18T03:38:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.351+0000 7fbf36fff700 1 -- 172.21.15.49:0/471490046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbf30165020 con 0x7fbf300cd940 2023-07-18T03:38:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.352+0000 7fbf2d7fa700 1 -- 172.21.15.49:0/471490046 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbf24017090 con 0x7fbf300cd940 2023-07-18T03:38:46.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2023-07-18T03:38:46.348 INFO:teuthology.orchestra.run.smithi049.stdout:64424509446 2023-07-18T03:38:46.348 DEBUG:teuthology.parallel:result is None 2023-07-18T03:38:46.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.356+0000 7fbf36fff700 1 -- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbf1c06adb0 msgr2=0x7fbf1c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.356+0000 7fbf36fff700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbf1c06adb0 0x7fbf1c06d260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf28000f80 tx=0x7fbf28010000 comp rx=0 tx=0).stop 2023-07-18T03:38:46.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.356+0000 7fbf36fff700 1 -- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf300cd940 msgr2=0x7fbf300c44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.356+0000 7fbf36fff700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf300cd940 0x7fbf300c44c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400c480 tx=0x7fbf2400c790 comp rx=0 tx=0).stop 2023-07-18T03:38:46.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.356+0000 7fbf36fff700 1 -- 172.21.15.49:0/471490046 shutdown_connections 2023-07-18T03:38:46.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.356+0000 7fbf36fff700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf30160d50 0x7fbf300c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.356+0000 7fbf36fff700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbf1c06adb0 0x7fbf1c06d260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.356+0000 7fbf36fff700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf3006cd60 0x7fbf300b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.356+0000 7fbf36fff700 1 --2- 172.21.15.49:0/471490046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf300cd940 0x7fbf300c44c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.356+0000 7fbf36fff700 1 -- 172.21.15.49:0/471490046 >> 172.21.15.49:0/471490046 conn(0x7fbf300b05d0 msgr2=0x7fbf300cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:46.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.356+0000 7fbf36fff700 1 -- 172.21.15.49:0/471490046 shutdown_connections 2023-07-18T03:38:46.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.356+0000 7fbf36fff700 1 -- 172.21.15.49:0/471490046 wait complete. 2023-07-18T03:38:46.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.0 2023-07-18T03:38:46.355 DEBUG:teuthology.parallel:result is None 2023-07-18T03:38:46.357 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.3 2023-07-18T03:38:46.358 DEBUG:teuthology.parallel:result is None 2023-07-18T03:38:46.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.370+0000 7f6cba3ff700 1 -- 172.21.15.49:0/353740814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6c94005870 con 0x7f6cb40ce2f0 2023-07-18T03:38:46.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.370+0000 7f6cb0ff9700 1 -- 172.21.15.49:0/353740814 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6cac019020 con 0x7f6cb40ce2f0 2023-07-18T03:38:46.365 INFO:teuthology.orchestra.run.smithi049.stdout:64424509447 2023-07-18T03:38:46.367 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2023-07-18T03:38:46.367 DEBUG:teuthology.parallel:result is None 2023-07-18T03:38:46.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.374+0000 7f6cba3ff700 1 -- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6ca006adb0 msgr2=0x7f6ca006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.374+0000 7f6cba3ff700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6ca006adb0 0x7f6ca006d260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40012e0 tx=0x7f6ca4001270 comp rx=0 tx=0).stop 2023-07-18T03:38:46.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.374+0000 7f6cba3ff700 1 -- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40ce2f0 msgr2=0x7f6cb4089f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.374+0000 7f6cba3ff700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40ce2f0 0x7f6cb4089f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009ea0 tx=0x7f6cac00e430 comp rx=0 tx=0).stop 2023-07-18T03:38:46.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.374+0000 7f6cba3ff700 1 -- 172.21.15.49:0/353740814 shutdown_connections 2023-07-18T03:38:46.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.374+0000 7f6cba3ff700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6cb416da60 0x7f6cb4165430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.374+0000 7f6cba3ff700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cb40ce2f0 0x7f6cb4089f20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.374+0000 7f6cba3ff700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6ca006adb0 0x7f6ca006d260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.374+0000 7f6cba3ff700 1 --2- 172.21.15.49:0/353740814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cb416cfa0 0x7f6cb4162cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.374+0000 7f6cba3ff700 1 -- 172.21.15.49:0/353740814 >> 172.21.15.49:0/353740814 conn(0x7f6cb40b05d0 msgr2=0x7f6cb4070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:46.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.374+0000 7f6cba3ff700 1 -- 172.21.15.49:0/353740814 shutdown_connections 2023-07-18T03:38:46.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.374+0000 7f6cba3ff700 1 -- 172.21.15.49:0/353740814 wait complete. 2023-07-18T03:38:46.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2023-07-18T03:38:46.377 DEBUG:teuthology.parallel:result is None 2023-07-18T03:38:46.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2023-07-18T03:38:46.389 DEBUG:teuthology.parallel:result is None 2023-07-18T03:38:46.389 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-18T03:38:46.390 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:38:46.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.637+0000 7fe958dff700 1 Processor -- start 2023-07-18T03:38:46.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.637+0000 7fe958dff700 1 -- start start 2023-07-18T03:38:46.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.639+0000 7fe958dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe9540b87d0 0x7fe954160150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:46.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.639+0000 7fe958dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9540cfc60 0x7fe954160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:46.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.639+0000 7fe958dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9540cd4e0 con 0x7fe95406ae40 2023-07-18T03:38:46.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.639+0000 7fe958dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9540cdcf0 con 0x7fe9540b87d0 2023-07-18T03:38:46.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.639+0000 7fe958dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9540d00d0 con 0x7fe9540cfc60 2023-07-18T03:38:46.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.639+0000 7fe952d9d700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe95406ae40 0x7fe9540b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41668/0 (socket says 172.21.15.49:41668) 2023-07-18T03:38:46.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.639+0000 7fe952d9d700 1 -- 172.21.15.49:0/4004434263 learned_addr learned my addr 172.21.15.49:0/4004434263 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:46.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.639+0000 7fe94bfff700 1 --2- 172.21.15.49:0/4004434263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9540cfc60 0x7fe954160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:46.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.639+0000 7fe95259c700 1 --2- 172.21.15.49:0/4004434263 >> v2:172.21.15.107:3300/0 conn(0x7fe9540b87d0 0x7fe954160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:46.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.639+0000 7fe951d9b700 1 -- 172.21.15.49:0/4004434263 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985484350 0 0) 0x7fe9540cd4e0 con 0x7fe95406ae40 2023-07-18T03:38:46.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.639+0000 7fe951d9b700 1 -- 172.21.15.49:0/4004434263 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9400035b0 con 0x7fe95406ae40 2023-07-18T03:38:46.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.640+0000 7fe94bfff700 1 -- 172.21.15.49:0/4004434263 >> v2:172.21.15.107:3300/0 conn(0x7fe9540b87d0 msgr2=0x7fe954160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:46.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.640+0000 7fe94bfff700 1 --2- 172.21.15.49:0/4004434263 >> v2:172.21.15.107:3300/0 conn(0x7fe9540b87d0 0x7fe954160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.640+0000 7fe94bfff700 1 -- 172.21.15.49:0/4004434263 >> v1:172.21.15.49:6789/0 conn(0x7fe95406ae40 legacy=0x7fe9540b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.640+0000 7fe94bfff700 1 -- 172.21.15.49:0/4004434263 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9540cd730 con 0x7fe9540cfc60 2023-07-18T03:38:46.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.640+0000 7fe95259c700 1 --2- 172.21.15.49:0/4004434263 >> v2:172.21.15.107:3300/0 conn(0x7fe9540b87d0 0x7fe954160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:46.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.640+0000 7fe94bfff700 1 --2- 172.21.15.49:0/4004434263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9540cfc60 0x7fe954160690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009840 tx=0x7fe93c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4661a5b250054a3e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:46.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.640+0000 7fe951d9b700 1 -- 172.21.15.49:0/4004434263 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c011830 con 0x7fe9540cfc60 2023-07-18T03:38:46.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.640+0000 7fe951d9b700 1 -- 172.21.15.49:0/4004434263 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c011990 con 0x7fe9540cfc60 2023-07-18T03:38:46.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.640+0000 7fe951d9b700 1 -- 172.21.15.49:0/4004434263 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c011d20 con 0x7fe9540cfc60 2023-07-18T03:38:46.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.641+0000 7fe958dff700 1 -- 172.21.15.49:0/4004434263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9540cfc60 msgr2=0x7fe954160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.641+0000 7fe958dff700 1 --2- 172.21.15.49:0/4004434263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9540cfc60 0x7fe954160690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009840 tx=0x7fe93c009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:46.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.641+0000 7fe958dff700 1 -- 172.21.15.49:0/4004434263 shutdown_connections 2023-07-18T03:38:46.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.641+0000 7fe958dff700 1 --2- 172.21.15.49:0/4004434263 >> v2:172.21.15.107:3300/0 conn(0x7fe9540b87d0 0x7fe954160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.641+0000 7fe958dff700 1 --2- 172.21.15.49:0/4004434263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9540cfc60 0x7fe954160690 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.641+0000 7fe958dff700 1 -- 172.21.15.49:0/4004434263 >> 172.21.15.49:0/4004434263 conn(0x7fe95406c140 msgr2=0x7fe95406c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:46.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.641+0000 7fe958dff700 1 -- 172.21.15.49:0/4004434263 shutdown_connections 2023-07-18T03:38:46.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.641+0000 7fe958dff700 1 -- 172.21.15.49:0/4004434263 wait complete. 2023-07-18T03:38:46.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.643+0000 7fe958dff700 1 Processor -- start 2023-07-18T03:38:46.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.643+0000 7fe958dff700 1 -- start start 2023-07-18T03:38:46.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.644+0000 7fe958dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9540b87d0 0x7fe9540b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:46.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.644+0000 7fe958dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9540cfc60 0x7fe95415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:46.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.644+0000 7fe958dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9540c7640 0x7fe9540b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:46.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.644+0000 7fe958dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95415b8f0 con 0x7fe9540b87d0 2023-07-18T03:38:46.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.644+0000 7fe958dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9540b3730 con 0x7fe9540cfc60 2023-07-18T03:38:46.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.644+0000 7fe958dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9540b2150 con 0x7fe9540c7640 2023-07-18T03:38:46.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.644+0000 7fe95259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9540b87d0 0x7fe9540b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:46.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.644+0000 7fe95259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9540b87d0 0x7fe9540b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40652/0 (socket says 172.21.15.49:40652) 2023-07-18T03:38:46.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.644+0000 7fe95259c700 1 -- 172.21.15.49:0/507084161 learned_addr learned my addr 172.21.15.49:0/507084161 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:46.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.644+0000 7fe952d9d700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9540c7640 0x7fe9540b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:46.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.644+0000 7fe94bfff700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9540cfc60 0x7fe95415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:46.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.645+0000 7fe95259c700 1 -- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9540c7640 msgr2=0x7fe9540b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.645+0000 7fe95259c700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9540c7640 0x7fe9540b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.645+0000 7fe95259c700 1 -- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9540cfc60 msgr2=0x7fe95415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.645+0000 7fe95259c700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9540cfc60 0x7fe95415b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.645+0000 7fe95259c700 1 -- 172.21.15.49:0/507084161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9440094a0 con 0x7fe9540b87d0 2023-07-18T03:38:46.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.645+0000 7fe95259c700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9540b87d0 0x7fe9540b4a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe94400ebe0 tx=0x7fe94400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:46.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.645+0000 7fe94affd700 1 -- 172.21.15.49:0/507084161 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94400c850 con 0x7fe9540b87d0 2023-07-18T03:38:46.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.645+0000 7fe94affd700 1 -- 172.21.15.49:0/507084161 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94400c9b0 con 0x7fe9540b87d0 2023-07-18T03:38:46.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.645+0000 7fe94affd700 1 -- 172.21.15.49:0/507084161 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94400cd40 con 0x7fe9540b87d0 2023-07-18T03:38:46.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.645+0000 7fe958dff700 1 -- 172.21.15.49:0/507084161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93c0094a0 con 0x7fe9540b87d0 2023-07-18T03:38:46.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.645+0000 7fe958dff700 1 -- 172.21.15.49:0/507084161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9540b2710 con 0x7fe9540b87d0 2023-07-18T03:38:46.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.646+0000 7fe958dff700 1 -- 172.21.15.49:0/507084161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9540b3040 con 0x7fe9540b87d0 2023-07-18T03:38:46.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.647+0000 7fe94affd700 1 -- 172.21.15.49:0/507084161 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe944013470 con 0x7fe9540b87d0 2023-07-18T03:38:46.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.648+0000 7fe94affd700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe93406ace0 0x7fe93406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:46.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.648+0000 7fe94affd700 1 -- 172.21.15.49:0/507084161 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe944089540 con 0x7fe9540b87d0 2023-07-18T03:38:46.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.649+0000 7fe94bfff700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe93406ace0 0x7fe93406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:46.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.649+0000 7fe94bfff700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe93406ace0 0x7fe93406d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe93c000c00 tx=0x7fe93c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:46.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.653+0000 7fe94affd700 1 -- 172.21.15.49:0/507084161 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe9440545e0 con 0x7fe9540b87d0 2023-07-18T03:38:46.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.857+0000 7fe958dff700 1 -- 172.21.15.49:0/507084161 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9540cda10 con 0x7fe93406ace0 2023-07-18T03:38:46.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.867+0000 7fe94affd700 1 -- 172.21.15.49:0/507084161 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286867 (secure 0 0 0) 0x7fe93c011b00 con 0x7fe93406ace0 2023-07-18T03:38:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:46.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.871+0000 7fe958dff700 1 -- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe93406ace0 msgr2=0x7fe93406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.871+0000 7fe958dff700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe93406ace0 0x7fe93406d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe93c000c00 tx=0x7fe93c008030 comp rx=0 tx=0).stop 2023-07-18T03:38:46.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.871+0000 7fe958dff700 1 -- 172.21.15.49:0/507084161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9540b87d0 msgr2=0x7fe9540b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:46.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.871+0000 7fe958dff700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9540b87d0 0x7fe9540b4a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe94400ebe0 tx=0x7fe94400eef0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.871+0000 7fe958dff700 1 -- 172.21.15.49:0/507084161 shutdown_connections 2023-07-18T03:38:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.871+0000 7fe958dff700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9540c7640 0x7fe9540b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.871+0000 7fe958dff700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe93406ace0 0x7fe93406d190 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.871+0000 7fe958dff700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9540b87d0 0x7fe9540b4a20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.871+0000 7fe958dff700 1 --2- 172.21.15.49:0/507084161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9540cfc60 0x7fe95415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:46.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.871+0000 7fe958dff700 1 -- 172.21.15.49:0/507084161 >> 172.21.15.49:0/507084161 conn(0x7fe95406c140 msgr2=0x7fe95415cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:46.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.871+0000 7fe958dff700 1 -- 172.21.15.49:0/507084161 shutdown_connections 2023-07-18T03:38:46.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:46.871+0000 7fe958dff700 1 -- 172.21.15.49:0/507084161 wait complete. 2023-07-18T03:38:46.872 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:38:46.885 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-07-18T03:38:46.099132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230856,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":227840,"kb_avail":754743864,"statfs":{"total":773094113280,"available":772857716736,"internally_reserved":0,"allocated":3022848,"data_stored":1835060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":233308160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.588613"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.122105+0000","last_change":"2023-07-18T03:38:36.538478+0000","last_active":"2023-07-18T03:38:38.122105+0000","last_peered":"2023-07-18T03:38:38.122105+0000","last_clean":"2023-07-18T03:38:38.122105+0000","last_became_active":"2023-07-18T03:38:36.538174+0000","last_became_peered":"2023-07-18T03:38:36.538174+0000","last_unstale":"2023-07-18T03:38:38.122105+0000","last_undegraded":"2023-07-18T03:38:38.122105+0000","last_fullsized":"2023-07-18T03:38:38.122105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:10:09.118957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629549+0000","last_change":"2023-07-18T03:38:36.538553+0000","last_active":"2023-07-18T03:38:37.629549+0000","last_peered":"2023-07-18T03:38:37.629549+0000","last_clean":"2023-07-18T03:38:37.629549+0000","last_became_active":"2023-07-18T03:38:36.538257+0000","last_became_peered":"2023-07-18T03:38:36.538257+0000","last_unstale":"2023-07-18T03:38:37.629549+0000","last_undegraded":"2023-07-18T03:38:37.629549+0000","last_fullsized":"2023-07-18T03:38:37.629549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:42.081234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515821+0000","last_change":"2023-07-18T03:38:36.526262+0000","last_active":"2023-07-18T03:38:37.515821+0000","last_peered":"2023-07-18T03:38:37.515821+0000","last_clean":"2023-07-18T03:38:37.515821+0000","last_became_active":"2023-07-18T03:38:36.525920+0000","last_became_peered":"2023-07-18T03:38:36.525920+0000","last_unstale":"2023-07-18T03:38:37.515821+0000","last_undegraded":"2023-07-18T03:38:37.515821+0000","last_fullsized":"2023-07-18T03:38:37.515821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:09.557961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121429+0000","last_change":"2023-07-18T03:38:36.535992+0000","last_active":"2023-07-18T03:38:38.121429+0000","last_peered":"2023-07-18T03:38:38.121429+0000","last_clean":"2023-07-18T03:38:38.121429+0000","last_became_active":"2023-07-18T03:38:36.535587+0000","last_became_peered":"2023-07-18T03:38:36.535587+0000","last_unstale":"2023-07-18T03:38:38.121429+0000","last_undegraded":"2023-07-18T03:38:38.121429+0000","last_fullsized":"2023-07-18T03:38:38.121429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:24.505474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121029+0000","last_change":"2023-07-18T03:38:36.535524+0000","last_active":"2023-07-18T03:38:38.121029+0000","last_peered":"2023-07-18T03:38:38.121029+0000","last_clean":"2023-07-18T03:38:38.121029+0000","last_became_active":"2023-07-18T03:38:36.534975+0000","last_became_peered":"2023-07-18T03:38:36.534975+0000","last_unstale":"2023-07-18T03:38:38.121029+0000","last_undegraded":"2023-07-18T03:38:38.121029+0000","last_fullsized":"2023-07-18T03:38:38.121029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:37:55.979750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515789+0000","last_change":"2023-07-18T03:38:36.526306+0000","last_active":"2023-07-18T03:38:37.515789+0000","last_peered":"2023-07-18T03:38:37.515789+0000","last_clean":"2023-07-18T03:38:37.515789+0000","last_became_active":"2023-07-18T03:38:36.525982+0000","last_became_peered":"2023-07-18T03:38:36.525982+0000","last_unstale":"2023-07-18T03:38:37.515789+0000","last_undegraded":"2023-07-18T03:38:37.515789+0000","last_fullsized":"2023-07-18T03:38:37.515789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:14:38.515543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516000+0000","last_change":"2023-07-18T03:38:36.527124+0000","last_active":"2023-07-18T03:38:37.516000+0000","last_peered":"2023-07-18T03:38:37.516000+0000","last_clean":"2023-07-18T03:38:37.516000+0000","last_became_active":"2023-07-18T03:38:36.526827+0000","last_became_peered":"2023-07-18T03:38:36.526827+0000","last_unstale":"2023-07-18T03:38:37.516000+0000","last_undegraded":"2023-07-18T03:38:37.516000+0000","last_fullsized":"2023-07-18T03:38:37.516000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:21:51.299094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629188+0000","last_change":"2023-07-18T03:38:36.529550+0000","last_active":"2023-07-18T03:38:37.629188+0000","last_peered":"2023-07-18T03:38:37.629188+0000","last_clean":"2023-07-18T03:38:37.629188+0000","last_became_active":"2023-07-18T03:38:36.529261+0000","last_became_peered":"2023-07-18T03:38:36.529261+0000","last_unstale":"2023-07-18T03:38:37.629188+0000","last_undegraded":"2023-07-18T03:38:37.629188+0000","last_fullsized":"2023-07-18T03:38:37.629188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:04.554555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121235+0000","last_change":"2023-07-18T03:38:36.534909+0000","last_active":"2023-07-18T03:38:38.121235+0000","last_peered":"2023-07-18T03:38:38.121235+0000","last_clean":"2023-07-18T03:38:38.121235+0000","last_became_active":"2023-07-18T03:38:36.534593+0000","last_became_peered":"2023-07-18T03:38:36.534593+0000","last_unstale":"2023-07-18T03:38:38.121235+0000","last_undegraded":"2023-07-18T03:38:38.121235+0000","last_fullsized":"2023-07-18T03:38:38.121235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:45:51.736824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629684+0000","last_change":"2023-07-18T03:38:36.536965+0000","last_active":"2023-07-18T03:38:37.629684+0000","last_peered":"2023-07-18T03:38:37.629684+0000","last_clean":"2023-07-18T03:38:37.629684+0000","last_became_active":"2023-07-18T03:38:36.536665+0000","last_became_peered":"2023-07-18T03:38:36.536665+0000","last_unstale":"2023-07-18T03:38:37.629684+0000","last_undegraded":"2023-07-18T03:38:37.629684+0000","last_fullsized":"2023-07-18T03:38:37.629684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:45.182024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515971+0000","last_change":"2023-07-18T03:38:36.526499+0000","last_active":"2023-07-18T03:38:37.515971+0000","last_peered":"2023-07-18T03:38:37.515971+0000","last_clean":"2023-07-18T03:38:37.515971+0000","last_became_active":"2023-07-18T03:38:36.525871+0000","last_became_peered":"2023-07-18T03:38:36.525871+0000","last_unstale":"2023-07-18T03:38:37.515971+0000","last_undegraded":"2023-07-18T03:38:37.515971+0000","last_fullsized":"2023-07-18T03:38:37.515971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:42.081234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121914+0000","last_change":"2023-07-18T03:38:36.524528+0000","last_active":"2023-07-18T03:38:38.121914+0000","last_peered":"2023-07-18T03:38:38.121914+0000","last_clean":"2023-07-18T03:38:38.121914+0000","last_became_active":"2023-07-18T03:38:36.523892+0000","last_became_peered":"2023-07-18T03:38:36.523892+0000","last_unstale":"2023-07-18T03:38:38.121914+0000","last_undegraded":"2023-07-18T03:38:38.121914+0000","last_fullsized":"2023-07-18T03:38:38.121914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:01:02.396334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121759+0000","last_change":"2023-07-18T03:38:36.533293+0000","last_active":"2023-07-18T03:38:38.121759+0000","last_peered":"2023-07-18T03:38:38.121759+0000","last_clean":"2023-07-18T03:38:38.121759+0000","last_became_active":"2023-07-18T03:38:36.532953+0000","last_became_peered":"2023-07-18T03:38:36.532953+0000","last_unstale":"2023-07-18T03:38:38.121759+0000","last_undegraded":"2023-07-18T03:38:38.121759+0000","last_fullsized":"2023-07-18T03:38:38.121759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:35:14.137991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629259+0000","last_change":"2023-07-18T03:38:36.536500+0000","last_active":"2023-07-18T03:38:37.629259+0000","last_peered":"2023-07-18T03:38:37.629259+0000","last_clean":"2023-07-18T03:38:37.629259+0000","last_became_active":"2023-07-18T03:38:36.536283+0000","last_became_peered":"2023-07-18T03:38:36.536283+0000","last_unstale":"2023-07-18T03:38:37.629259+0000","last_undegraded":"2023-07-18T03:38:37.629259+0000","last_fullsized":"2023-07-18T03:38:37.629259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:24.505474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516206+0000","last_change":"2023-07-18T03:38:36.530577+0000","last_active":"2023-07-18T03:38:37.516206+0000","last_peered":"2023-07-18T03:38:37.516206+0000","last_clean":"2023-07-18T03:38:37.516206+0000","last_became_active":"2023-07-18T03:38:36.530268+0000","last_became_peered":"2023-07-18T03:38:36.530268+0000","last_unstale":"2023-07-18T03:38:37.516206+0000","last_undegraded":"2023-07-18T03:38:37.516206+0000","last_fullsized":"2023-07-18T03:38:37.516206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:33:14.215460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516748+0000","last_change":"2023-07-18T03:38:36.526520+0000","last_active":"2023-07-18T03:38:37.516748+0000","last_peered":"2023-07-18T03:38:37.516748+0000","last_clean":"2023-07-18T03:38:37.516748+0000","last_became_active":"2023-07-18T03:38:36.526165+0000","last_became_peered":"2023-07-18T03:38:36.526165+0000","last_unstale":"2023-07-18T03:38:37.516748+0000","last_undegraded":"2023-07-18T03:38:37.516748+0000","last_fullsized":"2023-07-18T03:38:37.516748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:17:16.904471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121313+0000","last_change":"2023-07-18T03:38:36.531837+0000","last_active":"2023-07-18T03:38:38.121313+0000","last_peered":"2023-07-18T03:38:38.121313+0000","last_clean":"2023-07-18T03:38:38.121313+0000","last_became_active":"2023-07-18T03:38:36.531588+0000","last_became_peered":"2023-07-18T03:38:36.531588+0000","last_unstale":"2023-07-18T03:38:38.121313+0000","last_undegraded":"2023-07-18T03:38:38.121313+0000","last_fullsized":"2023-07-18T03:38:38.121313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:57:34.438689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516789+0000","last_change":"2023-07-18T03:38:36.536129+0000","last_active":"2023-07-18T03:38:37.516789+0000","last_peered":"2023-07-18T03:38:37.516789+0000","last_clean":"2023-07-18T03:38:37.516789+0000","last_became_active":"2023-07-18T03:38:36.535846+0000","last_became_peered":"2023-07-18T03:38:36.535846+0000","last_unstale":"2023-07-18T03:38:37.516789+0000","last_undegraded":"2023-07-18T03:38:37.516789+0000","last_fullsized":"2023-07-18T03:38:37.516789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:50:17.597467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.122134+0000","last_change":"2023-07-18T03:38:36.529212+0000","last_active":"2023-07-18T03:38:38.122134+0000","last_peered":"2023-07-18T03:38:38.122134+0000","last_clean":"2023-07-18T03:38:38.122134+0000","last_became_active":"2023-07-18T03:38:36.528956+0000","last_became_peered":"2023-07-18T03:38:36.528956+0000","last_unstale":"2023-07-18T03:38:38.122134+0000","last_undegraded":"2023-07-18T03:38:38.122134+0000","last_fullsized":"2023-07-18T03:38:38.122134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:23:08.908073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516481+0000","last_change":"2023-07-18T03:38:36.532777+0000","last_active":"2023-07-18T03:38:37.516481+0000","last_peered":"2023-07-18T03:38:37.516481+0000","last_clean":"2023-07-18T03:38:37.516481+0000","last_became_active":"2023-07-18T03:38:36.532519+0000","last_became_peered":"2023-07-18T03:38:36.532519+0000","last_unstale":"2023-07-18T03:38:37.516481+0000","last_undegraded":"2023-07-18T03:38:37.516481+0000","last_fullsized":"2023-07-18T03:38:37.516481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:27:43.633946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515979+0000","last_change":"2023-07-18T03:38:36.541130+0000","last_active":"2023-07-18T03:38:37.515979+0000","last_peered":"2023-07-18T03:38:37.515979+0000","last_clean":"2023-07-18T03:38:37.515979+0000","last_became_active":"2023-07-18T03:38:36.535483+0000","last_became_peered":"2023-07-18T03:38:36.535483+0000","last_unstale":"2023-07-18T03:38:37.515979+0000","last_undegraded":"2023-07-18T03:38:37.515979+0000","last_fullsized":"2023-07-18T03:38:37.515979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:46.558463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120549+0000","last_change":"2023-07-18T03:38:36.536797+0000","last_active":"2023-07-18T03:38:38.120549+0000","last_peered":"2023-07-18T03:38:38.120549+0000","last_clean":"2023-07-18T03:38:38.120549+0000","last_became_active":"2023-07-18T03:38:36.535389+0000","last_became_peered":"2023-07-18T03:38:36.535389+0000","last_unstale":"2023-07-18T03:38:38.120549+0000","last_undegraded":"2023-07-18T03:38:38.120549+0000","last_fullsized":"2023-07-18T03:38:38.120549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:22:44.555224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120428+0000","last_change":"2023-07-18T03:38:36.533007+0000","last_active":"2023-07-18T03:38:38.120428+0000","last_peered":"2023-07-18T03:38:38.120428+0000","last_clean":"2023-07-18T03:38:38.120428+0000","last_became_active":"2023-07-18T03:38:36.532714+0000","last_became_peered":"2023-07-18T03:38:36.532714+0000","last_unstale":"2023-07-18T03:38:38.120428+0000","last_undegraded":"2023-07-18T03:38:38.120428+0000","last_fullsized":"2023-07-18T03:38:38.120428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:14:38.515543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515725+0000","last_change":"2023-07-18T03:38:36.541077+0000","last_active":"2023-07-18T03:38:37.515725+0000","last_peered":"2023-07-18T03:38:37.515725+0000","last_clean":"2023-07-18T03:38:37.515725+0000","last_became_active":"2023-07-18T03:38:36.533768+0000","last_became_peered":"2023-07-18T03:38:36.533768+0000","last_unstale":"2023-07-18T03:38:37.515725+0000","last_undegraded":"2023-07-18T03:38:37.515725+0000","last_fullsized":"2023-07-18T03:38:37.515725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:42.081234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514860+0000","last_change":"2023-07-18T03:38:36.529999+0000","last_active":"2023-07-18T03:38:37.514860+0000","last_peered":"2023-07-18T03:38:37.514860+0000","last_clean":"2023-07-18T03:38:37.514860+0000","last_became_active":"2023-07-18T03:38:36.529489+0000","last_became_peered":"2023-07-18T03:38:36.529489+0000","last_unstale":"2023-07-18T03:38:37.514860+0000","last_undegraded":"2023-07-18T03:38:37.514860+0000","last_fullsized":"2023-07-18T03:38:37.514860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:24.505474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514898+0000","last_change":"2023-07-18T03:38:36.525279+0000","last_active":"2023-07-18T03:38:37.514898+0000","last_peered":"2023-07-18T03:38:37.514898+0000","last_clean":"2023-07-18T03:38:37.514898+0000","last_became_active":"2023-07-18T03:38:36.525018+0000","last_became_peered":"2023-07-18T03:38:36.525018+0000","last_unstale":"2023-07-18T03:38:37.514898+0000","last_undegraded":"2023-07-18T03:38:37.514898+0000","last_fullsized":"2023-07-18T03:38:37.514898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:47:14.281295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629309+0000","last_change":"2023-07-18T03:38:36.531164+0000","last_active":"2023-07-18T03:38:37.629309+0000","last_peered":"2023-07-18T03:38:37.629309+0000","last_clean":"2023-07-18T03:38:37.629309+0000","last_became_active":"2023-07-18T03:38:36.530907+0000","last_became_peered":"2023-07-18T03:38:36.530907+0000","last_unstale":"2023-07-18T03:38:37.629309+0000","last_undegraded":"2023-07-18T03:38:37.629309+0000","last_fullsized":"2023-07-18T03:38:37.629309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:47:14.281295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516140+0000","last_change":"2023-07-18T03:38:35.529655+0000","last_active":"2023-07-18T03:38:37.516140+0000","last_peered":"2023-07-18T03:38:37.516140+0000","last_clean":"2023-07-18T03:38:37.516140+0000","last_became_active":"2023-07-18T03:38:35.529257+0000","last_became_peered":"2023-07-18T03:38:35.529257+0000","last_unstale":"2023-07-18T03:38:37.516140+0000","last_undegraded":"2023-07-18T03:38:37.516140+0000","last_fullsized":"2023-07-18T03:38:37.516140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:52:51.944585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121222+0000","last_change":"2023-07-18T03:38:36.542322+0000","last_active":"2023-07-18T03:38:38.121222+0000","last_peered":"2023-07-18T03:38:38.121222+0000","last_clean":"2023-07-18T03:38:38.121222+0000","last_became_active":"2023-07-18T03:38:36.542038+0000","last_became_peered":"2023-07-18T03:38:36.542038+0000","last_unstale":"2023-07-18T03:38:38.121222+0000","last_undegraded":"2023-07-18T03:38:38.121222+0000","last_fullsized":"2023-07-18T03:38:38.121222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:33:14.215460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514288+0000","last_change":"2023-07-18T03:38:35.536467+0000","last_active":"2023-07-18T03:38:37.514288+0000","last_peered":"2023-07-18T03:38:37.514288+0000","last_clean":"2023-07-18T03:38:37.514288+0000","last_became_active":"2023-07-18T03:38:35.536198+0000","last_became_peered":"2023-07-18T03:38:35.536198+0000","last_unstale":"2023-07-18T03:38:37.514288+0000","last_undegraded":"2023-07-18T03:38:37.514288+0000","last_fullsized":"2023-07-18T03:38:37.514288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:01.259707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121012+0000","last_change":"2023-07-18T03:38:36.529534+0000","last_active":"2023-07-18T03:38:38.121012+0000","last_peered":"2023-07-18T03:38:38.121012+0000","last_clean":"2023-07-18T03:38:38.121012+0000","last_became_active":"2023-07-18T03:38:36.528958+0000","last_became_peered":"2023-07-18T03:38:36.528958+0000","last_unstale":"2023-07-18T03:38:38.121012+0000","last_undegraded":"2023-07-18T03:38:38.121012+0000","last_fullsized":"2023-07-18T03:38:38.121012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:39.607024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121102+0000","last_change":"2023-07-18T03:38:35.527369+0000","last_active":"2023-07-18T03:38:38.121102+0000","last_peered":"2023-07-18T03:38:38.121102+0000","last_clean":"2023-07-18T03:38:38.121102+0000","last_became_active":"2023-07-18T03:38:35.527091+0000","last_became_peered":"2023-07-18T03:38:35.527091+0000","last_unstale":"2023-07-18T03:38:38.121102+0000","last_undegraded":"2023-07-18T03:38:38.121102+0000","last_fullsized":"2023-07-18T03:38:38.121102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:21:50.290666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121460+0000","last_change":"2023-07-18T03:38:36.528845+0000","last_active":"2023-07-18T03:38:38.121460+0000","last_peered":"2023-07-18T03:38:38.121460+0000","last_clean":"2023-07-18T03:38:38.121460+0000","last_became_active":"2023-07-18T03:38:36.528525+0000","last_became_peered":"2023-07-18T03:38:36.528525+0000","last_unstale":"2023-07-18T03:38:38.121460+0000","last_undegraded":"2023-07-18T03:38:38.121460+0000","last_fullsized":"2023-07-18T03:38:38.121460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:16:39.663691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121461+0000","last_change":"2023-07-18T03:38:35.529837+0000","last_active":"2023-07-18T03:38:38.121461+0000","last_peered":"2023-07-18T03:38:38.121461+0000","last_clean":"2023-07-18T03:38:38.121461+0000","last_became_active":"2023-07-18T03:38:35.529486+0000","last_became_peered":"2023-07-18T03:38:35.529486+0000","last_unstale":"2023-07-18T03:38:38.121461+0000","last_undegraded":"2023-07-18T03:38:38.121461+0000","last_fullsized":"2023-07-18T03:38:38.121461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:20:07.344114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121086+0000","last_change":"2023-07-18T03:38:36.543236+0000","last_active":"2023-07-18T03:38:38.121086+0000","last_peered":"2023-07-18T03:38:38.121086+0000","last_clean":"2023-07-18T03:38:38.121086+0000","last_became_active":"2023-07-18T03:38:36.542960+0000","last_became_peered":"2023-07-18T03:38:36.542960+0000","last_unstale":"2023-07-18T03:38:38.121086+0000","last_undegraded":"2023-07-18T03:38:38.121086+0000","last_fullsized":"2023-07-18T03:38:38.121086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:46.558463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629501+0000","last_change":"2023-07-18T03:38:35.537672+0000","last_active":"2023-07-18T03:38:37.629501+0000","last_peered":"2023-07-18T03:38:37.629501+0000","last_clean":"2023-07-18T03:38:37.629501+0000","last_became_active":"2023-07-18T03:38:35.537362+0000","last_became_peered":"2023-07-18T03:38:35.537362+0000","last_unstale":"2023-07-18T03:38:37.629501+0000","last_undegraded":"2023-07-18T03:38:37.629501+0000","last_fullsized":"2023-07-18T03:38:37.629501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:38:40.610822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121013+0000","last_change":"2023-07-18T03:38:36.533526+0000","last_active":"2023-07-18T03:38:38.121013+0000","last_peered":"2023-07-18T03:38:38.121013+0000","last_clean":"2023-07-18T03:38:38.121013+0000","last_became_active":"2023-07-18T03:38:36.533278+0000","last_became_peered":"2023-07-18T03:38:36.533278+0000","last_unstale":"2023-07-18T03:38:38.121013+0000","last_undegraded":"2023-07-18T03:38:38.121013+0000","last_fullsized":"2023-07-18T03:38:38.121013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:45.182024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516465+0000","last_change":"2023-07-18T03:38:35.536258+0000","last_active":"2023-07-18T03:38:37.516465+0000","last_peered":"2023-07-18T03:38:37.516465+0000","last_clean":"2023-07-18T03:38:37.516465+0000","last_became_active":"2023-07-18T03:38:35.535939+0000","last_became_peered":"2023-07-18T03:38:35.535939+0000","last_unstale":"2023-07-18T03:38:37.516465+0000","last_undegraded":"2023-07-18T03:38:37.516465+0000","last_fullsized":"2023-07-18T03:38:37.516465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:30:23.496663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514229+0000","last_change":"2023-07-18T03:38:36.533896+0000","last_active":"2023-07-18T03:38:37.514229+0000","last_peered":"2023-07-18T03:38:37.514229+0000","last_clean":"2023-07-18T03:38:37.514229+0000","last_became_active":"2023-07-18T03:38:36.532384+0000","last_became_peered":"2023-07-18T03:38:36.532384+0000","last_unstale":"2023-07-18T03:38:37.514229+0000","last_undegraded":"2023-07-18T03:38:37.514229+0000","last_fullsized":"2023-07-18T03:38:37.514229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:13:15.481453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516636+0000","last_change":"2023-07-18T03:38:35.536771+0000","last_active":"2023-07-18T03:38:37.516636+0000","last_peered":"2023-07-18T03:38:37.516636+0000","last_clean":"2023-07-18T03:38:37.516636+0000","last_became_active":"2023-07-18T03:38:35.536423+0000","last_became_peered":"2023-07-18T03:38:35.536423+0000","last_unstale":"2023-07-18T03:38:37.516636+0000","last_undegraded":"2023-07-18T03:38:37.516636+0000","last_fullsized":"2023-07-18T03:38:37.516636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:30:23.496663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516904+0000","last_change":"2023-07-18T03:38:36.531396+0000","last_active":"2023-07-18T03:38:37.516904+0000","last_peered":"2023-07-18T03:38:37.516904+0000","last_clean":"2023-07-18T03:38:37.516904+0000","last_became_active":"2023-07-18T03:38:36.531117+0000","last_became_peered":"2023-07-18T03:38:36.531117+0000","last_unstale":"2023-07-18T03:38:37.516904+0000","last_undegraded":"2023-07-18T03:38:37.516904+0000","last_fullsized":"2023-07-18T03:38:37.516904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:36:15.063131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515828+0000","last_change":"2023-07-18T03:38:35.531528+0000","last_active":"2023-07-18T03:38:37.515828+0000","last_peered":"2023-07-18T03:38:37.515828+0000","last_clean":"2023-07-18T03:38:37.515828+0000","last_became_active":"2023-07-18T03:38:35.530889+0000","last_became_peered":"2023-07-18T03:38:35.530889+0000","last_unstale":"2023-07-18T03:38:37.515828+0000","last_undegraded":"2023-07-18T03:38:37.515828+0000","last_fullsized":"2023-07-18T03:38:37.515828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:52:51.944585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121161+0000","last_change":"2023-07-18T03:38:36.538483+0000","last_active":"2023-07-18T03:38:38.121161+0000","last_peered":"2023-07-18T03:38:38.121161+0000","last_clean":"2023-07-18T03:38:38.121161+0000","last_became_active":"2023-07-18T03:38:36.538133+0000","last_became_peered":"2023-07-18T03:38:36.538133+0000","last_unstale":"2023-07-18T03:38:38.121161+0000","last_undegraded":"2023-07-18T03:38:38.121161+0000","last_fullsized":"2023-07-18T03:38:38.121161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:27:36.142464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120772+0000","last_change":"2023-07-18T03:38:35.518650+0000","last_active":"2023-07-18T03:38:38.120772+0000","last_peered":"2023-07-18T03:38:38.120772+0000","last_clean":"2023-07-18T03:38:38.120772+0000","last_became_active":"2023-07-18T03:38:35.518285+0000","last_became_peered":"2023-07-18T03:38:35.518285+0000","last_unstale":"2023-07-18T03:38:38.120772+0000","last_undegraded":"2023-07-18T03:38:38.120772+0000","last_fullsized":"2023-07-18T03:38:38.120772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:57:59.447632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629401+0000","last_change":"2023-07-18T03:38:36.538615+0000","last_active":"2023-07-18T03:38:37.629401+0000","last_peered":"2023-07-18T03:38:37.629401+0000","last_clean":"2023-07-18T03:38:37.629401+0000","last_became_active":"2023-07-18T03:38:36.538362+0000","last_became_peered":"2023-07-18T03:38:36.538362+0000","last_unstale":"2023-07-18T03:38:37.629401+0000","last_undegraded":"2023-07-18T03:38:37.629401+0000","last_fullsized":"2023-07-18T03:38:37.629401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:14:38.515543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516038+0000","last_change":"2023-07-18T03:38:35.534762+0000","last_active":"2023-07-18T03:38:37.516038+0000","last_peered":"2023-07-18T03:38:37.516038+0000","last_clean":"2023-07-18T03:38:37.516038+0000","last_became_active":"2023-07-18T03:38:35.534529+0000","last_became_peered":"2023-07-18T03:38:35.534529+0000","last_unstale":"2023-07-18T03:38:37.516038+0000","last_undegraded":"2023-07-18T03:38:37.516038+0000","last_fullsized":"2023-07-18T03:38:37.516038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:27:35.134036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120813+0000","last_change":"2023-07-18T03:38:36.530844+0000","last_active":"2023-07-18T03:38:38.120813+0000","last_peered":"2023-07-18T03:38:38.120813+0000","last_clean":"2023-07-18T03:38:38.120813+0000","last_became_active":"2023-07-18T03:38:36.530545+0000","last_became_peered":"2023-07-18T03:38:36.530545+0000","last_unstale":"2023-07-18T03:38:38.120813+0000","last_undegraded":"2023-07-18T03:38:38.120813+0000","last_fullsized":"2023-07-18T03:38:38.120813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:39:26.376467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120972+0000","last_change":"2023-07-18T03:38:35.530210+0000","last_active":"2023-07-18T03:38:38.120972+0000","last_peered":"2023-07-18T03:38:38.120972+0000","last_clean":"2023-07-18T03:38:38.120972+0000","last_became_active":"2023-07-18T03:38:35.529906+0000","last_became_peered":"2023-07-18T03:38:35.529906+0000","last_unstale":"2023-07-18T03:38:38.120972+0000","last_undegraded":"2023-07-18T03:38:38.120972+0000","last_fullsized":"2023-07-18T03:38:38.120972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:59:17.569992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121717+0000","last_change":"2023-07-18T03:38:36.534252+0000","last_active":"2023-07-18T03:38:38.121717+0000","last_peered":"2023-07-18T03:38:38.121717+0000","last_clean":"2023-07-18T03:38:38.121717+0000","last_became_active":"2023-07-18T03:38:36.533986+0000","last_became_peered":"2023-07-18T03:38:36.533986+0000","last_unstale":"2023-07-18T03:38:38.121717+0000","last_undegraded":"2023-07-18T03:38:38.121717+0000","last_fullsized":"2023-07-18T03:38:38.121717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:13:47.285259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516720+0000","last_change":"2023-07-18T03:38:35.530100+0000","last_active":"2023-07-18T03:38:37.516720+0000","last_peered":"2023-07-18T03:38:37.516720+0000","last_clean":"2023-07-18T03:38:37.516720+0000","last_became_active":"2023-07-18T03:38:35.529084+0000","last_became_peered":"2023-07-18T03:38:35.529084+0000","last_unstale":"2023-07-18T03:38:37.516720+0000","last_undegraded":"2023-07-18T03:38:37.516720+0000","last_fullsized":"2023-07-18T03:38:37.516720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:25.804673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629276+0000","last_change":"2023-07-18T03:38:36.531903+0000","last_active":"2023-07-18T03:38:37.629276+0000","last_peered":"2023-07-18T03:38:37.629276+0000","last_clean":"2023-07-18T03:38:37.629276+0000","last_became_active":"2023-07-18T03:38:36.531503+0000","last_became_peered":"2023-07-18T03:38:36.531503+0000","last_unstale":"2023-07-18T03:38:37.629276+0000","last_undegraded":"2023-07-18T03:38:37.629276+0000","last_fullsized":"2023-07-18T03:38:37.629276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:16:39.663691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516435+0000","last_change":"2023-07-18T03:38:35.530991+0000","last_active":"2023-07-18T03:38:37.516435+0000","last_peered":"2023-07-18T03:38:37.516435+0000","last_clean":"2023-07-18T03:38:37.516435+0000","last_became_active":"2023-07-18T03:38:35.530676+0000","last_became_peered":"2023-07-18T03:38:35.530676+0000","last_unstale":"2023-07-18T03:38:37.516435+0000","last_undegraded":"2023-07-18T03:38:37.516435+0000","last_fullsized":"2023-07-18T03:38:37.516435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:43:42.440181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516298+0000","last_change":"2023-07-18T03:38:36.524535+0000","last_active":"2023-07-18T03:38:37.516298+0000","last_peered":"2023-07-18T03:38:37.516298+0000","last_clean":"2023-07-18T03:38:37.516298+0000","last_became_active":"2023-07-18T03:38:36.524235+0000","last_became_peered":"2023-07-18T03:38:36.524235+0000","last_unstale":"2023-07-18T03:38:37.516298+0000","last_undegraded":"2023-07-18T03:38:37.516298+0000","last_fullsized":"2023-07-18T03:38:37.516298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:21:51.299094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516290+0000","last_change":"2023-07-18T03:38:35.532333+0000","last_active":"2023-07-18T03:38:37.516290+0000","last_peered":"2023-07-18T03:38:37.516290+0000","last_clean":"2023-07-18T03:38:37.516290+0000","last_became_active":"2023-07-18T03:38:35.531899+0000","last_became_peered":"2023-07-18T03:38:35.531899+0000","last_unstale":"2023-07-18T03:38:37.516290+0000","last_undegraded":"2023-07-18T03:38:37.516290+0000","last_fullsized":"2023-07-18T03:38:37.516290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:27:42.625518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121237+0000","last_change":"2023-07-18T03:38:36.540597+0000","last_active":"2023-07-18T03:38:38.121237+0000","last_peered":"2023-07-18T03:38:38.121237+0000","last_clean":"2023-07-18T03:38:38.121237+0000","last_became_active":"2023-07-18T03:38:36.540313+0000","last_became_peered":"2023-07-18T03:38:36.540313+0000","last_unstale":"2023-07-18T03:38:38.121237+0000","last_undegraded":"2023-07-18T03:38:38.121237+0000","last_fullsized":"2023-07-18T03:38:38.121237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:02.268135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629372+0000","last_change":"2023-07-18T03:38:35.536145+0000","last_active":"2023-07-18T03:38:37.629372+0000","last_peered":"2023-07-18T03:38:37.629372+0000","last_clean":"2023-07-18T03:38:37.629372+0000","last_became_active":"2023-07-18T03:38:35.535827+0000","last_became_peered":"2023-07-18T03:38:35.535827+0000","last_unstale":"2023-07-18T03:38:37.629372+0000","last_undegraded":"2023-07-18T03:38:37.629372+0000","last_fullsized":"2023-07-18T03:38:37.629372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:44:02.784217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120807+0000","last_change":"2023-07-18T03:38:36.526183+0000","last_active":"2023-07-18T03:38:38.120807+0000","last_peered":"2023-07-18T03:38:38.120807+0000","last_clean":"2023-07-18T03:38:38.120807+0000","last_became_active":"2023-07-18T03:38:36.525853+0000","last_became_peered":"2023-07-18T03:38:36.525853+0000","last_unstale":"2023-07-18T03:38:38.120807+0000","last_undegraded":"2023-07-18T03:38:38.120807+0000","last_fullsized":"2023-07-18T03:38:38.120807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:51:42.726329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121779+0000","last_change":"2023-07-18T03:38:35.534166+0000","last_active":"2023-07-18T03:38:38.121779+0000","last_peered":"2023-07-18T03:38:38.121779+0000","last_clean":"2023-07-18T03:38:38.121779+0000","last_became_active":"2023-07-18T03:38:35.533900+0000","last_became_peered":"2023-07-18T03:38:35.533900+0000","last_unstale":"2023-07-18T03:38:38.121779+0000","last_undegraded":"2023-07-18T03:38:38.121779+0000","last_fullsized":"2023-07-18T03:38:38.121779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:17:15.896043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516546+0000","last_change":"2023-07-18T03:38:36.535669+0000","last_active":"2023-07-18T03:38:37.516546+0000","last_peered":"2023-07-18T03:38:37.516546+0000","last_clean":"2023-07-18T03:38:37.516546+0000","last_became_active":"2023-07-18T03:38:36.535338+0000","last_became_peered":"2023-07-18T03:38:36.535338+0000","last_unstale":"2023-07-18T03:38:37.516546+0000","last_undegraded":"2023-07-18T03:38:37.516546+0000","last_fullsized":"2023-07-18T03:38:37.516546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:43:43.448609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121279+0000","last_change":"2023-07-18T03:38:35.530541+0000","last_active":"2023-07-18T03:38:38.121279+0000","last_peered":"2023-07-18T03:38:38.121279+0000","last_clean":"2023-07-18T03:38:38.121279+0000","last_became_active":"2023-07-18T03:38:35.530220+0000","last_became_peered":"2023-07-18T03:38:35.530220+0000","last_unstale":"2023-07-18T03:38:38.121279+0000","last_undegraded":"2023-07-18T03:38:38.121279+0000","last_fullsized":"2023-07-18T03:38:38.121279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:53:15.570874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516299+0000","last_change":"2023-07-18T03:38:36.523648+0000","last_active":"2023-07-18T03:38:37.516299+0000","last_peered":"2023-07-18T03:38:37.516299+0000","last_clean":"2023-07-18T03:38:37.516299+0000","last_became_active":"2023-07-18T03:38:36.523346+0000","last_became_peered":"2023-07-18T03:38:36.523346+0000","last_unstale":"2023-07-18T03:38:37.516299+0000","last_undegraded":"2023-07-18T03:38:37.516299+0000","last_fullsized":"2023-07-18T03:38:37.516299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:46.558463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516714+0000","last_change":"2023-07-18T03:38:35.534663+0000","last_active":"2023-07-18T03:38:37.516714+0000","last_peered":"2023-07-18T03:38:37.516714+0000","last_clean":"2023-07-18T03:38:37.516714+0000","last_became_active":"2023-07-18T03:38:35.534363+0000","last_became_peered":"2023-07-18T03:38:35.534363+0000","last_unstale":"2023-07-18T03:38:37.516714+0000","last_undegraded":"2023-07-18T03:38:37.516714+0000","last_fullsized":"2023-07-18T03:38:37.516714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:13:46.276831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516932+0000","last_change":"2023-07-18T03:38:36.531919+0000","last_active":"2023-07-18T03:38:37.516932+0000","last_peered":"2023-07-18T03:38:37.516932+0000","last_clean":"2023-07-18T03:38:37.516932+0000","last_became_active":"2023-07-18T03:38:36.531729+0000","last_became_peered":"2023-07-18T03:38:36.531729+0000","last_unstale":"2023-07-18T03:38:37.516932+0000","last_undegraded":"2023-07-18T03:38:37.516932+0000","last_fullsized":"2023-07-18T03:38:37.516932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:02:54.254215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120962+0000","last_change":"2023-07-18T03:38:35.538915+0000","last_active":"2023-07-18T03:38:38.120962+0000","last_peered":"2023-07-18T03:38:38.120962+0000","last_clean":"2023-07-18T03:38:38.120962+0000","last_became_active":"2023-07-18T03:38:35.538539+0000","last_became_peered":"2023-07-18T03:38:35.538539+0000","last_unstale":"2023-07-18T03:38:38.120962+0000","last_undegraded":"2023-07-18T03:38:38.120962+0000","last_fullsized":"2023-07-18T03:38:38.120962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:52:51.944585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515021+0000","last_change":"2023-07-18T03:38:36.534839+0000","last_active":"2023-07-18T03:38:37.515021+0000","last_peered":"2023-07-18T03:38:37.515021+0000","last_clean":"2023-07-18T03:38:37.515021+0000","last_became_active":"2023-07-18T03:38:36.532743+0000","last_became_peered":"2023-07-18T03:38:36.532743+0000","last_unstale":"2023-07-18T03:38:37.515021+0000","last_undegraded":"2023-07-18T03:38:37.515021+0000","last_fullsized":"2023-07-18T03:38:37.515021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:14:38.515543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121583+0000","last_change":"2023-07-18T03:38:35.538083+0000","last_active":"2023-07-18T03:38:38.121583+0000","last_peered":"2023-07-18T03:38:38.121583+0000","last_clean":"2023-07-18T03:38:38.121583+0000","last_became_active":"2023-07-18T03:38:35.537692+0000","last_became_peered":"2023-07-18T03:38:35.537692+0000","last_unstale":"2023-07-18T03:38:38.121583+0000","last_undegraded":"2023-07-18T03:38:38.121583+0000","last_fullsized":"2023-07-18T03:38:38.121583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:25.804673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516238+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:38:37.516238+0000","last_peered":"2023-07-18T03:38:37.516238+0000","last_clean":"2023-07-18T03:38:37.516238+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:38:37.516238+0000","last_undegraded":"2023-07-18T03:38:37.516238+0000","last_fullsized":"2023-07-18T03:38:37.516238+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:38.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120829+0000","last_change":"2023-07-18T03:38:36.531962+0000","last_active":"2023-07-18T03:38:38.120829+0000","last_peered":"2023-07-18T03:38:38.120829+0000","last_clean":"2023-07-18T03:38:38.120829+0000","last_became_active":"2023-07-18T03:38:36.531710+0000","last_became_peered":"2023-07-18T03:38:36.531710+0000","last_unstale":"2023-07-18T03:38:38.120829+0000","last_undegraded":"2023-07-18T03:38:38.120829+0000","last_fullsized":"2023-07-18T03:38:38.120829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:42.081234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121276+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:38:38.121276+0000","last_peered":"2023-07-18T03:38:38.121276+0000","last_clean":"2023-07-18T03:38:38.121276+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:38:38.121276+0000","last_undegraded":"2023-07-18T03:38:38.121276+0000","last_fullsized":"2023-07-18T03:38:38.121276+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:27:36.456272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.198672+0000","last_change":"2023-07-18T03:38:35.522198+0000","last_active":"2023-07-18T03:38:38.198672+0000","last_peered":"2023-07-18T03:38:38.198672+0000","last_clean":"2023-07-18T03:38:38.198672+0000","last_became_active":"2023-07-18T03:38:35.521915+0000","last_became_peered":"2023-07-18T03:38:35.521915+0000","last_unstale":"2023-07-18T03:38:38.198672+0000","last_undegraded":"2023-07-18T03:38:38.198672+0000","last_fullsized":"2023-07-18T03:38:38.198672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:06:13.388319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516480+0000","last_change":"2023-07-18T03:38:36.537103+0000","last_active":"2023-07-18T03:38:37.516480+0000","last_peered":"2023-07-18T03:38:37.516480+0000","last_clean":"2023-07-18T03:38:37.516480+0000","last_became_active":"2023-07-18T03:38:36.534926+0000","last_became_peered":"2023-07-18T03:38:36.534926+0000","last_unstale":"2023-07-18T03:38:37.516480+0000","last_undegraded":"2023-07-18T03:38:37.516480+0000","last_fullsized":"2023-07-18T03:38:37.516480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:33:14.215460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514498+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:38:37.514498+0000","last_peered":"2023-07-18T03:38:37.514498+0000","last_clean":"2023-07-18T03:38:37.514498+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:38:37.514498+0000","last_undegraded":"2023-07-18T03:38:37.514498+0000","last_fullsized":"2023-07-18T03:38:37.514498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:31.720395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515826+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:38:37.515826+0000","last_peered":"2023-07-18T03:38:37.515826+0000","last_clean":"2023-07-18T03:38:37.515826+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:38:37.515826+0000","last_undegraded":"2023-07-18T03:38:37.515826+0000","last_fullsized":"2023-07-18T03:38:37.515826+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:38.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120579+0000","last_change":"2023-07-18T03:38:35.534781+0000","last_active":"2023-07-18T03:38:38.120579+0000","last_peered":"2023-07-18T03:38:38.120579+0000","last_clean":"2023-07-18T03:38:38.120579+0000","last_became_active":"2023-07-18T03:38:35.534485+0000","last_became_peered":"2023-07-18T03:38:35.534485+0000","last_unstale":"2023-07-18T03:38:38.120579+0000","last_undegraded":"2023-07-18T03:38:38.120579+0000","last_fullsized":"2023-07-18T03:38:38.120579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:42:36.215812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120857+0000","last_change":"2023-07-18T03:38:36.527606+0000","last_active":"2023-07-18T03:38:38.120857+0000","last_peered":"2023-07-18T03:38:38.120857+0000","last_clean":"2023-07-18T03:38:38.120857+0000","last_became_active":"2023-07-18T03:38:36.527267+0000","last_became_peered":"2023-07-18T03:38:36.527267+0000","last_unstale":"2023-07-18T03:38:38.120857+0000","last_undegraded":"2023-07-18T03:38:38.120857+0000","last_fullsized":"2023-07-18T03:38:38.120857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:32:24.221027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515516+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:38:37.515516+0000","last_peered":"2023-07-18T03:38:37.515516+0000","last_clean":"2023-07-18T03:38:37.515516+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:38:37.515516+0000","last_undegraded":"2023-07-18T03:38:37.515516+0000","last_fullsized":"2023-07-18T03:38:37.515516+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:02.380287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514037+0000","last_change":"2023-07-18T03:38:35.531556+0000","last_active":"2023-07-18T03:38:37.514037+0000","last_peered":"2023-07-18T03:38:37.514037+0000","last_clean":"2023-07-18T03:38:37.514037+0000","last_became_active":"2023-07-18T03:38:35.531268+0000","last_became_peered":"2023-07-18T03:38:35.531268+0000","last_unstale":"2023-07-18T03:38:37.514037+0000","last_undegraded":"2023-07-18T03:38:37.514037+0000","last_fullsized":"2023-07-18T03:38:37.514037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:59:17.569992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514994+0000","last_change":"2023-07-18T03:38:36.536571+0000","last_active":"2023-07-18T03:38:37.514994+0000","last_peered":"2023-07-18T03:38:37.514994+0000","last_clean":"2023-07-18T03:38:37.514994+0000","last_became_active":"2023-07-18T03:38:36.536187+0000","last_became_peered":"2023-07-18T03:38:36.536187+0000","last_unstale":"2023-07-18T03:38:37.514994+0000","last_undegraded":"2023-07-18T03:38:37.514994+0000","last_fullsized":"2023-07-18T03:38:37.514994+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:44:03.792645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515534+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:38:37.515534+0000","last_peered":"2023-07-18T03:38:37.515534+0000","last_clean":"2023-07-18T03:38:37.515534+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:38:37.515534+0000","last_undegraded":"2023-07-18T03:38:37.515534+0000","last_fullsized":"2023-07-18T03:38:37.515534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:38:34.441575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121349+0000","last_change":"2023-07-18T03:38:35.536386+0000","last_active":"2023-07-18T03:38:38.121349+0000","last_peered":"2023-07-18T03:38:38.121349+0000","last_clean":"2023-07-18T03:38:38.121349+0000","last_became_active":"2023-07-18T03:38:35.536066+0000","last_became_peered":"2023-07-18T03:38:35.536066+0000","last_unstale":"2023-07-18T03:38:38.121349+0000","last_undegraded":"2023-07-18T03:38:38.121349+0000","last_fullsized":"2023-07-18T03:38:38.121349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:43:42.440181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121258+0000","last_change":"2023-07-18T03:38:36.528088+0000","last_active":"2023-07-18T03:38:38.121258+0000","last_peered":"2023-07-18T03:38:38.121258+0000","last_clean":"2023-07-18T03:38:38.121258+0000","last_became_active":"2023-07-18T03:38:36.527760+0000","last_became_peered":"2023-07-18T03:38:36.527760+0000","last_unstale":"2023-07-18T03:38:38.121258+0000","last_undegraded":"2023-07-18T03:38:38.121258+0000","last_fullsized":"2023-07-18T03:38:38.121258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:38.898070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629222+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:38:37.629222+0000","last_peered":"2023-07-18T03:38:37.629222+0000","last_clean":"2023-07-18T03:38:37.629222+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:38:37.629222+0000","last_undegraded":"2023-07-18T03:38:37.629222+0000","last_fullsized":"2023-07-18T03:38:37.629222+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:50:38.894203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.517262+0000","last_change":"2023-07-18T03:38:35.526465+0000","last_active":"2023-07-18T03:38:37.517262+0000","last_peered":"2023-07-18T03:38:37.517262+0000","last_clean":"2023-07-18T03:38:37.517262+0000","last_became_active":"2023-07-18T03:38:35.525759+0000","last_became_peered":"2023-07-18T03:38:35.525759+0000","last_unstale":"2023-07-18T03:38:37.517262+0000","last_undegraded":"2023-07-18T03:38:37.517262+0000","last_fullsized":"2023-07-18T03:38:37.517262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:56:57.134347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514887+0000","last_change":"2023-07-18T03:38:36.523849+0000","last_active":"2023-07-18T03:38:37.514887+0000","last_peered":"2023-07-18T03:38:37.514887+0000","last_clean":"2023-07-18T03:38:37.514887+0000","last_became_active":"2023-07-18T03:38:36.523555+0000","last_became_peered":"2023-07-18T03:38:36.523555+0000","last_unstale":"2023-07-18T03:38:37.514887+0000","last_undegraded":"2023-07-18T03:38:37.514887+0000","last_fullsized":"2023-07-18T03:38:37.514887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:16:39.663691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121324+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:38:38.121324+0000","last_peered":"2023-07-18T03:38:38.121324+0000","last_clean":"2023-07-18T03:38:38.121324+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:38:38.121324+0000","last_undegraded":"2023-07-18T03:38:38.121324+0000","last_fullsized":"2023-07-18T03:38:38.121324+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:50:38.894203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514759+0000","last_change":"2023-07-18T03:38:35.530710+0000","last_active":"2023-07-18T03:38:37.514759+0000","last_peered":"2023-07-18T03:38:37.514759+0000","last_clean":"2023-07-18T03:38:37.514759+0000","last_became_active":"2023-07-18T03:38:35.530463+0000","last_became_peered":"2023-07-18T03:38:35.530463+0000","last_unstale":"2023-07-18T03:38:37.514759+0000","last_undegraded":"2023-07-18T03:38:37.514759+0000","last_fullsized":"2023-07-18T03:38:37.514759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:50:45.063449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121163+0000","last_change":"2023-07-18T03:38:36.535754+0000","last_active":"2023-07-18T03:38:38.121163+0000","last_peered":"2023-07-18T03:38:38.121163+0000","last_clean":"2023-07-18T03:38:38.121163+0000","last_became_active":"2023-07-18T03:38:36.535290+0000","last_became_peered":"2023-07-18T03:38:36.535290+0000","last_unstale":"2023-07-18T03:38:38.121163+0000","last_undegraded":"2023-07-18T03:38:38.121163+0000","last_fullsized":"2023-07-18T03:38:38.121163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:03.011261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121317+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:38:38.121317+0000","last_peered":"2023-07-18T03:38:38.121317+0000","last_clean":"2023-07-18T03:38:38.121317+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:38:38.121317+0000","last_undegraded":"2023-07-18T03:38:38.121317+0000","last_fullsized":"2023-07-18T03:38:38.121317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:42:57.376881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629589+0000","last_change":"2023-07-18T03:38:35.536982+0000","last_active":"2023-07-18T03:38:37.629589+0000","last_peered":"2023-07-18T03:38:37.629589+0000","last_clean":"2023-07-18T03:38:37.629589+0000","last_became_active":"2023-07-18T03:38:35.536739+0000","last_became_peered":"2023-07-18T03:38:35.536739+0000","last_unstale":"2023-07-18T03:38:37.629589+0000","last_undegraded":"2023-07-18T03:38:37.629589+0000","last_fullsized":"2023-07-18T03:38:37.629589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:52:51.944585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514520+0000","last_change":"2023-07-18T03:38:36.532972+0000","last_active":"2023-07-18T03:38:37.514520+0000","last_peered":"2023-07-18T03:38:37.514520+0000","last_clean":"2023-07-18T03:38:37.514520+0000","last_became_active":"2023-07-18T03:38:36.530512+0000","last_became_peered":"2023-07-18T03:38:36.530512+0000","last_unstale":"2023-07-18T03:38:37.514520+0000","last_undegraded":"2023-07-18T03:38:37.514520+0000","last_fullsized":"2023-07-18T03:38:37.514520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:53:16.579302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121293+0000","last_change":"2023-07-18T03:38:35.525179+0000","last_active":"2023-07-18T03:38:38.121293+0000","last_peered":"2023-07-18T03:38:38.121293+0000","last_clean":"2023-07-18T03:38:38.121293+0000","last_became_active":"2023-07-18T03:38:35.524912+0000","last_became_peered":"2023-07-18T03:38:35.524912+0000","last_unstale":"2023-07-18T03:38:38.121293+0000","last_undegraded":"2023-07-18T03:38:38.121293+0000","last_fullsized":"2023-07-18T03:38:38.121293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:23:28.704920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.122118+0000","last_change":"2023-07-18T03:38:36.529814+0000","last_active":"2023-07-18T03:38:38.122118+0000","last_peered":"2023-07-18T03:38:38.122118+0000","last_clean":"2023-07-18T03:38:38.122118+0000","last_became_active":"2023-07-18T03:38:36.529554+0000","last_became_peered":"2023-07-18T03:38:36.529554+0000","last_unstale":"2023-07-18T03:38:38.122118+0000","last_undegraded":"2023-07-18T03:38:38.122118+0000","last_fullsized":"2023-07-18T03:38:38.122118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:30:55.302127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629197+0000","last_change":"2023-07-18T03:38:35.529892+0000","last_active":"2023-07-18T03:38:37.629197+0000","last_peered":"2023-07-18T03:38:37.629197+0000","last_clean":"2023-07-18T03:38:37.629197+0000","last_became_active":"2023-07-18T03:38:35.529614+0000","last_became_peered":"2023-07-18T03:38:35.529614+0000","last_unstale":"2023-07-18T03:38:37.629197+0000","last_undegraded":"2023-07-18T03:38:37.629197+0000","last_fullsized":"2023-07-18T03:38:37.629197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:53:15.570874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.517120+0000","last_change":"2023-07-18T03:38:36.535652+0000","last_active":"2023-07-18T03:38:37.517120+0000","last_peered":"2023-07-18T03:38:37.517120+0000","last_clean":"2023-07-18T03:38:37.517120+0000","last_became_active":"2023-07-18T03:38:36.535378+0000","last_became_peered":"2023-07-18T03:38:36.535378+0000","last_unstale":"2023-07-18T03:38:37.517120+0000","last_undegraded":"2023-07-18T03:38:37.517120+0000","last_fullsized":"2023-07-18T03:38:37.517120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:20.087591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629487+0000","last_change":"2023-07-18T03:38:35.534949+0000","last_active":"2023-07-18T03:38:37.629487+0000","last_peered":"2023-07-18T03:38:37.629487+0000","last_clean":"2023-07-18T03:38:37.629487+0000","last_became_active":"2023-07-18T03:38:35.534685+0000","last_became_peered":"2023-07-18T03:38:35.534685+0000","last_unstale":"2023-07-18T03:38:37.629487+0000","last_undegraded":"2023-07-18T03:38:37.629487+0000","last_fullsized":"2023-07-18T03:38:37.629487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:08.549533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516411+0000","last_change":"2023-07-18T03:38:36.535757+0000","last_active":"2023-07-18T03:38:37.516411+0000","last_peered":"2023-07-18T03:38:37.516411+0000","last_clean":"2023-07-18T03:38:37.516411+0000","last_became_active":"2023-07-18T03:38:36.535452+0000","last_became_peered":"2023-07-18T03:38:36.535452+0000","last_unstale":"2023-07-18T03:38:37.516411+0000","last_undegraded":"2023-07-18T03:38:37.516411+0000","last_fullsized":"2023-07-18T03:38:37.516411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:27:36.142464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121576+0000","last_change":"2023-07-18T03:38:35.532954+0000","last_active":"2023-07-18T03:38:38.121576+0000","last_peered":"2023-07-18T03:38:38.121576+0000","last_clean":"2023-07-18T03:38:38.121576+0000","last_became_active":"2023-07-18T03:38:35.532617+0000","last_became_peered":"2023-07-18T03:38:35.532617+0000","last_unstale":"2023-07-18T03:38:38.121576+0000","last_undegraded":"2023-07-18T03:38:38.121576+0000","last_fullsized":"2023-07-18T03:38:38.121576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:30:23.496663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120577+0000","last_change":"2023-07-18T03:38:36.540001+0000","last_active":"2023-07-18T03:38:38.120577+0000","last_peered":"2023-07-18T03:38:38.120577+0000","last_clean":"2023-07-18T03:38:38.120577+0000","last_became_active":"2023-07-18T03:38:36.539743+0000","last_became_peered":"2023-07-18T03:38:36.539743+0000","last_unstale":"2023-07-18T03:38:38.120577+0000","last_undegraded":"2023-07-18T03:38:38.120577+0000","last_fullsized":"2023-07-18T03:38:38.120577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:38:41.619250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121427+0000","last_change":"2023-07-18T03:38:35.532342+0000","last_active":"2023-07-18T03:38:38.121427+0000","last_peered":"2023-07-18T03:38:38.121427+0000","last_clean":"2023-07-18T03:38:38.121427+0000","last_became_active":"2023-07-18T03:38:35.530138+0000","last_became_peered":"2023-07-18T03:38:35.530138+0000","last_unstale":"2023-07-18T03:38:38.121427+0000","last_undegraded":"2023-07-18T03:38:38.121427+0000","last_fullsized":"2023-07-18T03:38:38.121427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:59:26.646208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516677+0000","last_change":"2023-07-18T03:38:36.531861+0000","last_active":"2023-07-18T03:38:37.516677+0000","last_peered":"2023-07-18T03:38:37.516677+0000","last_clean":"2023-07-18T03:38:37.516677+0000","last_became_active":"2023-07-18T03:38:36.531563+0000","last_became_peered":"2023-07-18T03:38:36.531563+0000","last_unstale":"2023-07-18T03:38:37.516677+0000","last_undegraded":"2023-07-18T03:38:37.516677+0000","last_fullsized":"2023-07-18T03:38:37.516677+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:26.813102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121209+0000","last_change":"2023-07-18T03:38:36.537688+0000","last_active":"2023-07-18T03:38:38.121209+0000","last_peered":"2023-07-18T03:38:38.121209+0000","last_clean":"2023-07-18T03:38:38.121209+0000","last_became_active":"2023-07-18T03:38:36.537415+0000","last_became_peered":"2023-07-18T03:38:36.537415+0000","last_unstale":"2023-07-18T03:38:38.121209+0000","last_undegraded":"2023-07-18T03:38:38.121209+0000","last_fullsized":"2023-07-18T03:38:38.121209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:13:15.481453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120567+0000","last_change":"2023-07-18T03:38:36.542378+0000","last_active":"2023-07-18T03:38:38.120567+0000","last_peered":"2023-07-18T03:38:38.120567+0000","last_clean":"2023-07-18T03:38:38.120567+0000","last_became_active":"2023-07-18T03:38:36.542101+0000","last_became_peered":"2023-07-18T03:38:36.542101+0000","last_unstale":"2023-07-18T03:38:38.120567+0000","last_undegraded":"2023-07-18T03:38:38.120567+0000","last_fullsized":"2023-07-18T03:38:38.120567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:09.557961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515970+0000","last_change":"2023-07-18T03:38:36.531395+0000","last_active":"2023-07-18T03:38:37.515970+0000","last_peered":"2023-07-18T03:38:37.515970+0000","last_clean":"2023-07-18T03:38:37.515970+0000","last_became_active":"2023-07-18T03:38:36.531071+0000","last_became_peered":"2023-07-18T03:38:36.531071+0000","last_unstale":"2023-07-18T03:38:37.515970+0000","last_undegraded":"2023-07-18T03:38:37.515970+0000","last_fullsized":"2023-07-18T03:38:37.515970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:38:41.619250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121823+0000","last_change":"2023-07-18T03:38:36.534035+0000","last_active":"2023-07-18T03:38:38.121823+0000","last_peered":"2023-07-18T03:38:38.121823+0000","last_clean":"2023-07-18T03:38:38.121823+0000","last_became_active":"2023-07-18T03:38:36.533755+0000","last_became_peered":"2023-07-18T03:38:36.533755+0000","last_unstale":"2023-07-18T03:38:38.121823+0000","last_undegraded":"2023-07-18T03:38:38.121823+0000","last_fullsized":"2023-07-18T03:38:38.121823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:50:17.597467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514727+0000","last_change":"2023-07-18T03:38:36.524624+0000","last_active":"2023-07-18T03:38:37.514727+0000","last_peered":"2023-07-18T03:38:37.514727+0000","last_clean":"2023-07-18T03:38:37.514727+0000","last_became_active":"2023-07-18T03:38:36.524106+0000","last_became_peered":"2023-07-18T03:38:36.524106+0000","last_unstale":"2023-07-18T03:38:37.514727+0000","last_undegraded":"2023-07-18T03:38:37.514727+0000","last_fullsized":"2023-07-18T03:38:37.514727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:04.554555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":843776,"data_stored":673348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":81200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31268,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340572,"statfs":{"total":96636764160,"available":96604745728,"internally_reserved":0,"allocated":225280,"data_stored":81290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":815104,"data_stored":671568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":82820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":81852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":81782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31264,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340576,"statfs":{"total":96636764160,"available":96604749824,"internally_reserved":0,"allocated":221184,"data_stored":81200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:38:46.892 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:38:47.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.134+0000 7f4b093ff700 1 Processor -- start 2023-07-18T03:38:47.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.135+0000 7f4b093ff700 1 -- start start 2023-07-18T03:38:47.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.136+0000 7f4b093ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4b040d1220 0x7f4b040b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:47.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.136+0000 7f4b093ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b040d2ce0 con 0x7f4b040b1e00 2023-07-18T03:38:47.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.136+0000 7f4b093ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b040b3350 con 0x7f4b0415c950 2023-07-18T03:38:47.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.136+0000 7f4b093ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b040b3630 con 0x7f4b040d1220 2023-07-18T03:38:47.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.136+0000 7f4b037fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4b040b1e00 0x7f4b040afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41680/0 (socket says 172.21.15.49:41680) 2023-07-18T03:38:47.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.136+0000 7f4b037fe700 1 -- 172.21.15.49:0/1863347794 learned_addr learned my addr 172.21.15.49:0/1863347794 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:47.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.136+0000 7f4b027fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4b040d1220 0x7f4b040b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:47.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.137+0000 7f4b01ffb700 1 -- 172.21.15.49:0/1863347794 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492876949 0 0) 0x7f4b040b3350 con 0x7f4b0415c950 2023-07-18T03:38:47.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.137+0000 7f4b01ffb700 1 -- 172.21.15.49:0/1863347794 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aec0035b0 con 0x7f4b0415c950 2023-07-18T03:38:47.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.137+0000 7f4b027fc700 1 -- 172.21.15.49:0/1863347794 >> v1:172.21.15.107:6789/0 conn(0x7f4b0415c950 legacy=0x7f4b040d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.137+0000 7f4b027fc700 1 -- 172.21.15.49:0/1863347794 >> v1:172.21.15.49:6789/0 conn(0x7f4b040b1e00 legacy=0x7f4b040afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.137+0000 7f4b01ffb700 1 -- 172.21.15.49:0/1863347794 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454485137 0 0) 0x7f4b040d2ce0 con 0x7f4b040b1e00 2023-07-18T03:38:47.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.137+0000 7f4b027fc700 1 -- 172.21.15.49:0/1863347794 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b040b3910 con 0x7f4b040d1220 2023-07-18T03:38:47.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.137+0000 7f4b027fc700 1 --2- 172.21.15.49:0/1863347794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040d1220 0x7f4b040b43f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009840 tx=0x7f4af8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b7ad9102ebb6b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:47.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.137+0000 7f4b01ffb700 1 -- 172.21.15.49:0/1863347794 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8011830 con 0x7f4b040d1220 2023-07-18T03:38:47.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.137+0000 7f4b01ffb700 1 -- 172.21.15.49:0/1863347794 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af8011990 con 0x7f4b040d1220 2023-07-18T03:38:47.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.137+0000 7f4b01ffb700 1 -- 172.21.15.49:0/1863347794 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8011d20 con 0x7f4b040d1220 2023-07-18T03:38:47.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.138+0000 7f4b093ff700 1 -- 172.21.15.49:0/1863347794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040d1220 msgr2=0x7f4b040b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.138+0000 7f4b093ff700 1 --2- 172.21.15.49:0/1863347794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040d1220 0x7f4b040b43f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009840 tx=0x7f4af8009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:47.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.138+0000 7f4b093ff700 1 -- 172.21.15.49:0/1863347794 shutdown_connections 2023-07-18T03:38:47.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.138+0000 7f4b093ff700 1 --2- 172.21.15.49:0/1863347794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040d1220 0x7f4b040b43f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.138+0000 7f4b093ff700 1 -- 172.21.15.49:0/1863347794 >> 172.21.15.49:0/1863347794 conn(0x7f4b0409c1d0 msgr2=0x7f4b040d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:47.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.138+0000 7f4b093ff700 1 -- 172.21.15.49:0/1863347794 shutdown_connections 2023-07-18T03:38:47.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.138+0000 7f4b093ff700 1 -- 172.21.15.49:0/1863347794 wait complete. 2023-07-18T03:38:47.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.140+0000 7f4b093ff700 1 Processor -- start 2023-07-18T03:38:47.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.140+0000 7f4b093ff700 1 -- start start 2023-07-18T03:38:47.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b093ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b1e00 0x7f4b040cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:47.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b093ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b040d1220 0x7f4b040c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:47.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b093ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0415c950 0x7f4b041f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:47.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b093ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b040d03c0 con 0x7f4b040b1e00 2023-07-18T03:38:47.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b093ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b040c9840 con 0x7f4b0415c950 2023-07-18T03:38:47.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b093ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b040cfe00 con 0x7f4b040d1220 2023-07-18T03:38:47.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b02ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b1e00 0x7f4b040cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:47.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b02ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b1e00 0x7f4b040cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40666/0 (socket says 172.21.15.49:40666) 2023-07-18T03:38:47.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b02ffd700 1 -- 172.21.15.49:0/2056295366 learned_addr learned my addr 172.21.15.49:0/2056295366 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:47.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b037fe700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0415c950 0x7f4b041f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:47.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b027fc700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b040d1220 0x7f4b040c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:47.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b02ffd700 1 -- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b040d1220 msgr2=0x7f4b040c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b02ffd700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b040d1220 0x7f4b040c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b02ffd700 1 -- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0415c950 msgr2=0x7f4b041f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b02ffd700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0415c950 0x7f4b041f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b02ffd700 1 -- 172.21.15.49:0/2056295366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af80094a0 con 0x7f4b040b1e00 2023-07-18T03:38:47.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b027fc700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b040d1220 0x7f4b040c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:47.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.141+0000 7f4b037fe700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0415c950 0x7f4b041f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:47.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.142+0000 7f4b02ffd700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b1e00 0x7f4b040cf770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4af400c410 tx=0x7f4af400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:47.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.142+0000 7f4aeb7fe700 1 -- 172.21.15.49:0/2056295366 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af40052d0 con 0x7f4b040b1e00 2023-07-18T03:38:47.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.142+0000 7f4aeb7fe700 1 -- 172.21.15.49:0/2056295366 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af4005450 con 0x7f4b040b1e00 2023-07-18T03:38:47.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.142+0000 7f4aeb7fe700 1 -- 172.21.15.49:0/2056295366 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4003910 con 0x7f4b040b1e00 2023-07-18T03:38:47.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.142+0000 7f4b093ff700 1 -- 172.21.15.49:0/2056295366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b040d00e0 con 0x7f4b040b1e00 2023-07-18T03:38:47.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.142+0000 7f4b093ff700 1 -- 172.21.15.49:0/2056295366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b040b6180 con 0x7f4b040b1e00 2023-07-18T03:38:47.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.143+0000 7f4b093ff700 1 -- 172.21.15.49:0/2056295366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0404e280 con 0x7f4b040b1e00 2023-07-18T03:38:47.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.144+0000 7f4aeb7fe700 1 -- 172.21.15.49:0/2056295366 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4af4003a70 con 0x7f4b040b1e00 2023-07-18T03:38:47.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.145+0000 7f4aeb7fe700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4af006ace0 0x7f4af006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:47.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.145+0000 7f4aeb7fe700 1 -- 172.21.15.49:0/2056295366 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4af4088910 con 0x7f4b040b1e00 2023-07-18T03:38:47.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.146+0000 7f4b027fc700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4af006ace0 0x7f4af006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:47.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.146+0000 7f4b027fc700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4af006ace0 0x7f4af006d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4b040d1610 tx=0x7f4af801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:47.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.150+0000 7f4aeb7fe700 1 -- 172.21.15.49:0/2056295366 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4af4053b10 con 0x7f4b040b1e00 2023-07-18T03:38:47.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.353+0000 7f4b093ff700 1 -- 172.21.15.49:0/2056295366 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b040b3ed0 con 0x7f4af006ace0 2023-07-18T03:38:47.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.362+0000 7f4aeb7fe700 1 -- 172.21.15.49:0/2056295366 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286867 (secure 0 0 0) 0x7f4af8011b00 con 0x7f4af006ace0 2023-07-18T03:38:47.360 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.366+0000 7f4b093ff700 1 -- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4af006ace0 msgr2=0x7f4af006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.366+0000 7f4b093ff700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4af006ace0 0x7f4af006d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4b040d1610 tx=0x7f4af801c040 comp rx=0 tx=0).stop 2023-07-18T03:38:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.366+0000 7f4b093ff700 1 -- 172.21.15.49:0/2056295366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b1e00 msgr2=0x7f4b040cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.366+0000 7f4b093ff700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b1e00 0x7f4b040cf770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4af400c410 tx=0x7f4af400c7d0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.366+0000 7f4b093ff700 1 -- 172.21.15.49:0/2056295366 shutdown_connections 2023-07-18T03:38:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.366+0000 7f4b093ff700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b0415c950 0x7f4b041f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.366+0000 7f4b093ff700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4af006ace0 0x7f4af006d190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.366+0000 7f4b093ff700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b1e00 0x7f4b040cf770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.366+0000 7f4b093ff700 1 --2- 172.21.15.49:0/2056295366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b040d1220 0x7f4b040c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.366+0000 7f4b093ff700 1 -- 172.21.15.49:0/2056295366 >> 172.21.15.49:0/2056295366 conn(0x7f4b0409c1d0 msgr2=0x7f4b040d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.366+0000 7f4b093ff700 1 -- 172.21.15.49:0/2056295366 shutdown_connections 2023-07-18T03:38:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.366+0000 7f4b093ff700 1 -- 172.21.15.49:0/2056295366 wait complete. 2023-07-18T03:38:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:38:47.379 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-07-18T03:38:46.099132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230856,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":227840,"kb_avail":754743864,"statfs":{"total":773094113280,"available":772857716736,"internally_reserved":0,"allocated":3022848,"data_stored":1835060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":233308160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.588613"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.122105+0000","last_change":"2023-07-18T03:38:36.538478+0000","last_active":"2023-07-18T03:38:38.122105+0000","last_peered":"2023-07-18T03:38:38.122105+0000","last_clean":"2023-07-18T03:38:38.122105+0000","last_became_active":"2023-07-18T03:38:36.538174+0000","last_became_peered":"2023-07-18T03:38:36.538174+0000","last_unstale":"2023-07-18T03:38:38.122105+0000","last_undegraded":"2023-07-18T03:38:38.122105+0000","last_fullsized":"2023-07-18T03:38:38.122105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:10:09.118957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629549+0000","last_change":"2023-07-18T03:38:36.538553+0000","last_active":"2023-07-18T03:38:37.629549+0000","last_peered":"2023-07-18T03:38:37.629549+0000","last_clean":"2023-07-18T03:38:37.629549+0000","last_became_active":"2023-07-18T03:38:36.538257+0000","last_became_peered":"2023-07-18T03:38:36.538257+0000","last_unstale":"2023-07-18T03:38:37.629549+0000","last_undegraded":"2023-07-18T03:38:37.629549+0000","last_fullsized":"2023-07-18T03:38:37.629549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:42.081234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515821+0000","last_change":"2023-07-18T03:38:36.526262+0000","last_active":"2023-07-18T03:38:37.515821+0000","last_peered":"2023-07-18T03:38:37.515821+0000","last_clean":"2023-07-18T03:38:37.515821+0000","last_became_active":"2023-07-18T03:38:36.525920+0000","last_became_peered":"2023-07-18T03:38:36.525920+0000","last_unstale":"2023-07-18T03:38:37.515821+0000","last_undegraded":"2023-07-18T03:38:37.515821+0000","last_fullsized":"2023-07-18T03:38:37.515821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:09.557961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121429+0000","last_change":"2023-07-18T03:38:36.535992+0000","last_active":"2023-07-18T03:38:38.121429+0000","last_peered":"2023-07-18T03:38:38.121429+0000","last_clean":"2023-07-18T03:38:38.121429+0000","last_became_active":"2023-07-18T03:38:36.535587+0000","last_became_peered":"2023-07-18T03:38:36.535587+0000","last_unstale":"2023-07-18T03:38:38.121429+0000","last_undegraded":"2023-07-18T03:38:38.121429+0000","last_fullsized":"2023-07-18T03:38:38.121429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:24.505474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121029+0000","last_change":"2023-07-18T03:38:36.535524+0000","last_active":"2023-07-18T03:38:38.121029+0000","last_peered":"2023-07-18T03:38:38.121029+0000","last_clean":"2023-07-18T03:38:38.121029+0000","last_became_active":"2023-07-18T03:38:36.534975+0000","last_became_peered":"2023-07-18T03:38:36.534975+0000","last_unstale":"2023-07-18T03:38:38.121029+0000","last_undegraded":"2023-07-18T03:38:38.121029+0000","last_fullsized":"2023-07-18T03:38:38.121029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:37:55.979750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515789+0000","last_change":"2023-07-18T03:38:36.526306+0000","last_active":"2023-07-18T03:38:37.515789+0000","last_peered":"2023-07-18T03:38:37.515789+0000","last_clean":"2023-07-18T03:38:37.515789+0000","last_became_active":"2023-07-18T03:38:36.525982+0000","last_became_peered":"2023-07-18T03:38:36.525982+0000","last_unstale":"2023-07-18T03:38:37.515789+0000","last_undegraded":"2023-07-18T03:38:37.515789+0000","last_fullsized":"2023-07-18T03:38:37.515789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:14:38.515543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516000+0000","last_change":"2023-07-18T03:38:36.527124+0000","last_active":"2023-07-18T03:38:37.516000+0000","last_peered":"2023-07-18T03:38:37.516000+0000","last_clean":"2023-07-18T03:38:37.516000+0000","last_became_active":"2023-07-18T03:38:36.526827+0000","last_became_peered":"2023-07-18T03:38:36.526827+0000","last_unstale":"2023-07-18T03:38:37.516000+0000","last_undegraded":"2023-07-18T03:38:37.516000+0000","last_fullsized":"2023-07-18T03:38:37.516000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:21:51.299094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629188+0000","last_change":"2023-07-18T03:38:36.529550+0000","last_active":"2023-07-18T03:38:37.629188+0000","last_peered":"2023-07-18T03:38:37.629188+0000","last_clean":"2023-07-18T03:38:37.629188+0000","last_became_active":"2023-07-18T03:38:36.529261+0000","last_became_peered":"2023-07-18T03:38:36.529261+0000","last_unstale":"2023-07-18T03:38:37.629188+0000","last_undegraded":"2023-07-18T03:38:37.629188+0000","last_fullsized":"2023-07-18T03:38:37.629188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:04.554555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121235+0000","last_change":"2023-07-18T03:38:36.534909+0000","last_active":"2023-07-18T03:38:38.121235+0000","last_peered":"2023-07-18T03:38:38.121235+0000","last_clean":"2023-07-18T03:38:38.121235+0000","last_became_active":"2023-07-18T03:38:36.534593+0000","last_became_peered":"2023-07-18T03:38:36.534593+0000","last_unstale":"2023-07-18T03:38:38.121235+0000","last_undegraded":"2023-07-18T03:38:38.121235+0000","last_fullsized":"2023-07-18T03:38:38.121235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:45:51.736824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629684+0000","last_change":"2023-07-18T03:38:36.536965+0000","last_active":"2023-07-18T03:38:37.629684+0000","last_peered":"2023-07-18T03:38:37.629684+0000","last_clean":"2023-07-18T03:38:37.629684+0000","last_became_active":"2023-07-18T03:38:36.536665+0000","last_became_peered":"2023-07-18T03:38:36.536665+0000","last_unstale":"2023-07-18T03:38:37.629684+0000","last_undegraded":"2023-07-18T03:38:37.629684+0000","last_fullsized":"2023-07-18T03:38:37.629684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:45.182024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515971+0000","last_change":"2023-07-18T03:38:36.526499+0000","last_active":"2023-07-18T03:38:37.515971+0000","last_peered":"2023-07-18T03:38:37.515971+0000","last_clean":"2023-07-18T03:38:37.515971+0000","last_became_active":"2023-07-18T03:38:36.525871+0000","last_became_peered":"2023-07-18T03:38:36.525871+0000","last_unstale":"2023-07-18T03:38:37.515971+0000","last_undegraded":"2023-07-18T03:38:37.515971+0000","last_fullsized":"2023-07-18T03:38:37.515971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:42.081234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121914+0000","last_change":"2023-07-18T03:38:36.524528+0000","last_active":"2023-07-18T03:38:38.121914+0000","last_peered":"2023-07-18T03:38:38.121914+0000","last_clean":"2023-07-18T03:38:38.121914+0000","last_became_active":"2023-07-18T03:38:36.523892+0000","last_became_peered":"2023-07-18T03:38:36.523892+0000","last_unstale":"2023-07-18T03:38:38.121914+0000","last_undegraded":"2023-07-18T03:38:38.121914+0000","last_fullsized":"2023-07-18T03:38:38.121914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:01:02.396334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121759+0000","last_change":"2023-07-18T03:38:36.533293+0000","last_active":"2023-07-18T03:38:38.121759+0000","last_peered":"2023-07-18T03:38:38.121759+0000","last_clean":"2023-07-18T03:38:38.121759+0000","last_became_active":"2023-07-18T03:38:36.532953+0000","last_became_peered":"2023-07-18T03:38:36.532953+0000","last_unstale":"2023-07-18T03:38:38.121759+0000","last_undegraded":"2023-07-18T03:38:38.121759+0000","last_fullsized":"2023-07-18T03:38:38.121759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:35:14.137991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629259+0000","last_change":"2023-07-18T03:38:36.536500+0000","last_active":"2023-07-18T03:38:37.629259+0000","last_peered":"2023-07-18T03:38:37.629259+0000","last_clean":"2023-07-18T03:38:37.629259+0000","last_became_active":"2023-07-18T03:38:36.536283+0000","last_became_peered":"2023-07-18T03:38:36.536283+0000","last_unstale":"2023-07-18T03:38:37.629259+0000","last_undegraded":"2023-07-18T03:38:37.629259+0000","last_fullsized":"2023-07-18T03:38:37.629259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:24.505474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516206+0000","last_change":"2023-07-18T03:38:36.530577+0000","last_active":"2023-07-18T03:38:37.516206+0000","last_peered":"2023-07-18T03:38:37.516206+0000","last_clean":"2023-07-18T03:38:37.516206+0000","last_became_active":"2023-07-18T03:38:36.530268+0000","last_became_peered":"2023-07-18T03:38:36.530268+0000","last_unstale":"2023-07-18T03:38:37.516206+0000","last_undegraded":"2023-07-18T03:38:37.516206+0000","last_fullsized":"2023-07-18T03:38:37.516206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:33:14.215460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516748+0000","last_change":"2023-07-18T03:38:36.526520+0000","last_active":"2023-07-18T03:38:37.516748+0000","last_peered":"2023-07-18T03:38:37.516748+0000","last_clean":"2023-07-18T03:38:37.516748+0000","last_became_active":"2023-07-18T03:38:36.526165+0000","last_became_peered":"2023-07-18T03:38:36.526165+0000","last_unstale":"2023-07-18T03:38:37.516748+0000","last_undegraded":"2023-07-18T03:38:37.516748+0000","last_fullsized":"2023-07-18T03:38:37.516748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:17:16.904471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121313+0000","last_change":"2023-07-18T03:38:36.531837+0000","last_active":"2023-07-18T03:38:38.121313+0000","last_peered":"2023-07-18T03:38:38.121313+0000","last_clean":"2023-07-18T03:38:38.121313+0000","last_became_active":"2023-07-18T03:38:36.531588+0000","last_became_peered":"2023-07-18T03:38:36.531588+0000","last_unstale":"2023-07-18T03:38:38.121313+0000","last_undegraded":"2023-07-18T03:38:38.121313+0000","last_fullsized":"2023-07-18T03:38:38.121313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:57:34.438689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516789+0000","last_change":"2023-07-18T03:38:36.536129+0000","last_active":"2023-07-18T03:38:37.516789+0000","last_peered":"2023-07-18T03:38:37.516789+0000","last_clean":"2023-07-18T03:38:37.516789+0000","last_became_active":"2023-07-18T03:38:36.535846+0000","last_became_peered":"2023-07-18T03:38:36.535846+0000","last_unstale":"2023-07-18T03:38:37.516789+0000","last_undegraded":"2023-07-18T03:38:37.516789+0000","last_fullsized":"2023-07-18T03:38:37.516789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:50:17.597467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.122134+0000","last_change":"2023-07-18T03:38:36.529212+0000","last_active":"2023-07-18T03:38:38.122134+0000","last_peered":"2023-07-18T03:38:38.122134+0000","last_clean":"2023-07-18T03:38:38.122134+0000","last_became_active":"2023-07-18T03:38:36.528956+0000","last_became_peered":"2023-07-18T03:38:36.528956+0000","last_unstale":"2023-07-18T03:38:38.122134+0000","last_undegraded":"2023-07-18T03:38:38.122134+0000","last_fullsized":"2023-07-18T03:38:38.122134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:23:08.908073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516481+0000","last_change":"2023-07-18T03:38:36.532777+0000","last_active":"2023-07-18T03:38:37.516481+0000","last_peered":"2023-07-18T03:38:37.516481+0000","last_clean":"2023-07-18T03:38:37.516481+0000","last_became_active":"2023-07-18T03:38:36.532519+0000","last_became_peered":"2023-07-18T03:38:36.532519+0000","last_unstale":"2023-07-18T03:38:37.516481+0000","last_undegraded":"2023-07-18T03:38:37.516481+0000","last_fullsized":"2023-07-18T03:38:37.516481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:27:43.633946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515979+0000","last_change":"2023-07-18T03:38:36.541130+0000","last_active":"2023-07-18T03:38:37.515979+0000","last_peered":"2023-07-18T03:38:37.515979+0000","last_clean":"2023-07-18T03:38:37.515979+0000","last_became_active":"2023-07-18T03:38:36.535483+0000","last_became_peered":"2023-07-18T03:38:36.535483+0000","last_unstale":"2023-07-18T03:38:37.515979+0000","last_undegraded":"2023-07-18T03:38:37.515979+0000","last_fullsized":"2023-07-18T03:38:37.515979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:46.558463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120549+0000","last_change":"2023-07-18T03:38:36.536797+0000","last_active":"2023-07-18T03:38:38.120549+0000","last_peered":"2023-07-18T03:38:38.120549+0000","last_clean":"2023-07-18T03:38:38.120549+0000","last_became_active":"2023-07-18T03:38:36.535389+0000","last_became_peered":"2023-07-18T03:38:36.535389+0000","last_unstale":"2023-07-18T03:38:38.120549+0000","last_undegraded":"2023-07-18T03:38:38.120549+0000","last_fullsized":"2023-07-18T03:38:38.120549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:22:44.555224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120428+0000","last_change":"2023-07-18T03:38:36.533007+0000","last_active":"2023-07-18T03:38:38.120428+0000","last_peered":"2023-07-18T03:38:38.120428+0000","last_clean":"2023-07-18T03:38:38.120428+0000","last_became_active":"2023-07-18T03:38:36.532714+0000","last_became_peered":"2023-07-18T03:38:36.532714+0000","last_unstale":"2023-07-18T03:38:38.120428+0000","last_undegraded":"2023-07-18T03:38:38.120428+0000","last_fullsized":"2023-07-18T03:38:38.120428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:14:38.515543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515725+0000","last_change":"2023-07-18T03:38:36.541077+0000","last_active":"2023-07-18T03:38:37.515725+0000","last_peered":"2023-07-18T03:38:37.515725+0000","last_clean":"2023-07-18T03:38:37.515725+0000","last_became_active":"2023-07-18T03:38:36.533768+0000","last_became_peered":"2023-07-18T03:38:36.533768+0000","last_unstale":"2023-07-18T03:38:37.515725+0000","last_undegraded":"2023-07-18T03:38:37.515725+0000","last_fullsized":"2023-07-18T03:38:37.515725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:42.081234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514860+0000","last_change":"2023-07-18T03:38:36.529999+0000","last_active":"2023-07-18T03:38:37.514860+0000","last_peered":"2023-07-18T03:38:37.514860+0000","last_clean":"2023-07-18T03:38:37.514860+0000","last_became_active":"2023-07-18T03:38:36.529489+0000","last_became_peered":"2023-07-18T03:38:36.529489+0000","last_unstale":"2023-07-18T03:38:37.514860+0000","last_undegraded":"2023-07-18T03:38:37.514860+0000","last_fullsized":"2023-07-18T03:38:37.514860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:24.505474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514898+0000","last_change":"2023-07-18T03:38:36.525279+0000","last_active":"2023-07-18T03:38:37.514898+0000","last_peered":"2023-07-18T03:38:37.514898+0000","last_clean":"2023-07-18T03:38:37.514898+0000","last_became_active":"2023-07-18T03:38:36.525018+0000","last_became_peered":"2023-07-18T03:38:36.525018+0000","last_unstale":"2023-07-18T03:38:37.514898+0000","last_undegraded":"2023-07-18T03:38:37.514898+0000","last_fullsized":"2023-07-18T03:38:37.514898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:47:14.281295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629309+0000","last_change":"2023-07-18T03:38:36.531164+0000","last_active":"2023-07-18T03:38:37.629309+0000","last_peered":"2023-07-18T03:38:37.629309+0000","last_clean":"2023-07-18T03:38:37.629309+0000","last_became_active":"2023-07-18T03:38:36.530907+0000","last_became_peered":"2023-07-18T03:38:36.530907+0000","last_unstale":"2023-07-18T03:38:37.629309+0000","last_undegraded":"2023-07-18T03:38:37.629309+0000","last_fullsized":"2023-07-18T03:38:37.629309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:47:14.281295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516140+0000","last_change":"2023-07-18T03:38:35.529655+0000","last_active":"2023-07-18T03:38:37.516140+0000","last_peered":"2023-07-18T03:38:37.516140+0000","last_clean":"2023-07-18T03:38:37.516140+0000","last_became_active":"2023-07-18T03:38:35.529257+0000","last_became_peered":"2023-07-18T03:38:35.529257+0000","last_unstale":"2023-07-18T03:38:37.516140+0000","last_undegraded":"2023-07-18T03:38:37.516140+0000","last_fullsized":"2023-07-18T03:38:37.516140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:52:51.944585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121222+0000","last_change":"2023-07-18T03:38:36.542322+0000","last_active":"2023-07-18T03:38:38.121222+0000","last_peered":"2023-07-18T03:38:38.121222+0000","last_clean":"2023-07-18T03:38:38.121222+0000","last_became_active":"2023-07-18T03:38:36.542038+0000","last_became_peered":"2023-07-18T03:38:36.542038+0000","last_unstale":"2023-07-18T03:38:38.121222+0000","last_undegraded":"2023-07-18T03:38:38.121222+0000","last_fullsized":"2023-07-18T03:38:38.121222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:33:14.215460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514288+0000","last_change":"2023-07-18T03:38:35.536467+0000","last_active":"2023-07-18T03:38:37.514288+0000","last_peered":"2023-07-18T03:38:37.514288+0000","last_clean":"2023-07-18T03:38:37.514288+0000","last_became_active":"2023-07-18T03:38:35.536198+0000","last_became_peered":"2023-07-18T03:38:35.536198+0000","last_unstale":"2023-07-18T03:38:37.514288+0000","last_undegraded":"2023-07-18T03:38:37.514288+0000","last_fullsized":"2023-07-18T03:38:37.514288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:01.259707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121012+0000","last_change":"2023-07-18T03:38:36.529534+0000","last_active":"2023-07-18T03:38:38.121012+0000","last_peered":"2023-07-18T03:38:38.121012+0000","last_clean":"2023-07-18T03:38:38.121012+0000","last_became_active":"2023-07-18T03:38:36.528958+0000","last_became_peered":"2023-07-18T03:38:36.528958+0000","last_unstale":"2023-07-18T03:38:38.121012+0000","last_undegraded":"2023-07-18T03:38:38.121012+0000","last_fullsized":"2023-07-18T03:38:38.121012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:39.607024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121102+0000","last_change":"2023-07-18T03:38:35.527369+0000","last_active":"2023-07-18T03:38:38.121102+0000","last_peered":"2023-07-18T03:38:38.121102+0000","last_clean":"2023-07-18T03:38:38.121102+0000","last_became_active":"2023-07-18T03:38:35.527091+0000","last_became_peered":"2023-07-18T03:38:35.527091+0000","last_unstale":"2023-07-18T03:38:38.121102+0000","last_undegraded":"2023-07-18T03:38:38.121102+0000","last_fullsized":"2023-07-18T03:38:38.121102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:21:50.290666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121460+0000","last_change":"2023-07-18T03:38:36.528845+0000","last_active":"2023-07-18T03:38:38.121460+0000","last_peered":"2023-07-18T03:38:38.121460+0000","last_clean":"2023-07-18T03:38:38.121460+0000","last_became_active":"2023-07-18T03:38:36.528525+0000","last_became_peered":"2023-07-18T03:38:36.528525+0000","last_unstale":"2023-07-18T03:38:38.121460+0000","last_undegraded":"2023-07-18T03:38:38.121460+0000","last_fullsized":"2023-07-18T03:38:38.121460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:16:39.663691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121461+0000","last_change":"2023-07-18T03:38:35.529837+0000","last_active":"2023-07-18T03:38:38.121461+0000","last_peered":"2023-07-18T03:38:38.121461+0000","last_clean":"2023-07-18T03:38:38.121461+0000","last_became_active":"2023-07-18T03:38:35.529486+0000","last_became_peered":"2023-07-18T03:38:35.529486+0000","last_unstale":"2023-07-18T03:38:38.121461+0000","last_undegraded":"2023-07-18T03:38:38.121461+0000","last_fullsized":"2023-07-18T03:38:38.121461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:20:07.344114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121086+0000","last_change":"2023-07-18T03:38:36.543236+0000","last_active":"2023-07-18T03:38:38.121086+0000","last_peered":"2023-07-18T03:38:38.121086+0000","last_clean":"2023-07-18T03:38:38.121086+0000","last_became_active":"2023-07-18T03:38:36.542960+0000","last_became_peered":"2023-07-18T03:38:36.542960+0000","last_unstale":"2023-07-18T03:38:38.121086+0000","last_undegraded":"2023-07-18T03:38:38.121086+0000","last_fullsized":"2023-07-18T03:38:38.121086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:46.558463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629501+0000","last_change":"2023-07-18T03:38:35.537672+0000","last_active":"2023-07-18T03:38:37.629501+0000","last_peered":"2023-07-18T03:38:37.629501+0000","last_clean":"2023-07-18T03:38:37.629501+0000","last_became_active":"2023-07-18T03:38:35.537362+0000","last_became_peered":"2023-07-18T03:38:35.537362+0000","last_unstale":"2023-07-18T03:38:37.629501+0000","last_undegraded":"2023-07-18T03:38:37.629501+0000","last_fullsized":"2023-07-18T03:38:37.629501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:38:40.610822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121013+0000","last_change":"2023-07-18T03:38:36.533526+0000","last_active":"2023-07-18T03:38:38.121013+0000","last_peered":"2023-07-18T03:38:38.121013+0000","last_clean":"2023-07-18T03:38:38.121013+0000","last_became_active":"2023-07-18T03:38:36.533278+0000","last_became_peered":"2023-07-18T03:38:36.533278+0000","last_unstale":"2023-07-18T03:38:38.121013+0000","last_undegraded":"2023-07-18T03:38:38.121013+0000","last_fullsized":"2023-07-18T03:38:38.121013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:45.182024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516465+0000","last_change":"2023-07-18T03:38:35.536258+0000","last_active":"2023-07-18T03:38:37.516465+0000","last_peered":"2023-07-18T03:38:37.516465+0000","last_clean":"2023-07-18T03:38:37.516465+0000","last_became_active":"2023-07-18T03:38:35.535939+0000","last_became_peered":"2023-07-18T03:38:35.535939+0000","last_unstale":"2023-07-18T03:38:37.516465+0000","last_undegraded":"2023-07-18T03:38:37.516465+0000","last_fullsized":"2023-07-18T03:38:37.516465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:30:23.496663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514229+0000","last_change":"2023-07-18T03:38:36.533896+0000","last_active":"2023-07-18T03:38:37.514229+0000","last_peered":"2023-07-18T03:38:37.514229+0000","last_clean":"2023-07-18T03:38:37.514229+0000","last_became_active":"2023-07-18T03:38:36.532384+0000","last_became_peered":"2023-07-18T03:38:36.532384+0000","last_unstale":"2023-07-18T03:38:37.514229+0000","last_undegraded":"2023-07-18T03:38:37.514229+0000","last_fullsized":"2023-07-18T03:38:37.514229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:13:15.481453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516636+0000","last_change":"2023-07-18T03:38:35.536771+0000","last_active":"2023-07-18T03:38:37.516636+0000","last_peered":"2023-07-18T03:38:37.516636+0000","last_clean":"2023-07-18T03:38:37.516636+0000","last_became_active":"2023-07-18T03:38:35.536423+0000","last_became_peered":"2023-07-18T03:38:35.536423+0000","last_unstale":"2023-07-18T03:38:37.516636+0000","last_undegraded":"2023-07-18T03:38:37.516636+0000","last_fullsized":"2023-07-18T03:38:37.516636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:30:23.496663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516904+0000","last_change":"2023-07-18T03:38:36.531396+0000","last_active":"2023-07-18T03:38:37.516904+0000","last_peered":"2023-07-18T03:38:37.516904+0000","last_clean":"2023-07-18T03:38:37.516904+0000","last_became_active":"2023-07-18T03:38:36.531117+0000","last_became_peered":"2023-07-18T03:38:36.531117+0000","last_unstale":"2023-07-18T03:38:37.516904+0000","last_undegraded":"2023-07-18T03:38:37.516904+0000","last_fullsized":"2023-07-18T03:38:37.516904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:36:15.063131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515828+0000","last_change":"2023-07-18T03:38:35.531528+0000","last_active":"2023-07-18T03:38:37.515828+0000","last_peered":"2023-07-18T03:38:37.515828+0000","last_clean":"2023-07-18T03:38:37.515828+0000","last_became_active":"2023-07-18T03:38:35.530889+0000","last_became_peered":"2023-07-18T03:38:35.530889+0000","last_unstale":"2023-07-18T03:38:37.515828+0000","last_undegraded":"2023-07-18T03:38:37.515828+0000","last_fullsized":"2023-07-18T03:38:37.515828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:52:51.944585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121161+0000","last_change":"2023-07-18T03:38:36.538483+0000","last_active":"2023-07-18T03:38:38.121161+0000","last_peered":"2023-07-18T03:38:38.121161+0000","last_clean":"2023-07-18T03:38:38.121161+0000","last_became_active":"2023-07-18T03:38:36.538133+0000","last_became_peered":"2023-07-18T03:38:36.538133+0000","last_unstale":"2023-07-18T03:38:38.121161+0000","last_undegraded":"2023-07-18T03:38:38.121161+0000","last_fullsized":"2023-07-18T03:38:38.121161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:27:36.142464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120772+0000","last_change":"2023-07-18T03:38:35.518650+0000","last_active":"2023-07-18T03:38:38.120772+0000","last_peered":"2023-07-18T03:38:38.120772+0000","last_clean":"2023-07-18T03:38:38.120772+0000","last_became_active":"2023-07-18T03:38:35.518285+0000","last_became_peered":"2023-07-18T03:38:35.518285+0000","last_unstale":"2023-07-18T03:38:38.120772+0000","last_undegraded":"2023-07-18T03:38:38.120772+0000","last_fullsized":"2023-07-18T03:38:38.120772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:57:59.447632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629401+0000","last_change":"2023-07-18T03:38:36.538615+0000","last_active":"2023-07-18T03:38:37.629401+0000","last_peered":"2023-07-18T03:38:37.629401+0000","last_clean":"2023-07-18T03:38:37.629401+0000","last_became_active":"2023-07-18T03:38:36.538362+0000","last_became_peered":"2023-07-18T03:38:36.538362+0000","last_unstale":"2023-07-18T03:38:37.629401+0000","last_undegraded":"2023-07-18T03:38:37.629401+0000","last_fullsized":"2023-07-18T03:38:37.629401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:14:38.515543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516038+0000","last_change":"2023-07-18T03:38:35.534762+0000","last_active":"2023-07-18T03:38:37.516038+0000","last_peered":"2023-07-18T03:38:37.516038+0000","last_clean":"2023-07-18T03:38:37.516038+0000","last_became_active":"2023-07-18T03:38:35.534529+0000","last_became_peered":"2023-07-18T03:38:35.534529+0000","last_unstale":"2023-07-18T03:38:37.516038+0000","last_undegraded":"2023-07-18T03:38:37.516038+0000","last_fullsized":"2023-07-18T03:38:37.516038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:27:35.134036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120813+0000","last_change":"2023-07-18T03:38:36.530844+0000","last_active":"2023-07-18T03:38:38.120813+0000","last_peered":"2023-07-18T03:38:38.120813+0000","last_clean":"2023-07-18T03:38:38.120813+0000","last_became_active":"2023-07-18T03:38:36.530545+0000","last_became_peered":"2023-07-18T03:38:36.530545+0000","last_unstale":"2023-07-18T03:38:38.120813+0000","last_undegraded":"2023-07-18T03:38:38.120813+0000","last_fullsized":"2023-07-18T03:38:38.120813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:39:26.376467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120972+0000","last_change":"2023-07-18T03:38:35.530210+0000","last_active":"2023-07-18T03:38:38.120972+0000","last_peered":"2023-07-18T03:38:38.120972+0000","last_clean":"2023-07-18T03:38:38.120972+0000","last_became_active":"2023-07-18T03:38:35.529906+0000","last_became_peered":"2023-07-18T03:38:35.529906+0000","last_unstale":"2023-07-18T03:38:38.120972+0000","last_undegraded":"2023-07-18T03:38:38.120972+0000","last_fullsized":"2023-07-18T03:38:38.120972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:59:17.569992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121717+0000","last_change":"2023-07-18T03:38:36.534252+0000","last_active":"2023-07-18T03:38:38.121717+0000","last_peered":"2023-07-18T03:38:38.121717+0000","last_clean":"2023-07-18T03:38:38.121717+0000","last_became_active":"2023-07-18T03:38:36.533986+0000","last_became_peered":"2023-07-18T03:38:36.533986+0000","last_unstale":"2023-07-18T03:38:38.121717+0000","last_undegraded":"2023-07-18T03:38:38.121717+0000","last_fullsized":"2023-07-18T03:38:38.121717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:13:47.285259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516720+0000","last_change":"2023-07-18T03:38:35.530100+0000","last_active":"2023-07-18T03:38:37.516720+0000","last_peered":"2023-07-18T03:38:37.516720+0000","last_clean":"2023-07-18T03:38:37.516720+0000","last_became_active":"2023-07-18T03:38:35.529084+0000","last_became_peered":"2023-07-18T03:38:35.529084+0000","last_unstale":"2023-07-18T03:38:37.516720+0000","last_undegraded":"2023-07-18T03:38:37.516720+0000","last_fullsized":"2023-07-18T03:38:37.516720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:25.804673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629276+0000","last_change":"2023-07-18T03:38:36.531903+0000","last_active":"2023-07-18T03:38:37.629276+0000","last_peered":"2023-07-18T03:38:37.629276+0000","last_clean":"2023-07-18T03:38:37.629276+0000","last_became_active":"2023-07-18T03:38:36.531503+0000","last_became_peered":"2023-07-18T03:38:36.531503+0000","last_unstale":"2023-07-18T03:38:37.629276+0000","last_undegraded":"2023-07-18T03:38:37.629276+0000","last_fullsized":"2023-07-18T03:38:37.629276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:16:39.663691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516435+0000","last_change":"2023-07-18T03:38:35.530991+0000","last_active":"2023-07-18T03:38:37.516435+0000","last_peered":"2023-07-18T03:38:37.516435+0000","last_clean":"2023-07-18T03:38:37.516435+0000","last_became_active":"2023-07-18T03:38:35.530676+0000","last_became_peered":"2023-07-18T03:38:35.530676+0000","last_unstale":"2023-07-18T03:38:37.516435+0000","last_undegraded":"2023-07-18T03:38:37.516435+0000","last_fullsized":"2023-07-18T03:38:37.516435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:43:42.440181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516298+0000","last_change":"2023-07-18T03:38:36.524535+0000","last_active":"2023-07-18T03:38:37.516298+0000","last_peered":"2023-07-18T03:38:37.516298+0000","last_clean":"2023-07-18T03:38:37.516298+0000","last_became_active":"2023-07-18T03:38:36.524235+0000","last_became_peered":"2023-07-18T03:38:36.524235+0000","last_unstale":"2023-07-18T03:38:37.516298+0000","last_undegraded":"2023-07-18T03:38:37.516298+0000","last_fullsized":"2023-07-18T03:38:37.516298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:21:51.299094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516290+0000","last_change":"2023-07-18T03:38:35.532333+0000","last_active":"2023-07-18T03:38:37.516290+0000","last_peered":"2023-07-18T03:38:37.516290+0000","last_clean":"2023-07-18T03:38:37.516290+0000","last_became_active":"2023-07-18T03:38:35.531899+0000","last_became_peered":"2023-07-18T03:38:35.531899+0000","last_unstale":"2023-07-18T03:38:37.516290+0000","last_undegraded":"2023-07-18T03:38:37.516290+0000","last_fullsized":"2023-07-18T03:38:37.516290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:27:42.625518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121237+0000","last_change":"2023-07-18T03:38:36.540597+0000","last_active":"2023-07-18T03:38:38.121237+0000","last_peered":"2023-07-18T03:38:38.121237+0000","last_clean":"2023-07-18T03:38:38.121237+0000","last_became_active":"2023-07-18T03:38:36.540313+0000","last_became_peered":"2023-07-18T03:38:36.540313+0000","last_unstale":"2023-07-18T03:38:38.121237+0000","last_undegraded":"2023-07-18T03:38:38.121237+0000","last_fullsized":"2023-07-18T03:38:38.121237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:02.268135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629372+0000","last_change":"2023-07-18T03:38:35.536145+0000","last_active":"2023-07-18T03:38:37.629372+0000","last_peered":"2023-07-18T03:38:37.629372+0000","last_clean":"2023-07-18T03:38:37.629372+0000","last_became_active":"2023-07-18T03:38:35.535827+0000","last_became_peered":"2023-07-18T03:38:35.535827+0000","last_unstale":"2023-07-18T03:38:37.629372+0000","last_undegraded":"2023-07-18T03:38:37.629372+0000","last_fullsized":"2023-07-18T03:38:37.629372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:44:02.784217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120807+0000","last_change":"2023-07-18T03:38:36.526183+0000","last_active":"2023-07-18T03:38:38.120807+0000","last_peered":"2023-07-18T03:38:38.120807+0000","last_clean":"2023-07-18T03:38:38.120807+0000","last_became_active":"2023-07-18T03:38:36.525853+0000","last_became_peered":"2023-07-18T03:38:36.525853+0000","last_unstale":"2023-07-18T03:38:38.120807+0000","last_undegraded":"2023-07-18T03:38:38.120807+0000","last_fullsized":"2023-07-18T03:38:38.120807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:51:42.726329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121779+0000","last_change":"2023-07-18T03:38:35.534166+0000","last_active":"2023-07-18T03:38:38.121779+0000","last_peered":"2023-07-18T03:38:38.121779+0000","last_clean":"2023-07-18T03:38:38.121779+0000","last_became_active":"2023-07-18T03:38:35.533900+0000","last_became_peered":"2023-07-18T03:38:35.533900+0000","last_unstale":"2023-07-18T03:38:38.121779+0000","last_undegraded":"2023-07-18T03:38:38.121779+0000","last_fullsized":"2023-07-18T03:38:38.121779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:17:15.896043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516546+0000","last_change":"2023-07-18T03:38:36.535669+0000","last_active":"2023-07-18T03:38:37.516546+0000","last_peered":"2023-07-18T03:38:37.516546+0000","last_clean":"2023-07-18T03:38:37.516546+0000","last_became_active":"2023-07-18T03:38:36.535338+0000","last_became_peered":"2023-07-18T03:38:36.535338+0000","last_unstale":"2023-07-18T03:38:37.516546+0000","last_undegraded":"2023-07-18T03:38:37.516546+0000","last_fullsized":"2023-07-18T03:38:37.516546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:43:43.448609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121279+0000","last_change":"2023-07-18T03:38:35.530541+0000","last_active":"2023-07-18T03:38:38.121279+0000","last_peered":"2023-07-18T03:38:38.121279+0000","last_clean":"2023-07-18T03:38:38.121279+0000","last_became_active":"2023-07-18T03:38:35.530220+0000","last_became_peered":"2023-07-18T03:38:35.530220+0000","last_unstale":"2023-07-18T03:38:38.121279+0000","last_undegraded":"2023-07-18T03:38:38.121279+0000","last_fullsized":"2023-07-18T03:38:38.121279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:53:15.570874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516299+0000","last_change":"2023-07-18T03:38:36.523648+0000","last_active":"2023-07-18T03:38:37.516299+0000","last_peered":"2023-07-18T03:38:37.516299+0000","last_clean":"2023-07-18T03:38:37.516299+0000","last_became_active":"2023-07-18T03:38:36.523346+0000","last_became_peered":"2023-07-18T03:38:36.523346+0000","last_unstale":"2023-07-18T03:38:37.516299+0000","last_undegraded":"2023-07-18T03:38:37.516299+0000","last_fullsized":"2023-07-18T03:38:37.516299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:46.558463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516714+0000","last_change":"2023-07-18T03:38:35.534663+0000","last_active":"2023-07-18T03:38:37.516714+0000","last_peered":"2023-07-18T03:38:37.516714+0000","last_clean":"2023-07-18T03:38:37.516714+0000","last_became_active":"2023-07-18T03:38:35.534363+0000","last_became_peered":"2023-07-18T03:38:35.534363+0000","last_unstale":"2023-07-18T03:38:37.516714+0000","last_undegraded":"2023-07-18T03:38:37.516714+0000","last_fullsized":"2023-07-18T03:38:37.516714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:13:46.276831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516932+0000","last_change":"2023-07-18T03:38:36.531919+0000","last_active":"2023-07-18T03:38:37.516932+0000","last_peered":"2023-07-18T03:38:37.516932+0000","last_clean":"2023-07-18T03:38:37.516932+0000","last_became_active":"2023-07-18T03:38:36.531729+0000","last_became_peered":"2023-07-18T03:38:36.531729+0000","last_unstale":"2023-07-18T03:38:37.516932+0000","last_undegraded":"2023-07-18T03:38:37.516932+0000","last_fullsized":"2023-07-18T03:38:37.516932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:02:54.254215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120962+0000","last_change":"2023-07-18T03:38:35.538915+0000","last_active":"2023-07-18T03:38:38.120962+0000","last_peered":"2023-07-18T03:38:38.120962+0000","last_clean":"2023-07-18T03:38:38.120962+0000","last_became_active":"2023-07-18T03:38:35.538539+0000","last_became_peered":"2023-07-18T03:38:35.538539+0000","last_unstale":"2023-07-18T03:38:38.120962+0000","last_undegraded":"2023-07-18T03:38:38.120962+0000","last_fullsized":"2023-07-18T03:38:38.120962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:52:51.944585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515021+0000","last_change":"2023-07-18T03:38:36.534839+0000","last_active":"2023-07-18T03:38:37.515021+0000","last_peered":"2023-07-18T03:38:37.515021+0000","last_clean":"2023-07-18T03:38:37.515021+0000","last_became_active":"2023-07-18T03:38:36.532743+0000","last_became_peered":"2023-07-18T03:38:36.532743+0000","last_unstale":"2023-07-18T03:38:37.515021+0000","last_undegraded":"2023-07-18T03:38:37.515021+0000","last_fullsized":"2023-07-18T03:38:37.515021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:14:38.515543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121583+0000","last_change":"2023-07-18T03:38:35.538083+0000","last_active":"2023-07-18T03:38:38.121583+0000","last_peered":"2023-07-18T03:38:38.121583+0000","last_clean":"2023-07-18T03:38:38.121583+0000","last_became_active":"2023-07-18T03:38:35.537692+0000","last_became_peered":"2023-07-18T03:38:35.537692+0000","last_unstale":"2023-07-18T03:38:38.121583+0000","last_undegraded":"2023-07-18T03:38:38.121583+0000","last_fullsized":"2023-07-18T03:38:38.121583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:25.804673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516238+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:38:37.516238+0000","last_peered":"2023-07-18T03:38:37.516238+0000","last_clean":"2023-07-18T03:38:37.516238+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:38:37.516238+0000","last_undegraded":"2023-07-18T03:38:37.516238+0000","last_fullsized":"2023-07-18T03:38:37.516238+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:38.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120829+0000","last_change":"2023-07-18T03:38:36.531962+0000","last_active":"2023-07-18T03:38:38.120829+0000","last_peered":"2023-07-18T03:38:38.120829+0000","last_clean":"2023-07-18T03:38:38.120829+0000","last_became_active":"2023-07-18T03:38:36.531710+0000","last_became_peered":"2023-07-18T03:38:36.531710+0000","last_unstale":"2023-07-18T03:38:38.120829+0000","last_undegraded":"2023-07-18T03:38:38.120829+0000","last_fullsized":"2023-07-18T03:38:38.120829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:42.081234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121276+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:38:38.121276+0000","last_peered":"2023-07-18T03:38:38.121276+0000","last_clean":"2023-07-18T03:38:38.121276+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:38:38.121276+0000","last_undegraded":"2023-07-18T03:38:38.121276+0000","last_fullsized":"2023-07-18T03:38:38.121276+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:27:36.456272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.198672+0000","last_change":"2023-07-18T03:38:35.522198+0000","last_active":"2023-07-18T03:38:38.198672+0000","last_peered":"2023-07-18T03:38:38.198672+0000","last_clean":"2023-07-18T03:38:38.198672+0000","last_became_active":"2023-07-18T03:38:35.521915+0000","last_became_peered":"2023-07-18T03:38:35.521915+0000","last_unstale":"2023-07-18T03:38:38.198672+0000","last_undegraded":"2023-07-18T03:38:38.198672+0000","last_fullsized":"2023-07-18T03:38:38.198672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:06:13.388319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516480+0000","last_change":"2023-07-18T03:38:36.537103+0000","last_active":"2023-07-18T03:38:37.516480+0000","last_peered":"2023-07-18T03:38:37.516480+0000","last_clean":"2023-07-18T03:38:37.516480+0000","last_became_active":"2023-07-18T03:38:36.534926+0000","last_became_peered":"2023-07-18T03:38:36.534926+0000","last_unstale":"2023-07-18T03:38:37.516480+0000","last_undegraded":"2023-07-18T03:38:37.516480+0000","last_fullsized":"2023-07-18T03:38:37.516480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:33:14.215460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514498+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:38:37.514498+0000","last_peered":"2023-07-18T03:38:37.514498+0000","last_clean":"2023-07-18T03:38:37.514498+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:38:37.514498+0000","last_undegraded":"2023-07-18T03:38:37.514498+0000","last_fullsized":"2023-07-18T03:38:37.514498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:31.720395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":169,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515826+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:38:37.515826+0000","last_peered":"2023-07-18T03:38:37.515826+0000","last_clean":"2023-07-18T03:38:37.515826+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:38:37.515826+0000","last_undegraded":"2023-07-18T03:38:37.515826+0000","last_fullsized":"2023-07-18T03:38:37.515826+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:38.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120579+0000","last_change":"2023-07-18T03:38:35.534781+0000","last_active":"2023-07-18T03:38:38.120579+0000","last_peered":"2023-07-18T03:38:38.120579+0000","last_clean":"2023-07-18T03:38:38.120579+0000","last_became_active":"2023-07-18T03:38:35.534485+0000","last_became_peered":"2023-07-18T03:38:35.534485+0000","last_unstale":"2023-07-18T03:38:38.120579+0000","last_undegraded":"2023-07-18T03:38:38.120579+0000","last_fullsized":"2023-07-18T03:38:38.120579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:42:36.215812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120857+0000","last_change":"2023-07-18T03:38:36.527606+0000","last_active":"2023-07-18T03:38:38.120857+0000","last_peered":"2023-07-18T03:38:38.120857+0000","last_clean":"2023-07-18T03:38:38.120857+0000","last_became_active":"2023-07-18T03:38:36.527267+0000","last_became_peered":"2023-07-18T03:38:36.527267+0000","last_unstale":"2023-07-18T03:38:38.120857+0000","last_undegraded":"2023-07-18T03:38:38.120857+0000","last_fullsized":"2023-07-18T03:38:38.120857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:32:24.221027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515516+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:38:37.515516+0000","last_peered":"2023-07-18T03:38:37.515516+0000","last_clean":"2023-07-18T03:38:37.515516+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:38:37.515516+0000","last_undegraded":"2023-07-18T03:38:37.515516+0000","last_fullsized":"2023-07-18T03:38:37.515516+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:02.380287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514037+0000","last_change":"2023-07-18T03:38:35.531556+0000","last_active":"2023-07-18T03:38:37.514037+0000","last_peered":"2023-07-18T03:38:37.514037+0000","last_clean":"2023-07-18T03:38:37.514037+0000","last_became_active":"2023-07-18T03:38:35.531268+0000","last_became_peered":"2023-07-18T03:38:35.531268+0000","last_unstale":"2023-07-18T03:38:37.514037+0000","last_undegraded":"2023-07-18T03:38:37.514037+0000","last_fullsized":"2023-07-18T03:38:37.514037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:59:17.569992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514994+0000","last_change":"2023-07-18T03:38:36.536571+0000","last_active":"2023-07-18T03:38:37.514994+0000","last_peered":"2023-07-18T03:38:37.514994+0000","last_clean":"2023-07-18T03:38:37.514994+0000","last_became_active":"2023-07-18T03:38:36.536187+0000","last_became_peered":"2023-07-18T03:38:36.536187+0000","last_unstale":"2023-07-18T03:38:37.514994+0000","last_undegraded":"2023-07-18T03:38:37.514994+0000","last_fullsized":"2023-07-18T03:38:37.514994+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:44:03.792645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515534+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:38:37.515534+0000","last_peered":"2023-07-18T03:38:37.515534+0000","last_clean":"2023-07-18T03:38:37.515534+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:38:37.515534+0000","last_undegraded":"2023-07-18T03:38:37.515534+0000","last_fullsized":"2023-07-18T03:38:37.515534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:38:34.441575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121349+0000","last_change":"2023-07-18T03:38:35.536386+0000","last_active":"2023-07-18T03:38:38.121349+0000","last_peered":"2023-07-18T03:38:38.121349+0000","last_clean":"2023-07-18T03:38:38.121349+0000","last_became_active":"2023-07-18T03:38:35.536066+0000","last_became_peered":"2023-07-18T03:38:35.536066+0000","last_unstale":"2023-07-18T03:38:38.121349+0000","last_undegraded":"2023-07-18T03:38:38.121349+0000","last_fullsized":"2023-07-18T03:38:38.121349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:43:42.440181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121258+0000","last_change":"2023-07-18T03:38:36.528088+0000","last_active":"2023-07-18T03:38:38.121258+0000","last_peered":"2023-07-18T03:38:38.121258+0000","last_clean":"2023-07-18T03:38:38.121258+0000","last_became_active":"2023-07-18T03:38:36.527760+0000","last_became_peered":"2023-07-18T03:38:36.527760+0000","last_unstale":"2023-07-18T03:38:38.121258+0000","last_undegraded":"2023-07-18T03:38:38.121258+0000","last_fullsized":"2023-07-18T03:38:38.121258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:38.898070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629222+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:38:37.629222+0000","last_peered":"2023-07-18T03:38:37.629222+0000","last_clean":"2023-07-18T03:38:37.629222+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:38:37.629222+0000","last_undegraded":"2023-07-18T03:38:37.629222+0000","last_fullsized":"2023-07-18T03:38:37.629222+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:50:38.894203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.517262+0000","last_change":"2023-07-18T03:38:35.526465+0000","last_active":"2023-07-18T03:38:37.517262+0000","last_peered":"2023-07-18T03:38:37.517262+0000","last_clean":"2023-07-18T03:38:37.517262+0000","last_became_active":"2023-07-18T03:38:35.525759+0000","last_became_peered":"2023-07-18T03:38:35.525759+0000","last_unstale":"2023-07-18T03:38:37.517262+0000","last_undegraded":"2023-07-18T03:38:37.517262+0000","last_fullsized":"2023-07-18T03:38:37.517262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:56:57.134347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514887+0000","last_change":"2023-07-18T03:38:36.523849+0000","last_active":"2023-07-18T03:38:37.514887+0000","last_peered":"2023-07-18T03:38:37.514887+0000","last_clean":"2023-07-18T03:38:37.514887+0000","last_became_active":"2023-07-18T03:38:36.523555+0000","last_became_peered":"2023-07-18T03:38:36.523555+0000","last_unstale":"2023-07-18T03:38:37.514887+0000","last_undegraded":"2023-07-18T03:38:37.514887+0000","last_fullsized":"2023-07-18T03:38:37.514887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:16:39.663691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121324+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:38:38.121324+0000","last_peered":"2023-07-18T03:38:38.121324+0000","last_clean":"2023-07-18T03:38:38.121324+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:38:38.121324+0000","last_undegraded":"2023-07-18T03:38:38.121324+0000","last_fullsized":"2023-07-18T03:38:38.121324+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:50:38.894203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514759+0000","last_change":"2023-07-18T03:38:35.530710+0000","last_active":"2023-07-18T03:38:37.514759+0000","last_peered":"2023-07-18T03:38:37.514759+0000","last_clean":"2023-07-18T03:38:37.514759+0000","last_became_active":"2023-07-18T03:38:35.530463+0000","last_became_peered":"2023-07-18T03:38:35.530463+0000","last_unstale":"2023-07-18T03:38:37.514759+0000","last_undegraded":"2023-07-18T03:38:37.514759+0000","last_fullsized":"2023-07-18T03:38:37.514759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:50:45.063449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121163+0000","last_change":"2023-07-18T03:38:36.535754+0000","last_active":"2023-07-18T03:38:38.121163+0000","last_peered":"2023-07-18T03:38:38.121163+0000","last_clean":"2023-07-18T03:38:38.121163+0000","last_became_active":"2023-07-18T03:38:36.535290+0000","last_became_peered":"2023-07-18T03:38:36.535290+0000","last_unstale":"2023-07-18T03:38:38.121163+0000","last_undegraded":"2023-07-18T03:38:38.121163+0000","last_fullsized":"2023-07-18T03:38:38.121163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:08:03.011261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121317+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:38:38.121317+0000","last_peered":"2023-07-18T03:38:38.121317+0000","last_clean":"2023-07-18T03:38:38.121317+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:38:38.121317+0000","last_undegraded":"2023-07-18T03:38:38.121317+0000","last_fullsized":"2023-07-18T03:38:38.121317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:42:57.376881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629589+0000","last_change":"2023-07-18T03:38:35.536982+0000","last_active":"2023-07-18T03:38:37.629589+0000","last_peered":"2023-07-18T03:38:37.629589+0000","last_clean":"2023-07-18T03:38:37.629589+0000","last_became_active":"2023-07-18T03:38:35.536739+0000","last_became_peered":"2023-07-18T03:38:35.536739+0000","last_unstale":"2023-07-18T03:38:37.629589+0000","last_undegraded":"2023-07-18T03:38:37.629589+0000","last_fullsized":"2023-07-18T03:38:37.629589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:52:51.944585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514520+0000","last_change":"2023-07-18T03:38:36.532972+0000","last_active":"2023-07-18T03:38:37.514520+0000","last_peered":"2023-07-18T03:38:37.514520+0000","last_clean":"2023-07-18T03:38:37.514520+0000","last_became_active":"2023-07-18T03:38:36.530512+0000","last_became_peered":"2023-07-18T03:38:36.530512+0000","last_unstale":"2023-07-18T03:38:37.514520+0000","last_undegraded":"2023-07-18T03:38:37.514520+0000","last_fullsized":"2023-07-18T03:38:37.514520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:53:16.579302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121293+0000","last_change":"2023-07-18T03:38:35.525179+0000","last_active":"2023-07-18T03:38:38.121293+0000","last_peered":"2023-07-18T03:38:38.121293+0000","last_clean":"2023-07-18T03:38:38.121293+0000","last_became_active":"2023-07-18T03:38:35.524912+0000","last_became_peered":"2023-07-18T03:38:35.524912+0000","last_unstale":"2023-07-18T03:38:38.121293+0000","last_undegraded":"2023-07-18T03:38:38.121293+0000","last_fullsized":"2023-07-18T03:38:38.121293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:23:28.704920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.122118+0000","last_change":"2023-07-18T03:38:36.529814+0000","last_active":"2023-07-18T03:38:38.122118+0000","last_peered":"2023-07-18T03:38:38.122118+0000","last_clean":"2023-07-18T03:38:38.122118+0000","last_became_active":"2023-07-18T03:38:36.529554+0000","last_became_peered":"2023-07-18T03:38:36.529554+0000","last_unstale":"2023-07-18T03:38:38.122118+0000","last_undegraded":"2023-07-18T03:38:38.122118+0000","last_fullsized":"2023-07-18T03:38:38.122118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:30:55.302127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629197+0000","last_change":"2023-07-18T03:38:35.529892+0000","last_active":"2023-07-18T03:38:37.629197+0000","last_peered":"2023-07-18T03:38:37.629197+0000","last_clean":"2023-07-18T03:38:37.629197+0000","last_became_active":"2023-07-18T03:38:35.529614+0000","last_became_peered":"2023-07-18T03:38:35.529614+0000","last_unstale":"2023-07-18T03:38:37.629197+0000","last_undegraded":"2023-07-18T03:38:37.629197+0000","last_fullsized":"2023-07-18T03:38:37.629197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:53:15.570874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.517120+0000","last_change":"2023-07-18T03:38:36.535652+0000","last_active":"2023-07-18T03:38:37.517120+0000","last_peered":"2023-07-18T03:38:37.517120+0000","last_clean":"2023-07-18T03:38:37.517120+0000","last_became_active":"2023-07-18T03:38:36.535378+0000","last_became_peered":"2023-07-18T03:38:36.535378+0000","last_unstale":"2023-07-18T03:38:37.517120+0000","last_undegraded":"2023-07-18T03:38:37.517120+0000","last_fullsized":"2023-07-18T03:38:37.517120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:20.087591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.629487+0000","last_change":"2023-07-18T03:38:35.534949+0000","last_active":"2023-07-18T03:38:37.629487+0000","last_peered":"2023-07-18T03:38:37.629487+0000","last_clean":"2023-07-18T03:38:37.629487+0000","last_became_active":"2023-07-18T03:38:35.534685+0000","last_became_peered":"2023-07-18T03:38:35.534685+0000","last_unstale":"2023-07-18T03:38:37.629487+0000","last_undegraded":"2023-07-18T03:38:37.629487+0000","last_fullsized":"2023-07-18T03:38:37.629487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:08.549533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516411+0000","last_change":"2023-07-18T03:38:36.535757+0000","last_active":"2023-07-18T03:38:37.516411+0000","last_peered":"2023-07-18T03:38:37.516411+0000","last_clean":"2023-07-18T03:38:37.516411+0000","last_became_active":"2023-07-18T03:38:36.535452+0000","last_became_peered":"2023-07-18T03:38:36.535452+0000","last_unstale":"2023-07-18T03:38:37.516411+0000","last_undegraded":"2023-07-18T03:38:37.516411+0000","last_fullsized":"2023-07-18T03:38:37.516411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:27:36.142464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121576+0000","last_change":"2023-07-18T03:38:35.532954+0000","last_active":"2023-07-18T03:38:38.121576+0000","last_peered":"2023-07-18T03:38:38.121576+0000","last_clean":"2023-07-18T03:38:38.121576+0000","last_became_active":"2023-07-18T03:38:35.532617+0000","last_became_peered":"2023-07-18T03:38:35.532617+0000","last_unstale":"2023-07-18T03:38:38.121576+0000","last_undegraded":"2023-07-18T03:38:38.121576+0000","last_fullsized":"2023-07-18T03:38:38.121576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:30:23.496663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120577+0000","last_change":"2023-07-18T03:38:36.540001+0000","last_active":"2023-07-18T03:38:38.120577+0000","last_peered":"2023-07-18T03:38:38.120577+0000","last_clean":"2023-07-18T03:38:38.120577+0000","last_became_active":"2023-07-18T03:38:36.539743+0000","last_became_peered":"2023-07-18T03:38:36.539743+0000","last_unstale":"2023-07-18T03:38:38.120577+0000","last_undegraded":"2023-07-18T03:38:38.120577+0000","last_fullsized":"2023-07-18T03:38:38.120577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:38:41.619250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121427+0000","last_change":"2023-07-18T03:38:35.532342+0000","last_active":"2023-07-18T03:38:38.121427+0000","last_peered":"2023-07-18T03:38:38.121427+0000","last_clean":"2023-07-18T03:38:38.121427+0000","last_became_active":"2023-07-18T03:38:35.530138+0000","last_became_peered":"2023-07-18T03:38:35.530138+0000","last_unstale":"2023-07-18T03:38:38.121427+0000","last_undegraded":"2023-07-18T03:38:38.121427+0000","last_fullsized":"2023-07-18T03:38:38.121427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:34.490348+0000","last_clean_scrub_stamp":"2023-07-18T03:38:34.490348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:59:26.646208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.516677+0000","last_change":"2023-07-18T03:38:36.531861+0000","last_active":"2023-07-18T03:38:37.516677+0000","last_peered":"2023-07-18T03:38:37.516677+0000","last_clean":"2023-07-18T03:38:37.516677+0000","last_became_active":"2023-07-18T03:38:36.531563+0000","last_became_peered":"2023-07-18T03:38:36.531563+0000","last_unstale":"2023-07-18T03:38:37.516677+0000","last_undegraded":"2023-07-18T03:38:37.516677+0000","last_fullsized":"2023-07-18T03:38:37.516677+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:26.813102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121209+0000","last_change":"2023-07-18T03:38:36.537688+0000","last_active":"2023-07-18T03:38:38.121209+0000","last_peered":"2023-07-18T03:38:38.121209+0000","last_clean":"2023-07-18T03:38:38.121209+0000","last_became_active":"2023-07-18T03:38:36.537415+0000","last_became_peered":"2023-07-18T03:38:36.537415+0000","last_unstale":"2023-07-18T03:38:38.121209+0000","last_undegraded":"2023-07-18T03:38:38.121209+0000","last_fullsized":"2023-07-18T03:38:38.121209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:13:15.481453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.120567+0000","last_change":"2023-07-18T03:38:36.542378+0000","last_active":"2023-07-18T03:38:38.120567+0000","last_peered":"2023-07-18T03:38:38.120567+0000","last_clean":"2023-07-18T03:38:38.120567+0000","last_became_active":"2023-07-18T03:38:36.542101+0000","last_became_peered":"2023-07-18T03:38:36.542101+0000","last_unstale":"2023-07-18T03:38:38.120567+0000","last_undegraded":"2023-07-18T03:38:38.120567+0000","last_fullsized":"2023-07-18T03:38:38.120567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:09.557961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.515970+0000","last_change":"2023-07-18T03:38:36.531395+0000","last_active":"2023-07-18T03:38:37.515970+0000","last_peered":"2023-07-18T03:38:37.515970+0000","last_clean":"2023-07-18T03:38:37.515970+0000","last_became_active":"2023-07-18T03:38:36.531071+0000","last_became_peered":"2023-07-18T03:38:36.531071+0000","last_unstale":"2023-07-18T03:38:37.515970+0000","last_undegraded":"2023-07-18T03:38:37.515970+0000","last_fullsized":"2023-07-18T03:38:37.515970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:38:41.619250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:38.121823+0000","last_change":"2023-07-18T03:38:36.534035+0000","last_active":"2023-07-18T03:38:38.121823+0000","last_peered":"2023-07-18T03:38:38.121823+0000","last_clean":"2023-07-18T03:38:38.121823+0000","last_became_active":"2023-07-18T03:38:36.533755+0000","last_became_peered":"2023-07-18T03:38:36.533755+0000","last_unstale":"2023-07-18T03:38:38.121823+0000","last_undegraded":"2023-07-18T03:38:38.121823+0000","last_fullsized":"2023-07-18T03:38:38.121823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:50:17.597467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-18T03:38:37.514727+0000","last_change":"2023-07-18T03:38:36.524624+0000","last_active":"2023-07-18T03:38:37.514727+0000","last_peered":"2023-07-18T03:38:37.514727+0000","last_clean":"2023-07-18T03:38:37.514727+0000","last_became_active":"2023-07-18T03:38:36.524106+0000","last_became_peered":"2023-07-18T03:38:36.524106+0000","last_unstale":"2023-07-18T03:38:37.514727+0000","last_undegraded":"2023-07-18T03:38:37.514727+0000","last_fullsized":"2023-07-18T03:38:37.514727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:35.498776+0000","last_clean_scrub_stamp":"2023-07-18T03:38:35.498776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:04.554555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":843776,"data_stored":673348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":81200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31268,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340572,"statfs":{"total":96636764160,"available":96604745728,"internally_reserved":0,"allocated":225280,"data_stored":81290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":815104,"data_stored":671568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":82820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":81852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":81782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31264,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340576,"statfs":{"total":96636764160,"available":96604749824,"internally_reserved":0,"allocated":221184,"data_stored":81200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:38:47.386 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-18T03:38:47.386 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-18T03:38:47.386 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-07-18T03:38:47.386 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-18T03:38:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.627+0000 7fef779ff700 1 Processor -- start 2023-07-18T03:38:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.627+0000 7fef779ff700 1 -- start start 2023-07-18T03:38:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.628+0000 7fef779ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fef700d1220 0x7fef700b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.628+0000 7fef779ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef700b1e00 0x7fef700afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.628+0000 7fef779ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef700d2ce0 con 0x7fef7015c950 2023-07-18T03:38:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.628+0000 7fef779ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef700b3350 con 0x7fef700d1220 2023-07-18T03:38:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.629+0000 7fef779ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef700b3630 con 0x7fef700b1e00 2023-07-18T03:38:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.629+0000 7fef7579b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fef7015c950 0x7fef700d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41694/0 (socket says 172.21.15.49:41694) 2023-07-18T03:38:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.629+0000 7fef7579b700 1 -- 172.21.15.49:0/2435508277 learned_addr learned my addr 172.21.15.49:0/2435508277 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:47.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.629+0000 7fef75f9c700 1 --2- 172.21.15.49:0/2435508277 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef700b1e00 0x7fef700afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:47.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.629+0000 7fef74f9a700 1 --2- 172.21.15.49:0/2435508277 >> v2:172.21.15.107:3300/0 conn(0x7fef700d1220 0x7fef700b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:47.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.629+0000 7fef67fff700 1 -- 172.21.15.49:0/2435508277 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864421060 0 0) 0x7fef700d2ce0 con 0x7fef7015c950 2023-07-18T03:38:47.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.629+0000 7fef67fff700 1 -- 172.21.15.49:0/2435508277 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef5c0035b0 con 0x7fef7015c950 2023-07-18T03:38:47.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.629+0000 7fef74f9a700 1 -- 172.21.15.49:0/2435508277 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef700b1e00 msgr2=0x7fef700afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:47.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.629+0000 7fef74f9a700 1 --2- 172.21.15.49:0/2435508277 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef700b1e00 0x7fef700afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.629+0000 7fef74f9a700 1 -- 172.21.15.49:0/2435508277 >> v1:172.21.15.49:6789/0 conn(0x7fef7015c950 legacy=0x7fef700d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.629+0000 7fef74f9a700 1 -- 172.21.15.49:0/2435508277 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef700b3910 con 0x7fef700d1220 2023-07-18T03:38:47.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.629+0000 7fef75f9c700 1 --2- 172.21.15.49:0/2435508277 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef700b1e00 0x7fef700afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:47.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.630+0000 7fef74f9a700 1 --2- 172.21.15.49:0/2435508277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef700d1220 0x7fef700b43f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef60009840 tx=0x7fef60009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ebdbcb7fa95e9b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:47.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.630+0000 7fef67fff700 1 -- 172.21.15.49:0/2435508277 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60011830 con 0x7fef700d1220 2023-07-18T03:38:47.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.630+0000 7fef67fff700 1 -- 172.21.15.49:0/2435508277 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef60011990 con 0x7fef700d1220 2023-07-18T03:38:47.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.630+0000 7fef67fff700 1 -- 172.21.15.49:0/2435508277 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60011d20 con 0x7fef700d1220 2023-07-18T03:38:47.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.630+0000 7fef779ff700 1 -- 172.21.15.49:0/2435508277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef700d1220 msgr2=0x7fef700b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.630+0000 7fef779ff700 1 --2- 172.21.15.49:0/2435508277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef700d1220 0x7fef700b43f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef60009840 tx=0x7fef60009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:47.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.631+0000 7fef779ff700 1 -- 172.21.15.49:0/2435508277 shutdown_connections 2023-07-18T03:38:47.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.631+0000 7fef779ff700 1 --2- 172.21.15.49:0/2435508277 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef700b1e00 0x7fef700afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.631+0000 7fef779ff700 1 --2- 172.21.15.49:0/2435508277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef700d1220 0x7fef700b43f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.631+0000 7fef779ff700 1 -- 172.21.15.49:0/2435508277 >> 172.21.15.49:0/2435508277 conn(0x7fef7009c1d0 msgr2=0x7fef700d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:47.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.631+0000 7fef779ff700 1 -- 172.21.15.49:0/2435508277 shutdown_connections 2023-07-18T03:38:47.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.631+0000 7fef779ff700 1 -- 172.21.15.49:0/2435508277 wait complete. 2023-07-18T03:38:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.632+0000 7fef779ff700 1 Processor -- start 2023-07-18T03:38:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.633+0000 7fef779ff700 1 -- start start 2023-07-18T03:38:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.633+0000 7fef779ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef700b1e00 0x7fef7015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.633+0000 7fef779ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef700d1220 0x7fef701f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.633+0000 7fef779ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7015c950 0x7fef701f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.633+0000 7fef779ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7015e120 con 0x7fef7015c950 2023-07-18T03:38:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef779ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef70157da0 con 0x7fef700d1220 2023-07-18T03:38:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef779ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef70157190 con 0x7fef700b1e00 2023-07-18T03:38:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef75f9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7015c950 0x7fef701f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef75f9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7015c950 0x7fef701f57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40668/0 (socket says 172.21.15.49:40668) 2023-07-18T03:38:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef75f9c700 1 -- 172.21.15.49:0/1116787578 learned_addr learned my addr 172.21.15.49:0/1116787578 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef7579b700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef700b1e00 0x7fef7015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef74f9a700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef700d1220 0x7fef701f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef75f9c700 1 -- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef700b1e00 msgr2=0x7fef7015cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef75f9c700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef700b1e00 0x7fef7015cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef75f9c700 1 -- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef700d1220 msgr2=0x7fef701f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef75f9c700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef700d1220 0x7fef701f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef75f9c700 1 -- 172.21.15.49:0/1116787578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef68009a70 con 0x7fef7015c950 2023-07-18T03:38:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef7579b700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef700b1e00 0x7fef7015cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef74f9a700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef700d1220 0x7fef701f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.634+0000 7fef75f9c700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7015c950 0x7fef701f57a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef68009f00 tx=0x7fef6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.635+0000 7fef65ffb700 1 -- 172.21.15.49:0/1116787578 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68009d70 con 0x7fef7015c950 2023-07-18T03:38:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.635+0000 7fef65ffb700 1 -- 172.21.15.49:0/1116787578 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6800f040 con 0x7fef7015c950 2023-07-18T03:38:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.635+0000 7fef65ffb700 1 -- 172.21.15.49:0/1116787578 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68014680 con 0x7fef7015c950 2023-07-18T03:38:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.635+0000 7fef779ff700 1 -- 172.21.15.49:0/1116787578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef600094a0 con 0x7fef7015c950 2023-07-18T03:38:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.635+0000 7fef779ff700 1 -- 172.21.15.49:0/1116787578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef70157750 con 0x7fef7015c950 2023-07-18T03:38:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.636+0000 7fef779ff700 1 -- 172.21.15.49:0/1116787578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7015e610 con 0x7fef7015c950 2023-07-18T03:38:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.637+0000 7fef65ffb700 1 -- 172.21.15.49:0/1116787578 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef680147e0 con 0x7fef7015c950 2023-07-18T03:38:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.638+0000 7fef65ffb700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fef5406ace0 0x7fef5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.638+0000 7fef65ffb700 1 -- 172.21.15.49:0/1116787578 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fef6808a3a0 con 0x7fef7015c950 2023-07-18T03:38:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.638+0000 7fef7579b700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fef5406ace0 0x7fef5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.639+0000 7fef7579b700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fef5406ace0 0x7fef5406d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0010e0 tx=0x7fef6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.642+0000 7fef65ffb700 1 -- 172.21.15.49:0/1116787578 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fef68058e40 con 0x7fef7015c950 2023-07-18T03:38:47.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.896+0000 7fef779ff700 1 -- 172.21.15.49:0/1116787578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fef7004e280 con 0x7fef7015c950 2023-07-18T03:38:47.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.897+0000 7fef65ffb700 1 -- 172.21.15.49:0/1116787578 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fef680589d0 con 0x7fef7015c950 2023-07-18T03:38:47.892 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:47.892 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-18T03:38:47.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.899+0000 7fef779ff700 1 -- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fef5406ace0 msgr2=0x7fef5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.899+0000 7fef779ff700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fef5406ace0 0x7fef5406d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0010e0 tx=0x7fef6c009040 comp rx=0 tx=0).stop 2023-07-18T03:38:47.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.899+0000 7fef779ff700 1 -- 172.21.15.49:0/1116787578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7015c950 msgr2=0x7fef701f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:47.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.899+0000 7fef779ff700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7015c950 0x7fef701f57a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef68009f00 tx=0x7fef6800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.899+0000 7fef779ff700 1 -- 172.21.15.49:0/1116787578 shutdown_connections 2023-07-18T03:38:47.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.899+0000 7fef779ff700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fef5406ace0 0x7fef5406d190 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.899+0000 7fef779ff700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fef7015c950 0x7fef701f57a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.899+0000 7fef779ff700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef700b1e00 0x7fef7015cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.899+0000 7fef779ff700 1 --2- 172.21.15.49:0/1116787578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef700d1220 0x7fef701f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:47.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.899+0000 7fef779ff700 1 -- 172.21.15.49:0/1116787578 >> 172.21.15.49:0/1116787578 conn(0x7fef7009c1d0 msgr2=0x7fef7006ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:47.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.899+0000 7fef779ff700 1 -- 172.21.15.49:0/1116787578 shutdown_connections 2023-07-18T03:38:47.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:47.899+0000 7fef779ff700 1 -- 172.21.15.49:0/1116787578 wait complete. 2023-07-18T03:38:47.912 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-07-18T03:38:47.913 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:48.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.123+0000 7fe6e95ff700 1 Processor -- start 2023-07-18T03:38:48.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.124+0000 7fe6e95ff700 1 -- start start 2023-07-18T03:38:48.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.125+0000 7fe6e95ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe6e40d1220 0x7fe6e40b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:48.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.125+0000 7fe6e95ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e40d2ce0 con 0x7fe6e415c950 2023-07-18T03:38:48.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.125+0000 7fe6e95ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e40b3350 con 0x7fe6e40b1e00 2023-07-18T03:38:48.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.125+0000 7fe6e95ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e40b3630 con 0x7fe6e40d1220 2023-07-18T03:38:48.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.125+0000 7fe6e2ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe6e415c950 0x7fe6e40d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41702/0 (socket says 172.21.15.49:41702) 2023-07-18T03:38:48.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.125+0000 7fe6e2ffd700 1 -- 172.21.15.49:0/1333151773 learned_addr learned my addr 172.21.15.49:0/1333151773 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:48.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.125+0000 7fe6dbfff700 1 --2- 172.21.15.49:0/1333151773 >> v2:172.21.15.107:3300/0 conn(0x7fe6e40d1220 0x7fe6e40b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:48.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.126+0000 7fe6e27fc700 1 -- 172.21.15.49:0/1333151773 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264077156 0 0) 0x7fe6e40d2ce0 con 0x7fe6e415c950 2023-07-18T03:38:48.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.126+0000 7fe6e27fc700 1 -- 172.21.15.49:0/1333151773 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6d00035b0 con 0x7fe6e415c950 2023-07-18T03:38:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.126+0000 7fe6e27fc700 1 -- 172.21.15.49:0/1333151773 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863800903 0 0) 0x7fe6e40b3350 con 0x7fe6e40b1e00 2023-07-18T03:38:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.126+0000 7fe6e27fc700 1 -- 172.21.15.49:0/1333151773 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6e40d2ce0 con 0x7fe6e40b1e00 2023-07-18T03:38:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.126+0000 7fe6dbfff700 1 -- 172.21.15.49:0/1333151773 >> v1:172.21.15.107:6789/0 conn(0x7fe6e40b1e00 legacy=0x7fe6e40afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.126+0000 7fe6dbfff700 1 -- 172.21.15.49:0/1333151773 >> v1:172.21.15.49:6789/0 conn(0x7fe6e415c950 legacy=0x7fe6e40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.126+0000 7fe6dbfff700 1 -- 172.21.15.49:0/1333151773 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e40b3910 con 0x7fe6e40d1220 2023-07-18T03:38:48.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.126+0000 7fe6dbfff700 1 --2- 172.21.15.49:0/1333151773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e40d1220 0x7fe6e40b43f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009840 tx=0x7fe6cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca0962e7346175 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:48.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.126+0000 7fe6e27fc700 1 -- 172.21.15.49:0/1333151773 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc011830 con 0x7fe6e40d1220 2023-07-18T03:38:48.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.126+0000 7fe6e27fc700 1 -- 172.21.15.49:0/1333151773 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6cc011990 con 0x7fe6e40d1220 2023-07-18T03:38:48.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.126+0000 7fe6e27fc700 1 -- 172.21.15.49:0/1333151773 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc011d20 con 0x7fe6e40d1220 2023-07-18T03:38:48.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.127+0000 7fe6e95ff700 1 -- 172.21.15.49:0/1333151773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e40d1220 msgr2=0x7fe6e40b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.127+0000 7fe6e95ff700 1 --2- 172.21.15.49:0/1333151773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e40d1220 0x7fe6e40b43f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009840 tx=0x7fe6cc009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:48.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.127+0000 7fe6e95ff700 1 -- 172.21.15.49:0/1333151773 shutdown_connections 2023-07-18T03:38:48.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.127+0000 7fe6e95ff700 1 --2- 172.21.15.49:0/1333151773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e40d1220 0x7fe6e40b43f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.127+0000 7fe6e95ff700 1 -- 172.21.15.49:0/1333151773 >> 172.21.15.49:0/1333151773 conn(0x7fe6e409c1d0 msgr2=0x7fe6e40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:48.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.127+0000 7fe6e95ff700 1 -- 172.21.15.49:0/1333151773 shutdown_connections 2023-07-18T03:38:48.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.127+0000 7fe6e95ff700 1 -- 172.21.15.49:0/1333151773 wait complete. 2023-07-18T03:38:48.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.130+0000 7fe6e95ff700 1 Processor -- start 2023-07-18T03:38:48.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.130+0000 7fe6e95ff700 1 -- start start 2023-07-18T03:38:48.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.131+0000 7fe6e95ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6e40b1e00 0x7fe6e40c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:48.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.131+0000 7fe6e95ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6e40d1220 0x7fe6e40c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:48.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.131+0000 7fe6e95ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e415c950 0x7fe6e40cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:48.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.131+0000 7fe6e95ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e4155090 con 0x7fe6e40b1e00 2023-07-18T03:38:48.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.131+0000 7fe6e95ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e40cba00 con 0x7fe6e415c950 2023-07-18T03:38:48.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.131+0000 7fe6e95ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e40cc9b0 con 0x7fe6e40d1220 2023-07-18T03:38:48.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.131+0000 7fe6e2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6e40b1e00 0x7fe6e40c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:48.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.131+0000 7fe6e2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6e40b1e00 0x7fe6e40c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40672/0 (socket says 172.21.15.49:40672) 2023-07-18T03:38:48.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.131+0000 7fe6e2ffd700 1 -- 172.21.15.49:0/186424898 learned_addr learned my addr 172.21.15.49:0/186424898 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:48.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6dbfff700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6e40d1220 0x7fe6e40c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:48.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6e37fe700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e415c950 0x7fe6e40cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:48.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6e2ffd700 1 -- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6e40d1220 msgr2=0x7fe6e40c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6e2ffd700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6e40d1220 0x7fe6e40c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6e2ffd700 1 -- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e415c950 msgr2=0x7fe6e40cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6e2ffd700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e415c950 0x7fe6e40cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6e2ffd700 1 -- 172.21.15.49:0/186424898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc0094a0 con 0x7fe6e40b1e00 2023-07-18T03:38:48.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6dbfff700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6e40d1220 0x7fe6e40c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:48.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6e37fe700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e415c950 0x7fe6e40cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:48.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6e2ffd700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6e40b1e00 0x7fe6e40c9200 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400c500 tx=0x7fe6d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:48.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6db7fe700 1 -- 172.21.15.49:0/186424898 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d40074e0 con 0x7fe6e40b1e00 2023-07-18T03:38:48.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6e95ff700 1 -- 172.21.15.49:0/186424898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e41f62d0 con 0x7fe6e40b1e00 2023-07-18T03:38:48.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6db7fe700 1 -- 172.21.15.49:0/186424898 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d4007640 con 0x7fe6e40b1e00 2023-07-18T03:38:48.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6db7fe700 1 -- 172.21.15.49:0/186424898 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4005a30 con 0x7fe6e40b1e00 2023-07-18T03:38:48.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.132+0000 7fe6e95ff700 1 -- 172.21.15.49:0/186424898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e41549d0 con 0x7fe6e40b1e00 2023-07-18T03:38:48.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.133+0000 7fe6e95ff700 1 -- 172.21.15.49:0/186424898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e41551d0 con 0x7fe6e40b1e00 2023-07-18T03:38:48.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.134+0000 7fe6db7fe700 1 -- 172.21.15.49:0/186424898 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe6d4005b90 con 0x7fe6e40b1e00 2023-07-18T03:38:48.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.135+0000 7fe6db7fe700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe6c406ace0 0x7fe6c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:48.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.135+0000 7fe6db7fe700 1 -- 172.21.15.49:0/186424898 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe6d4088870 con 0x7fe6e40b1e00 2023-07-18T03:38:48.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.136+0000 7fe6dbfff700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe6c406ace0 0x7fe6c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:48.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.136+0000 7fe6dbfff700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe6c406ace0 0x7fe6c406d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40d1610 tx=0x7fe6cc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:48.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.140+0000 7fe6db7fe700 1 -- 172.21.15.49:0/186424898 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe6e41551d0 con 0x7fe6e40b1e00 2023-07-18T03:38:48.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.351+0000 7fe6e95ff700 1 -- 172.21.15.49:0/186424898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6e41551d0 con 0x7fe6e40b1e00 2023-07-18T03:38:48.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.353+0000 7fe6db7fe700 1 -- 172.21.15.49:0/186424898 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18185 (secure 0 0 0) 0x7fe6e41551d0 con 0x7fe6e40b1e00 2023-07-18T03:38:48.348 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:48.349 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:37.510006+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:48.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.355+0000 7fe6e95ff700 1 -- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe6c406ace0 msgr2=0x7fe6c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.355+0000 7fe6e95ff700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe6c406ace0 0x7fe6c406d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40d1610 tx=0x7fe6cc01c040 comp rx=0 tx=0).stop 2023-07-18T03:38:48.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.355+0000 7fe6e95ff700 1 -- 172.21.15.49:0/186424898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6e40b1e00 msgr2=0x7fe6e40c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.355+0000 7fe6e95ff700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6e40b1e00 0x7fe6e40c9200 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400c500 tx=0x7fe6d400c8c0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.355+0000 7fe6e95ff700 1 -- 172.21.15.49:0/186424898 shutdown_connections 2023-07-18T03:38:48.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.355+0000 7fe6e95ff700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6e415c950 0x7fe6e40cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.355+0000 7fe6e95ff700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe6c406ace0 0x7fe6c406d190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.355+0000 7fe6e95ff700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6e40b1e00 0x7fe6e40c9200 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.355+0000 7fe6e95ff700 1 --2- 172.21.15.49:0/186424898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6e40d1220 0x7fe6e40c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.355+0000 7fe6e95ff700 1 -- 172.21.15.49:0/186424898 >> 172.21.15.49:0/186424898 conn(0x7fe6e409c1d0 msgr2=0x7fe6e40b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:48.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.355+0000 7fe6e95ff700 1 -- 172.21.15.49:0/186424898 shutdown_connections 2023-07-18T03:38:48.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.355+0000 7fe6e95ff700 1 -- 172.21.15.49:0/186424898 wait complete. 2023-07-18T03:38:48.368 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:38:33.773236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:38:35.023692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:38:48.369 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:38:48.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.580+0000 7fbce1bff700 1 Processor -- start 2023-07-18T03:38:48.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.581+0000 7fbce1bff700 1 -- start start 2023-07-18T03:38:48.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.582+0000 7fbce1bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbcdc0d0b10 0x7fbcdc0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:48.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.582+0000 7fbce1bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbcdc0b0e20 0x7fbcdc0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:48.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.582+0000 7fbce1bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcdc0b8e30 con 0x7fbcdc0b8a60 2023-07-18T03:38:48.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.582+0000 7fbce1bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcdc0d14d0 con 0x7fbcdc0d0b10 2023-07-18T03:38:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.582+0000 7fbce1bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc0afaa0 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.582+0000 7fbcdaffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbcdc0d0b10 0x7fbcdc0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.582+0000 7fbcdaffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbcdc0d0b10 0x7fbcdc0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40678/0 (socket says 172.21.15.49:40678) 2023-07-18T03:38:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.582+0000 7fbcdb7fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fbcdc0b8a60 0x7fbcdc0b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41708/0 (socket says 172.21.15.49:41708) 2023-07-18T03:38:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.582+0000 7fbcdaffd700 1 -- 172.21.15.49:0/4262385152 learned_addr learned my addr 172.21.15.49:0/4262385152 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:48.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.582+0000 7fbcdbfff700 1 --2- 172.21.15.49:0/4262385152 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbcdc0b0e20 0x7fbcdc0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:48.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.583+0000 7fbcdaffd700 1 -- 172.21.15.49:0/4262385152 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbcdc0b0e20 msgr2=0x7fbcdc0b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:48.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.583+0000 7fbcdaffd700 1 --2- 172.21.15.49:0/4262385152 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbcdc0b0e20 0x7fbcdc0b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.583+0000 7fbcdaffd700 1 -- 172.21.15.49:0/4262385152 >> v1:172.21.15.49:6789/0 conn(0x7fbcdc0b8a60 legacy=0x7fbcdc0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.583+0000 7fbcdaffd700 1 -- 172.21.15.49:0/4262385152 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc0afd80 con 0x7fbcdc0d0b10 2023-07-18T03:38:48.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.583+0000 7fbcdbfff700 1 --2- 172.21.15.49:0/4262385152 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbcdc0b0e20 0x7fbcdc0b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:48.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.583+0000 7fbcda7fc700 1 -- 172.21.15.49:0/4262385152 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244293627 0 0) 0x7fbcdc0b8e30 con 0x7fbcdc0b8a60 2023-07-18T03:38:48.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.583+0000 7fbcdaffd700 1 --2- 172.21.15.49:0/4262385152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc0d0b10 0x7fbcdc0d0f90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00098f0 tx=0x7fbcd0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c3ddb48f1335be server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:48.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.583+0000 7fbcda7fc700 1 -- 172.21.15.49:0/4262385152 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000b8a0 con 0x7fbcdc0d0b10 2023-07-18T03:38:48.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.583+0000 7fbcda7fc700 1 -- 172.21.15.49:0/4262385152 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd000ba00 con 0x7fbcdc0d0b10 2023-07-18T03:38:48.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.583+0000 7fbcda7fc700 1 -- 172.21.15.49:0/4262385152 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000bd90 con 0x7fbcdc0d0b10 2023-07-18T03:38:48.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.584+0000 7fbce1bff700 1 -- 172.21.15.49:0/4262385152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc0d0b10 msgr2=0x7fbcdc0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.584+0000 7fbce1bff700 1 --2- 172.21.15.49:0/4262385152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc0d0b10 0x7fbcdc0d0f90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00098f0 tx=0x7fbcd0009c00 comp rx=0 tx=0).stop 2023-07-18T03:38:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.584+0000 7fbce1bff700 1 -- 172.21.15.49:0/4262385152 shutdown_connections 2023-07-18T03:38:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.584+0000 7fbce1bff700 1 --2- 172.21.15.49:0/4262385152 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbcdc0b0e20 0x7fbcdc0b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.584+0000 7fbce1bff700 1 --2- 172.21.15.49:0/4262385152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc0d0b10 0x7fbcdc0d0f90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.584+0000 7fbce1bff700 1 -- 172.21.15.49:0/4262385152 >> 172.21.15.49:0/4262385152 conn(0x7fbcdc09c1b0 msgr2=0x7fbcdc0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.584+0000 7fbce1bff700 1 -- 172.21.15.49:0/4262385152 shutdown_connections 2023-07-18T03:38:48.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.584+0000 7fbce1bff700 1 -- 172.21.15.49:0/4262385152 wait complete. 2023-07-18T03:38:48.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.586+0000 7fbce1bff700 1 Processor -- start 2023-07-18T03:38:48.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.586+0000 7fbce1bff700 1 -- start start 2023-07-18T03:38:48.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbce1bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc0b0e20 0x7fbcdc149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:48.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbce1bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbcdc0b8a60 0x7fbcdc0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:48.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbce1bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcdc0d0b10 0x7fbcdc0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:48.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbce1bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc149e70 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbce1bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc0d0880 con 0x7fbcdc0d0b10 2023-07-18T03:38:48.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbce1bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc0cfd00 con 0x7fbcdc0b8a60 2023-07-18T03:38:48.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbcdb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc0b0e20 0x7fbcdc149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:48.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbcdb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc0b0e20 0x7fbcdc149930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40690/0 (socket says 172.21.15.49:40690) 2023-07-18T03:38:48.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbcdb7fe700 1 -- 172.21.15.49:0/110473218 learned_addr learned my addr 172.21.15.49:0/110473218 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:48.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbcdb7fe700 1 -- 172.21.15.49:0/110473218 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbcdc0b8a60 msgr2=0x7fbcdc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbcdb7fe700 1 --2- 172.21.15.49:0/110473218 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbcdc0b8a60 0x7fbcdc0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbcdb7fe700 1 -- 172.21.15.49:0/110473218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcdc0d0b10 msgr2=0x7fbcdc0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbcdb7fe700 1 --2- 172.21.15.49:0/110473218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcdc0d0b10 0x7fbcdc0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.587+0000 7fbcdb7fe700 1 -- 172.21.15.49:0/110473218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd0009550 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.588+0000 7fbcdb7fe700 1 --2- 172.21.15.49:0/110473218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc0b0e20 0x7fbcdc149930 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0011e0 tx=0x7fbccc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:48.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.588+0000 7fbcc3fff700 1 -- 172.21.15.49:0/110473218 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc004c80 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.588+0000 7fbcc3fff700 1 -- 172.21.15.49:0/110473218 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc004e00 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.588+0000 7fbcc3fff700 1 -- 172.21.15.49:0/110473218 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc007310 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.588+0000 7fbce1bff700 1 -- 172.21.15.49:0/110473218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcdc0cffe0 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.588+0000 7fbce1bff700 1 -- 172.21.15.49:0/110473218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcdc0d05a0 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.589+0000 7fbce1bff700 1 -- 172.21.15.49:0/110473218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcdc074350 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.590+0000 7fbcc3fff700 1 -- 172.21.15.49:0/110473218 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbccc007470 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.591+0000 7fbcc3fff700 1 --2- 172.21.15.49:0/110473218 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbcc806ace0 0x7fbcc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:48.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.591+0000 7fbcc3fff700 1 -- 172.21.15.49:0/110473218 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbccc089210 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.591+0000 7fbcdaffd700 1 --2- 172.21.15.49:0/110473218 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbcc806ace0 0x7fbcc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:48.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.592+0000 7fbcdaffd700 1 --2- 172.21.15.49:0/110473218 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbcc806ace0 0x7fbcc806d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0a0dc0 tx=0x7fbcd0010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:48.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.595+0000 7fbcc3fff700 1 -- 172.21.15.49:0/110473218 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbccc0542b0 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.799+0000 7fbce1bff700 1 -- 172.21.15.49:0/110473218 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbcdc0d1610 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.800+0000 7fbcc3fff700 1 -- 172.21.15.49:0/110473218 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbccc0578d0 con 0x7fbcdc0b0e20 2023-07-18T03:38:48.796 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:38:48.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.802+0000 7fbce1bff700 1 -- 172.21.15.49:0/110473218 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbcc806ace0 msgr2=0x7fbcc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.802+0000 7fbce1bff700 1 --2- 172.21.15.49:0/110473218 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbcc806ace0 0x7fbcc806d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0a0dc0 tx=0x7fbcd0010000 comp rx=0 tx=0).stop 2023-07-18T03:38:48.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.802+0000 7fbce1bff700 1 -- 172.21.15.49:0/110473218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc0b0e20 msgr2=0x7fbcdc149930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:48.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.802+0000 7fbce1bff700 1 --2- 172.21.15.49:0/110473218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc0b0e20 0x7fbcdc149930 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0011e0 tx=0x7fbccc00cae0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.802+0000 7fbce1bff700 1 -- 172.21.15.49:0/110473218 shutdown_connections 2023-07-18T03:38:48.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.802+0000 7fbce1bff700 1 --2- 172.21.15.49:0/110473218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcdc0d0b10 0x7fbcdc0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.802+0000 7fbce1bff700 1 --2- 172.21.15.49:0/110473218 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbcc806ace0 0x7fbcc806d190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.802+0000 7fbce1bff700 1 --2- 172.21.15.49:0/110473218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcdc0b0e20 0x7fbcdc149930 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.802+0000 7fbce1bff700 1 --2- 172.21.15.49:0/110473218 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbcdc0b8a60 0x7fbcdc0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:48.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.802+0000 7fbce1bff700 1 -- 172.21.15.49:0/110473218 >> 172.21.15.49:0/110473218 conn(0x7fbcdc09c1b0 msgr2=0x7fbcdc0b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:48.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.802+0000 7fbce1bff700 1 -- 172.21.15.49:0/110473218 shutdown_connections 2023-07-18T03:38:48.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:48.802+0000 7fbce1bff700 1 -- 172.21.15.49:0/110473218 wait complete. 2023-07-18T03:38:48.816 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:38:49.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.027+0000 7ffa92dff700 1 Processor -- start 2023-07-18T03:38:49.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.028+0000 7ffa92dff700 1 -- start start 2023-07-18T03:38:49.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.029+0000 7ffa92dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0d0b10 0x7ffa8c0d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.029+0000 7ffa92dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8c0d14a0 con 0x7ffa8c0b8a60 2023-07-18T03:38:49.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.029+0000 7ffa92dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8c0b8e30 con 0x7ffa8c0b0e20 2023-07-18T03:38:49.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.029+0000 7ffa92dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0afaa0 con 0x7ffa8c0d0b10 2023-07-18T03:38:49.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.029+0000 7ffa90b9b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ffa8c0b8a60 0x7ffa8c0b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35152/0 (socket says 172.21.15.49:35152) 2023-07-18T03:38:49.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.029+0000 7ffa90b9b700 1 -- 172.21.15.49:0/843542183 learned_addr learned my addr 172.21.15.49:0/843542183 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:49.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.029+0000 7ffa8bfff700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0d0b10 0x7ffa8c0d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.029+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446222462 0 0) 0x7ffa80001110 con 0x7ffa8c0b8a60 2023-07-18T03:38:49.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa7c0035b0 con 0x7ffa8c0b8a60 2023-07-18T03:38:49.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513674474 0 0) 0x7ffa8c0b8e30 con 0x7ffa8c0b0e20 2023-07-18T03:38:49.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa80001110 con 0x7ffa8c0b0e20 2023-07-18T03:38:49.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 968110089 0 0) 0x7ffa7c0035b0 con 0x7ffa8c0b8a60 2023-07-18T03:38:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0d0b10 msgr2=0x7ffa8c0d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0d0b10 0x7ffa8c0d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 >> v1:172.21.15.107:6789/0 conn(0x7ffa8c0b0e20 legacy=0x7ffa8c0b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c0afd80 con 0x7ffa8c0b8a60 2023-07-18T03:38:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7ffa80001800 con 0x7ffa8c0b8a60 2023-07-18T03:38:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8bfff700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0d0b10 0x7ffa8c0d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:49.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 >> v1:172.21.15.49:6789/0 conn(0x7ffa8c0b8a60 legacy=0x7ffa8c0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0d0b10 0x7ffa7c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa7c004150 0x7ffa7c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa7c004d90 0x7ffa7c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0afaa0 con 0x7ffa7c004150 2023-07-18T03:38:49.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c00baf0 con 0x7ffa7c004d90 2023-07-18T03:38:49.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c00bcb0 con 0x7ffa8c0d0b10 2023-07-18T03:38:49.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa8bfff700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa7c004150 0x7ffa7c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa90b9b700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0d0b10 0x7ffa7c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.030+0000 7ffa9139c700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa7c004d90 0x7ffa7c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.031+0000 7ffa8bfff700 1 -- 172.21.15.49:0/843542183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0d0b10 msgr2=0x7ffa7c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:49.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.031+0000 7ffa8bfff700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0d0b10 0x7ffa7c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.031+0000 7ffa8bfff700 1 -- 172.21.15.49:0/843542183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa7c004d90 msgr2=0x7ffa7c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:49.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.031+0000 7ffa8bfff700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa7c004d90 0x7ffa7c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.031+0000 7ffa8bfff700 1 -- 172.21.15.49:0/843542183 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa780094a0 con 0x7ffa7c004150 2023-07-18T03:38:49.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.031+0000 7ffa8bfff700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa7c004150 0x7ffa7c004850 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009d10 tx=0x7ffa7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9350448b284953d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:49.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.031+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78009e60 con 0x7ffa7c004150 2023-07-18T03:38:49.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.031+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7800e9a0 con 0x7ffa7c004150 2023-07-18T03:38:49.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.031+0000 7ffa8b7fe700 1 -- 172.21.15.49:0/843542183 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800ed50 con 0x7ffa7c004150 2023-07-18T03:38:49.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.032+0000 7ffa92dff700 1 -- 172.21.15.49:0/843542183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa7c004150 msgr2=0x7ffa7c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.032+0000 7ffa92dff700 1 --2- 172.21.15.49:0/843542183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa7c004150 0x7ffa7c004850 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009d10 tx=0x7ffa7c005880 comp rx=0 tx=0).stop 2023-07-18T03:38:49.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.032+0000 7ffa9139c700 1 -- 172.21.15.49:0/843542183 reap_dead start 2023-07-18T03:38:49.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.032+0000 7ffa92dff700 1 -- 172.21.15.49:0/843542183 shutdown_connections 2023-07-18T03:38:49.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.032+0000 7ffa92dff700 1 -- 172.21.15.49:0/843542183 >> 172.21.15.49:0/843542183 conn(0x7ffa8c09c1b0 msgr2=0x7ffa8c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:49.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.032+0000 7ffa92dff700 1 -- 172.21.15.49:0/843542183 shutdown_connections 2023-07-18T03:38:49.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.032+0000 7ffa92dff700 1 -- 172.21.15.49:0/843542183 wait complete. 2023-07-18T03:38:49.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.034+0000 7ffa92dff700 1 Processor -- start 2023-07-18T03:38:49.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.034+0000 7ffa92dff700 1 -- start start 2023-07-18T03:38:49.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa92dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa7c004150 0x7ffa8c0cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa92dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa8c1553f0 0x7ffa8c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa92dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c15f2c0 0x7ffa8c15f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa92dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c150c80 con 0x7ffa8c1553f0 2023-07-18T03:38:49.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa92dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c15fc70 con 0x7ffa7c004150 2023-07-18T03:38:49.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa92dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c154f60 con 0x7ffa8c15f2c0 2023-07-18T03:38:49.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa8bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa8c1553f0 0x7ffa8c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa8bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa8c1553f0 0x7ffa8c0cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34592/0 (socket says 172.21.15.49:34592) 2023-07-18T03:38:49.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa8bfff700 1 -- 172.21.15.49:0/2513347820 learned_addr learned my addr 172.21.15.49:0/2513347820 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:49.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa9139c700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c15f2c0 0x7ffa8c15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa90b9b700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa7c004150 0x7ffa8c0cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa8bfff700 1 -- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c15f2c0 msgr2=0x7ffa8c15f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.035+0000 7ffa8bfff700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c15f2c0 0x7ffa8c15f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.036+0000 7ffa8bfff700 1 -- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa7c004150 msgr2=0x7ffa8c0cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.036+0000 7ffa8bfff700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa7c004150 0x7ffa8c0cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.036+0000 7ffa8bfff700 1 -- 172.21.15.49:0/2513347820 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c154370 con 0x7ffa8c1553f0 2023-07-18T03:38:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.036+0000 7ffa90b9b700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa7c004150 0x7ffa8c0cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.036+0000 7ffa8bfff700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa8c1553f0 0x7ffa8c0cd550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800e790 tx=0x7ffa780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.036+0000 7ffa897fa700 1 -- 172.21.15.49:0/2513347820 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78020070 con 0x7ffa8c1553f0 2023-07-18T03:38:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.036+0000 7ffa897fa700 1 -- 172.21.15.49:0/2513347820 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa78009e60 con 0x7ffa8c1553f0 2023-07-18T03:38:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.036+0000 7ffa92dff700 1 -- 172.21.15.49:0/2513347820 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8c154630 con 0x7ffa8c1553f0 2023-07-18T03:38:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.036+0000 7ffa897fa700 1 -- 172.21.15.49:0/2513347820 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78003af0 con 0x7ffa8c1553f0 2023-07-18T03:38:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.036+0000 7ffa92dff700 1 -- 172.21.15.49:0/2513347820 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c154bf0 con 0x7ffa8c1553f0 2023-07-18T03:38:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.037+0000 7ffa92dff700 1 -- 172.21.15.49:0/2513347820 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c154d80 con 0x7ffa8c1553f0 2023-07-18T03:38:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.038+0000 7ffa897fa700 1 -- 172.21.15.49:0/2513347820 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffa78003c50 con 0x7ffa8c1553f0 2023-07-18T03:38:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.039+0000 7ffa897fa700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa7006ace0 0x7ffa7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.040+0000 7ffa897fa700 1 -- 172.21.15.49:0/2513347820 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffa7808a590 con 0x7ffa8c1553f0 2023-07-18T03:38:49.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.040+0000 7ffa90b9b700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa7006ace0 0x7ffa7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.041+0000 7ffa90b9b700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa7006ace0 0x7ffa7006d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa80000c00 tx=0x7ffa80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:49.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.043+0000 7ffa897fa700 1 -- 172.21.15.49:0/2513347820 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffa780555b0 con 0x7ffa8c1553f0 2023-07-18T03:38:49.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.251+0000 7ffa92dff700 1 -- 172.21.15.49:0/2513347820 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa8c074350 con 0x7ffa8c1553f0 2023-07-18T03:38:49.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.252+0000 7ffa897fa700 1 -- 172.21.15.49:0/2513347820 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa78058bd0 con 0x7ffa8c1553f0 2023-07-18T03:38:49.247 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:38:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.254+0000 7ffa92dff700 1 -- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa7006ace0 msgr2=0x7ffa7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.254+0000 7ffa92dff700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa7006ace0 0x7ffa7006d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa80000c00 tx=0x7ffa80009040 comp rx=0 tx=0).stop 2023-07-18T03:38:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.254+0000 7ffa92dff700 1 -- 172.21.15.49:0/2513347820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa8c1553f0 msgr2=0x7ffa8c0cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.254+0000 7ffa92dff700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa8c1553f0 0x7ffa8c0cd550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800e790 tx=0x7ffa780047c0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.254+0000 7ffa92dff700 1 -- 172.21.15.49:0/2513347820 shutdown_connections 2023-07-18T03:38:49.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.254+0000 7ffa92dff700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c15f2c0 0x7ffa8c15f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.254+0000 7ffa92dff700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa7c004150 0x7ffa8c0cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.254+0000 7ffa92dff700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa7006ace0 0x7ffa7006d190 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.254+0000 7ffa92dff700 1 --2- 172.21.15.49:0/2513347820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa8c1553f0 0x7ffa8c0cd550 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.254+0000 7ffa92dff700 1 -- 172.21.15.49:0/2513347820 >> 172.21.15.49:0/2513347820 conn(0x7ffa8c09c1b0 msgr2=0x7ffa8c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:49.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.254+0000 7ffa92dff700 1 -- 172.21.15.49:0/2513347820 shutdown_connections 2023-07-18T03:38:49.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.254+0000 7ffa92dff700 1 -- 172.21.15.49:0/2513347820 wait complete. 2023-07-18T03:38:49.267 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:38:49.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.480+0000 7f13039ff700 1 Processor -- start 2023-07-18T03:38:49.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.480+0000 7f13039ff700 1 -- start start 2023-07-18T03:38:49.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.481+0000 7f13039ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f12fc0b8a60 0x7f12fc0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.481+0000 7f13039ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12fc0b4530 0x7f12fc0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.481+0000 7f13039ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f12fc0d1050 0x7f12fc0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.481+0000 7f13039ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12fc0d14d0 con 0x7f12fc0d1050 2023-07-18T03:38:49.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.481+0000 7f13039ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12fc0b8e30 con 0x7f12fc0b8a60 2023-07-18T03:38:49.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.481+0000 7f13039ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0afaa0 con 0x7f12fc0b4530 2023-07-18T03:38:49.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.481+0000 7f1301f9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f12fc0d1050 0x7f12fc0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.481+0000 7f1301f9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f12fc0d1050 0x7f12fc0b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34602/0 (socket says 172.21.15.49:34602) 2023-07-18T03:38:49.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.481+0000 7f1301f9c700 1 -- 172.21.15.49:0/3524501294 learned_addr learned my addr 172.21.15.49:0/3524501294 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:49.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.481+0000 7f1300f9a700 1 --2- 172.21.15.49:0/3524501294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12fc0b4530 0x7f12fc0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.481+0000 7f130179b700 1 --2- 172.21.15.49:0/3524501294 >> v2:172.21.15.107:3300/0 conn(0x7f12fc0b8a60 0x7f12fc0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.482+0000 7f1301f9c700 1 -- 172.21.15.49:0/3524501294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12fc0b4530 msgr2=0x7f12fc0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:49.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.482+0000 7f1301f9c700 1 --2- 172.21.15.49:0/3524501294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12fc0b4530 0x7f12fc0d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.482+0000 7f1301f9c700 1 -- 172.21.15.49:0/3524501294 >> v2:172.21.15.107:3300/0 conn(0x7f12fc0b8a60 msgr2=0x7f12fc0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:49.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.482+0000 7f1301f9c700 1 --2- 172.21.15.49:0/3524501294 >> v2:172.21.15.107:3300/0 conn(0x7f12fc0b8a60 0x7f12fc0b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.482+0000 7f1301f9c700 1 -- 172.21.15.49:0/3524501294 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc0afd80 con 0x7f12fc0d1050 2023-07-18T03:38:49.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.482+0000 7f1301f9c700 1 --2- 172.21.15.49:0/3524501294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12fc0d1050 0x7f12fc0b25d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0b5b00 tx=0x7f12f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef22438ca87c338a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:49.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.482+0000 7f12f3fff700 1 -- 172.21.15.49:0/3524501294 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f400e070 con 0x7f12fc0d1050 2023-07-18T03:38:49.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.482+0000 7f12f3fff700 1 -- 172.21.15.49:0/3524501294 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f400b910 con 0x7f12fc0d1050 2023-07-18T03:38:49.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.482+0000 7f12f3fff700 1 -- 172.21.15.49:0/3524501294 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f400bcc0 con 0x7f12fc0d1050 2023-07-18T03:38:49.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.483+0000 7f13039ff700 1 -- 172.21.15.49:0/3524501294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12fc0d1050 msgr2=0x7f12fc0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.483+0000 7f13039ff700 1 --2- 172.21.15.49:0/3524501294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12fc0d1050 0x7f12fc0b25d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0b5b00 tx=0x7f12f400b330 comp rx=0 tx=0).stop 2023-07-18T03:38:49.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.483+0000 7f13039ff700 1 -- 172.21.15.49:0/3524501294 shutdown_connections 2023-07-18T03:38:49.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.483+0000 7f13039ff700 1 --2- 172.21.15.49:0/3524501294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12fc0d1050 0x7f12fc0b25d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.483+0000 7f13039ff700 1 --2- 172.21.15.49:0/3524501294 >> v2:172.21.15.107:3300/0 conn(0x7f12fc0b8a60 0x7f12fc0b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.483+0000 7f13039ff700 1 --2- 172.21.15.49:0/3524501294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12fc0b4530 0x7f12fc0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.483+0000 7f13039ff700 1 -- 172.21.15.49:0/3524501294 >> 172.21.15.49:0/3524501294 conn(0x7f12fc09c1b0 msgr2=0x7f12fc0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:49.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.483+0000 7f13039ff700 1 -- 172.21.15.49:0/3524501294 shutdown_connections 2023-07-18T03:38:49.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.483+0000 7f13039ff700 1 -- 172.21.15.49:0/3524501294 wait complete. 2023-07-18T03:38:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.485+0000 7f13039ff700 1 Processor -- start 2023-07-18T03:38:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.485+0000 7f13039ff700 1 -- start start 2023-07-18T03:38:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.486+0000 7f13039ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12fc0b4530 0x7f12fc15a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.486+0000 7f13039ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc0b8a60 0x7f12fc1f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.486+0000 7f13039ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12fc0d1050 0x7f12fc06b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.486+0000 7f13039ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc1561e0 con 0x7f12fc0b4530 2023-07-18T03:38:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.486+0000 7f13039ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0cc250 con 0x7f12fc0b8a60 2023-07-18T03:38:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.486+0000 7f13039ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc1f6350 con 0x7f12fc0d1050 2023-07-18T03:38:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.486+0000 7f130179b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12fc0b4530 0x7f12fc15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.486+0000 7f130179b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12fc0b4530 0x7f12fc15a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34606/0 (socket says 172.21.15.49:34606) 2023-07-18T03:38:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.486+0000 7f130179b700 1 -- 172.21.15.49:0/3736613768 learned_addr learned my addr 172.21.15.49:0/3736613768 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.486+0000 7f1300f9a700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc0b8a60 0x7f12fc1f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.486+0000 7f1301f9c700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12fc0d1050 0x7f12fc06b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.487+0000 7f130179b700 1 -- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12fc0d1050 msgr2=0x7f12fc06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.487+0000 7f130179b700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12fc0d1050 0x7f12fc06b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.487+0000 7f130179b700 1 -- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc0b8a60 msgr2=0x7f12fc1f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.487+0000 7f130179b700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc0b8a60 0x7f12fc1f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.487+0000 7f130179b700 1 -- 172.21.15.49:0/3736613768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f4009b30 con 0x7f12fc0b4530 2023-07-18T03:38:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.487+0000 7f130179b700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12fc0b4530 0x7f12fc15a490 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f12f800e9b0 tx=0x7f12f800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.487+0000 7f12f1ffb700 1 -- 172.21.15.49:0/3736613768 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8009710 con 0x7f12fc0b4530 2023-07-18T03:38:49.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.487+0000 7f12f1ffb700 1 -- 172.21.15.49:0/3736613768 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f800ee50 con 0x7f12fc0b4530 2023-07-18T03:38:49.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.487+0000 7f12f1ffb700 1 -- 172.21.15.49:0/3736613768 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f800ca50 con 0x7f12fc0b4530 2023-07-18T03:38:49.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.487+0000 7f13039ff700 1 -- 172.21.15.49:0/3736613768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc1f6630 con 0x7f12fc0b4530 2023-07-18T03:38:49.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.487+0000 7f13039ff700 1 -- 172.21.15.49:0/3736613768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc159c90 con 0x7f12fc0b4530 2023-07-18T03:38:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.488+0000 7f13039ff700 1 -- 172.21.15.49:0/3736613768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc074350 con 0x7f12fc0b4530 2023-07-18T03:38:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.489+0000 7f12f1ffb700 1 -- 172.21.15.49:0/3736613768 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f12f800cbb0 con 0x7f12fc0b4530 2023-07-18T03:38:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.491+0000 7f12f1ffb700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f12e006acb0 0x7f12e006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.491+0000 7f12f1ffb700 1 -- 172.21.15.49:0/3736613768 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f12f80893d0 con 0x7f12fc0b4530 2023-07-18T03:38:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.491+0000 7f1300f9a700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f12e006acb0 0x7f12e006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.492+0000 7f1300f9a700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f12e006acb0 0x7f12e006d160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0d08a0 tx=0x7f12ec009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.495+0000 7f12f1ffb700 1 -- 172.21.15.49:0/3736613768 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f12f8057f00 con 0x7f12fc0b4530 2023-07-18T03:38:49.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.701+0000 7f13039ff700 1 -- 172.21.15.49:0/3736613768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12fc1e55e0 con 0x7f12fc0b4530 2023-07-18T03:38:49.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.702+0000 7f12f1ffb700 1 -- 172.21.15.49:0/3736613768 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12f800ce60 con 0x7f12fc0b4530 2023-07-18T03:38:49.698 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:38:49.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.704+0000 7f13039ff700 1 -- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f12e006acb0 msgr2=0x7f12e006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.705+0000 7f13039ff700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f12e006acb0 0x7f12e006d160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0d08a0 tx=0x7f12ec009210 comp rx=0 tx=0).stop 2023-07-18T03:38:49.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.705+0000 7f13039ff700 1 -- 172.21.15.49:0/3736613768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12fc0b4530 msgr2=0x7f12fc15a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.705+0000 7f13039ff700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12fc0b4530 0x7f12fc15a490 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f12f800e9b0 tx=0x7f12f800ecc0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.705+0000 7f13039ff700 1 -- 172.21.15.49:0/3736613768 shutdown_connections 2023-07-18T03:38:49.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.705+0000 7f13039ff700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12fc0d1050 0x7f12fc06b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.705+0000 7f13039ff700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f12e006acb0 0x7f12e006d160 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.705+0000 7f13039ff700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12fc0b4530 0x7f12fc15a490 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.705+0000 7f13039ff700 1 --2- 172.21.15.49:0/3736613768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc0b8a60 0x7f12fc1f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.705+0000 7f13039ff700 1 -- 172.21.15.49:0/3736613768 >> 172.21.15.49:0/3736613768 conn(0x7f12fc09c1b0 msgr2=0x7f12fc0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:49.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.705+0000 7f13039ff700 1 -- 172.21.15.49:0/3736613768 shutdown_connections 2023-07-18T03:38:49.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.705+0000 7f13039ff700 1 -- 172.21.15.49:0/3736613768 wait complete. 2023-07-18T03:38:49.718 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:38:49.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.930+0000 7fa97f3ff700 1 Processor -- start 2023-07-18T03:38:49.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.930+0000 7fa97f3ff700 1 -- start start 2023-07-18T03:38:49.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97f3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa9780b8a60 0x7fa9780b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97f3ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa9780b4530 0x7fa9780d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97f3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9780d1490 con 0x7fa9780d1050 2023-07-18T03:38:49.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97f3ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9780b8e30 con 0x7fa9780b4530 2023-07-18T03:38:49.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97f3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9780b1e30 con 0x7fa9780b8a60 2023-07-18T03:38:49.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97c99a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa9780b4530 0x7fa9780d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97c99a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa9780b4530 0x7fa9780d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34610/0 (socket says 172.21.15.49:34610) 2023-07-18T03:38:49.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97c99a700 1 -- 172.21.15.49:0/3545495768 learned_addr learned my addr 172.21.15.49:0/3545495768 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:49.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97d19b700 1 --2- 172.21.15.49:0/3545495768 >> v2:172.21.15.107:3300/0 conn(0x7fa9780b8a60 0x7fa9780b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97c99a700 1 -- 172.21.15.49:0/3545495768 >> v2:172.21.15.107:3300/0 conn(0x7fa9780b8a60 msgr2=0x7fa9780b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97c99a700 1 --2- 172.21.15.49:0/3545495768 >> v2:172.21.15.107:3300/0 conn(0x7fa9780b8a60 0x7fa9780b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97c99a700 1 -- 172.21.15.49:0/3545495768 >> v1:172.21.15.107:6789/0 conn(0x7fa9780d1050 legacy=0x7fa9780b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.932+0000 7fa97c99a700 1 -- 172.21.15.49:0/3545495768 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9780afaa0 con 0x7fa9780b4530 2023-07-18T03:38:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.933+0000 7fa97c99a700 1 --2- 172.21.15.49:0/3545495768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9780b4530 0x7fa9780d0b10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa9780b5b00 tx=0x7fa968009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542dd0cb4717da45 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.933+0000 7fa96ffff700 1 -- 172.21.15.49:0/3545495768 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968011830 con 0x7fa9780b4530 2023-07-18T03:38:49.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.933+0000 7fa96ffff700 1 -- 172.21.15.49:0/3545495768 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa968011990 con 0x7fa9780b4530 2023-07-18T03:38:49.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.933+0000 7fa96ffff700 1 -- 172.21.15.49:0/3545495768 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968011d20 con 0x7fa9780b4530 2023-07-18T03:38:49.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.934+0000 7fa97f3ff700 1 -- 172.21.15.49:0/3545495768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9780b4530 msgr2=0x7fa9780d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.934+0000 7fa97f3ff700 1 --2- 172.21.15.49:0/3545495768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9780b4530 0x7fa9780d0b10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa9780b5b00 tx=0x7fa968009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.934+0000 7fa97f3ff700 1 -- 172.21.15.49:0/3545495768 shutdown_connections 2023-07-18T03:38:49.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.934+0000 7fa97f3ff700 1 --2- 172.21.15.49:0/3545495768 >> v2:172.21.15.107:3300/0 conn(0x7fa9780b8a60 0x7fa9780b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.934+0000 7fa97f3ff700 1 --2- 172.21.15.49:0/3545495768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9780b4530 0x7fa9780d0b10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.934+0000 7fa97f3ff700 1 -- 172.21.15.49:0/3545495768 >> 172.21.15.49:0/3545495768 conn(0x7fa97809c1b0 msgr2=0x7fa9780d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:49.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.934+0000 7fa97f3ff700 1 -- 172.21.15.49:0/3545495768 shutdown_connections 2023-07-18T03:38:49.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.934+0000 7fa97f3ff700 1 -- 172.21.15.49:0/3545495768 wait complete. 2023-07-18T03:38:49.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.935+0000 7fa97f3ff700 1 Processor -- start 2023-07-18T03:38:49.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.936+0000 7fa97f3ff700 1 -- start start 2023-07-18T03:38:49.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97f3ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9780b4530 0x7fa9780ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97f3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9780b8a60 0x7fa9780cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97f3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9780d1050 0x7fa9780cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97f3ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97815f590 con 0x7fa9780b4530 2023-07-18T03:38:49.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97f3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97815fac0 con 0x7fa9780b8a60 2023-07-18T03:38:49.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97f3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97815f7e0 con 0x7fa9780d1050 2023-07-18T03:38:49.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97d19b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9780b4530 0x7fa9780ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97d19b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9780b4530 0x7fa9780ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34618/0 (socket says 172.21.15.49:34618) 2023-07-18T03:38:49.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97d19b700 1 -- 172.21.15.49:0/2254044903 learned_addr learned my addr 172.21.15.49:0/2254044903 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:49.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97d99c700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9780d1050 0x7fa9780cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97c99a700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9780b8a60 0x7fa9780cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97d19b700 1 -- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9780d1050 msgr2=0x7fa9780cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97d19b700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9780d1050 0x7fa9780cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97d19b700 1 -- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9780b8a60 msgr2=0x7fa9780cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97d19b700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9780b8a60 0x7fa9780cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97d19b700 1 -- 172.21.15.49:0/2254044903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa968009550 con 0x7fa9780b4530 2023-07-18T03:38:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.937+0000 7fa97d19b700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9780b4530 0x7fa9780ca010 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa97400e9e0 tx=0x7fa97400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.938+0000 7fa96dffb700 1 -- 172.21.15.49:0/2254044903 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974009710 con 0x7fa9780b4530 2023-07-18T03:38:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.938+0000 7fa96dffb700 1 -- 172.21.15.49:0/2254044903 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97400c850 con 0x7fa9780b4530 2023-07-18T03:38:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.938+0000 7fa96dffb700 1 -- 172.21.15.49:0/2254044903 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97400cbc0 con 0x7fa9780b4530 2023-07-18T03:38:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.938+0000 7fa97f3ff700 1 -- 172.21.15.49:0/2254044903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9781f4000 con 0x7fa9780b4530 2023-07-18T03:38:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.938+0000 7fa97f3ff700 1 -- 172.21.15.49:0/2254044903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9780ca550 con 0x7fa9780b4530 2023-07-18T03:38:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.939+0000 7fa97f3ff700 1 -- 172.21.15.49:0/2254044903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa978074350 con 0x7fa9780b4530 2023-07-18T03:38:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.940+0000 7fa96dffb700 1 -- 172.21.15.49:0/2254044903 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa97400cd20 con 0x7fa9780b4530 2023-07-18T03:38:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.941+0000 7fa96dffb700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa95c06ace0 0x7fa95c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.941+0000 7fa96dffb700 1 -- 172.21.15.49:0/2254044903 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa974089510 con 0x7fa9780b4530 2023-07-18T03:38:49.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.941+0000 7fa97c99a700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa95c06ace0 0x7fa95c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:49.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.942+0000 7fa97c99a700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa95c06ace0 0x7fa95c06d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa968009c10 tx=0x7fa968008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:49.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:49.945+0000 7fa96dffb700 1 -- 172.21.15.49:0/2254044903 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa9740580e0 con 0x7fa9780b4530 2023-07-18T03:38:50.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.154+0000 7fa97f3ff700 1 -- 172.21.15.49:0/2254044903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9780b4310 con 0x7fa9780b4530 2023-07-18T03:38:50.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.154+0000 7fa96dffb700 1 -- 172.21.15.49:0/2254044903 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa974057c70 con 0x7fa9780b4530 2023-07-18T03:38:50.150 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:38:50.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.157+0000 7fa97f3ff700 1 -- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa95c06ace0 msgr2=0x7fa95c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:50.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.157+0000 7fa97f3ff700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa95c06ace0 0x7fa95c06d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa968009c10 tx=0x7fa968008030 comp rx=0 tx=0).stop 2023-07-18T03:38:50.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.157+0000 7fa97f3ff700 1 -- 172.21.15.49:0/2254044903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9780b4530 msgr2=0x7fa9780ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:50.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.157+0000 7fa97f3ff700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9780b4530 0x7fa9780ca010 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa97400e9e0 tx=0x7fa97400eda0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.157+0000 7fa97f3ff700 1 -- 172.21.15.49:0/2254044903 shutdown_connections 2023-07-18T03:38:50.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.157+0000 7fa97f3ff700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9780d1050 0x7fa9780cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.157+0000 7fa97f3ff700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa95c06ace0 0x7fa95c06d190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.157+0000 7fa97f3ff700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9780b4530 0x7fa9780ca010 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.157+0000 7fa97f3ff700 1 --2- 172.21.15.49:0/2254044903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9780b8a60 0x7fa9780cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.157+0000 7fa97f3ff700 1 -- 172.21.15.49:0/2254044903 >> 172.21.15.49:0/2254044903 conn(0x7fa97809c1b0 msgr2=0x7fa9780b7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:50.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.157+0000 7fa97f3ff700 1 -- 172.21.15.49:0/2254044903 shutdown_connections 2023-07-18T03:38:50.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.157+0000 7fa97f3ff700 1 -- 172.21.15.49:0/2254044903 wait complete. 2023-07-18T03:38:50.170 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:38:50.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.380+0000 7f9111bff700 1 Processor -- start 2023-07-18T03:38:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.380+0000 7f9111bff700 1 -- start start 2023-07-18T03:38:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.381+0000 7f9111bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f910c0b3e90 0x7f910c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.381+0000 7f9111bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f910c15a8c0 0x7f910c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.381+0000 7f9111bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910c0ce110 con 0x7f910c15e890 2023-07-18T03:38:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.381+0000 7f9111bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910c0cf380 con 0x7f910c15a8c0 2023-07-18T03:38:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.381+0000 7f9111bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910c15b660 con 0x7f910c0b3e90 2023-07-18T03:38:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.381+0000 7f910bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f910c15a8c0 0x7f910c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.381+0000 7f910bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f910c15a8c0 0x7f910c15ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34630/0 (socket says 172.21.15.49:34630) 2023-07-18T03:38:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.381+0000 7f910bfff700 1 -- 172.21.15.49:0/404809027 learned_addr learned my addr 172.21.15.49:0/404809027 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.382+0000 7f910affd700 1 --2- 172.21.15.49:0/404809027 >> v2:172.21.15.107:3300/0 conn(0x7f910c0b3e90 0x7f910c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.382+0000 7f910bfff700 1 -- 172.21.15.49:0/404809027 >> v2:172.21.15.107:3300/0 conn(0x7f910c0b3e90 msgr2=0x7f910c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.382+0000 7f910bfff700 1 --2- 172.21.15.49:0/404809027 >> v2:172.21.15.107:3300/0 conn(0x7f910c0b3e90 0x7f910c0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.382+0000 7f910bfff700 1 -- 172.21.15.49:0/404809027 >> v1:172.21.15.107:6789/0 conn(0x7f910c15e890 legacy=0x7f910c0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.382+0000 7f910bfff700 1 -- 172.21.15.49:0/404809027 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c0ce360 con 0x7f910c15a8c0 2023-07-18T03:38:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.382+0000 7f910bfff700 1 --2- 172.21.15.49:0/404809027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c15a8c0 0x7f910c15ad20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f910c06b730 tx=0x7f910400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9d8a1743371205 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.382+0000 7f910a7fc700 1 -- 172.21.15.49:0/404809027 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910400e070 con 0x7f910c15a8c0 2023-07-18T03:38:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.382+0000 7f910a7fc700 1 -- 172.21.15.49:0/404809027 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910400b9c0 con 0x7f910c15a8c0 2023-07-18T03:38:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.382+0000 7f910a7fc700 1 -- 172.21.15.49:0/404809027 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910400bd70 con 0x7f910c15a8c0 2023-07-18T03:38:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.383+0000 7f9111bff700 1 -- 172.21.15.49:0/404809027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c15a8c0 msgr2=0x7f910c15ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.383+0000 7f9111bff700 1 --2- 172.21.15.49:0/404809027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c15a8c0 0x7f910c15ad20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f910c06b730 tx=0x7f910400b330 comp rx=0 tx=0).stop 2023-07-18T03:38:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.383+0000 7f9111bff700 1 -- 172.21.15.49:0/404809027 shutdown_connections 2023-07-18T03:38:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.383+0000 7f9111bff700 1 --2- 172.21.15.49:0/404809027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c15a8c0 0x7f910c15ad20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.383+0000 7f9111bff700 1 --2- 172.21.15.49:0/404809027 >> v2:172.21.15.107:3300/0 conn(0x7f910c0b3e90 0x7f910c0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.383+0000 7f9111bff700 1 -- 172.21.15.49:0/404809027 >> 172.21.15.49:0/404809027 conn(0x7f910c069f30 msgr2=0x7f910c06a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.383+0000 7f9111bff700 1 -- 172.21.15.49:0/404809027 shutdown_connections 2023-07-18T03:38:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.383+0000 7f9111bff700 1 -- 172.21.15.49:0/404809027 wait complete. 2023-07-18T03:38:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.385+0000 7f9111bff700 1 Processor -- start 2023-07-18T03:38:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.385+0000 7f9111bff700 1 -- start start 2023-07-18T03:38:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.386+0000 7f9111bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c0b3e90 0x7f910c150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.386+0000 7f9111bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f910c15a8c0 0x7f910c1e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.386+0000 7f9111bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c15e890 0x7f910c159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.386+0000 7f9111bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c150d60 con 0x7f910c0b3e90 2023-07-18T03:38:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.386+0000 7f9111bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c070970 con 0x7f910c15e890 2023-07-18T03:38:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.386+0000 7f9111bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910c070c50 con 0x7f910c15a8c0 2023-07-18T03:38:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.386+0000 7f910b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c0b3e90 0x7f910c150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.386+0000 7f910b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c0b3e90 0x7f910c150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34640/0 (socket says 172.21.15.49:34640) 2023-07-18T03:38:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.386+0000 7f910b7fe700 1 -- 172.21.15.49:0/3115834965 learned_addr learned my addr 172.21.15.49:0/3115834965 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f910bfff700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c15e890 0x7f910c159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f910affd700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f910c15a8c0 0x7f910c1e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f910b7fe700 1 -- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f910c15a8c0 msgr2=0x7f910c1e3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f910b7fe700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f910c15a8c0 0x7f910c1e3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f910b7fe700 1 -- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c15e890 msgr2=0x7f910c159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f910b7fe700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c15e890 0x7f910c159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f910b7fe700 1 -- 172.21.15.49:0/3115834965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9104009b30 con 0x7f910c0b3e90 2023-07-18T03:38:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f910bfff700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c15e890 0x7f910c159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f910affd700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f910c15a8c0 0x7f910c1e3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f910b7fe700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c0b3e90 0x7f910c150820 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c4e0 tx=0x7f90fc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f90f3fff700 1 -- 172.21.15.49:0/3115834965 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc0052d0 con 0x7f910c0b3e90 2023-07-18T03:38:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f90f3fff700 1 -- 172.21.15.49:0/3115834965 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc005430 con 0x7f910c0b3e90 2023-07-18T03:38:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f90f3fff700 1 -- 172.21.15.49:0/3115834965 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc0038c0 con 0x7f910c0b3e90 2023-07-18T03:38:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f9111bff700 1 -- 172.21.15.49:0/3115834965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910c070f30 con 0x7f910c0b3e90 2023-07-18T03:38:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.387+0000 7f9111bff700 1 -- 172.21.15.49:0/3115834965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910c0714f0 con 0x7f910c0b3e90 2023-07-18T03:38:50.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.388+0000 7f9111bff700 1 -- 172.21.15.49:0/3115834965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910c066f00 con 0x7f910c0b3e90 2023-07-18T03:38:50.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.389+0000 7f90f3fff700 1 -- 172.21.15.49:0/3115834965 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f90fc003a20 con 0x7f910c0b3e90 2023-07-18T03:38:50.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.390+0000 7f90f3fff700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f90f806ace0 0x7f90f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:50.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.390+0000 7f90f3fff700 1 -- 172.21.15.49:0/3115834965 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f90fc088940 con 0x7f910c0b3e90 2023-07-18T03:38:50.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.390+0000 7f910affd700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f90f806ace0 0x7f90f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:50.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.391+0000 7f910affd700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f90f806ace0 0x7f90f806d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f910c0add30 tx=0x7f9100008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:50.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.395+0000 7f90f3fff700 1 -- 172.21.15.49:0/3115834965 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f90fc0539e0 con 0x7f910c0b3e90 2023-07-18T03:38:50.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.645+0000 7f9111bff700 1 -- 172.21.15.49:0/3115834965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f910c0cdcc0 con 0x7f910c0b3e90 2023-07-18T03:38:50.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.645+0000 7f90f3fff700 1 -- 172.21.15.49:0/3115834965 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4709 (secure 0 0 0) 0x7f90fc057000 con 0x7f910c0b3e90 2023-07-18T03:38:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:50.640 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1477016053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1477016053},{"type":"v1","addr":"172.21.15.107:6835","nonce":1477016053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2998451500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2998451500},{"type":"v1","addr":"172.21.15.49:6835","nonce":2998451500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4247281109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4247281109},{"type":"v1","addr":"172.21.15.49:6833","nonce":4247281109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:38:36.025442+0000","modified":"2023-07-18T03:38:39.535929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1127901907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1127901907},{"type":"v1","addr":"172.21.15.107:6837","nonce":1127901907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:38:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.647+0000 7f9111bff700 1 -- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f90f806ace0 msgr2=0x7f90f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:50.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.647+0000 7f9111bff700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f90f806ace0 0x7f90f806d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f910c0add30 tx=0x7f9100008040 comp rx=0 tx=0).stop 2023-07-18T03:38:50.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.647+0000 7f9111bff700 1 -- 172.21.15.49:0/3115834965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c0b3e90 msgr2=0x7f910c150820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:50.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.647+0000 7f9111bff700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c0b3e90 0x7f910c150820 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c4e0 tx=0x7f90fc00c8a0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.647+0000 7f9111bff700 1 -- 172.21.15.49:0/3115834965 shutdown_connections 2023-07-18T03:38:50.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.647+0000 7f9111bff700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f910c15e890 0x7f910c159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.647+0000 7f9111bff700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f90f806ace0 0x7f90f806d190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.647+0000 7f9111bff700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910c0b3e90 0x7f910c150820 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.647+0000 7f9111bff700 1 --2- 172.21.15.49:0/3115834965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f910c15a8c0 0x7f910c1e3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.647+0000 7f9111bff700 1 -- 172.21.15.49:0/3115834965 >> 172.21.15.49:0/3115834965 conn(0x7f910c069f30 msgr2=0x7f910c0b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.648+0000 7f9111bff700 1 -- 172.21.15.49:0/3115834965 shutdown_connections 2023-07-18T03:38:50.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.648+0000 7f9111bff700 1 -- 172.21.15.49:0/3115834965 wait complete. 2023-07-18T03:38:50.645 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2023-07-18T03:38:50.661 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:50.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.869+0000 7f75681ff700 1 Processor -- start 2023-07-18T03:38:50.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.870+0000 7f75681ff700 1 -- start start 2023-07-18T03:38:50.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.871+0000 7f75681ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f756015c950 0x7f75600b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:50.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.871+0000 7f75681ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f75600d1220 0x7f75600afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:50.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.871+0000 7f75681ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75600d2ce0 con 0x7f75600b1e00 2023-07-18T03:38:50.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.871+0000 7f75681ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75600b3350 con 0x7f75600d1220 2023-07-18T03:38:50.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.871+0000 7f75681ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75600b3630 con 0x7f756015c950 2023-07-18T03:38:50.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.871+0000 7f756679c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f75600d1220 0x7f75600afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:50.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.871+0000 7f756679c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f75600d1220 0x7f75600afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34646/0 (socket says 172.21.15.49:34646) 2023-07-18T03:38:50.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.871+0000 7f7565f9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f756015c950 0x7f75600b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:50.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.871+0000 7f756679c700 1 -- 172.21.15.49:0/1481247555 learned_addr learned my addr 172.21.15.49:0/1481247555 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:50.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.871+0000 7f756579a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f75600b1e00 0x7f75600d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35166/0 (socket says 172.21.15.49:35166) 2023-07-18T03:38:50.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.872+0000 7f756679c700 1 -- 172.21.15.49:0/1481247555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f756015c950 msgr2=0x7f75600b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:50.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.872+0000 7f756679c700 1 --2- 172.21.15.49:0/1481247555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f756015c950 0x7f75600b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.872+0000 7f756679c700 1 -- 172.21.15.49:0/1481247555 >> v1:172.21.15.49:6789/0 conn(0x7f75600b1e00 legacy=0x7f75600d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:50.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.872+0000 7f756679c700 1 -- 172.21.15.49:0/1481247555 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75600b3910 con 0x7f75600d1220 2023-07-18T03:38:50.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.872+0000 7f7564f99700 1 -- 172.21.15.49:0/1481247555 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566613407 0 0) 0x7f75600d2ce0 con 0x7f75600b1e00 2023-07-18T03:38:50.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.872+0000 7f756679c700 1 --2- 172.21.15.49:0/1481247555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75600d1220 0x7f75600afac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7558009e50 tx=0x7f755800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370ffc14189dd58f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:50.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.872+0000 7f7564f99700 1 -- 172.21.15.49:0/1481247555 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755800e070 con 0x7f75600d1220 2023-07-18T03:38:50.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.872+0000 7f7564f99700 1 -- 172.21.15.49:0/1481247555 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755800b910 con 0x7f75600d1220 2023-07-18T03:38:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.872+0000 7f7564f99700 1 -- 172.21.15.49:0/1481247555 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755800bcc0 con 0x7f75600d1220 2023-07-18T03:38:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.873+0000 7f75681ff700 1 -- 172.21.15.49:0/1481247555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75600d1220 msgr2=0x7f75600afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.873+0000 7f75681ff700 1 --2- 172.21.15.49:0/1481247555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75600d1220 0x7f75600afac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7558009e50 tx=0x7f755800b330 comp rx=0 tx=0).stop 2023-07-18T03:38:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.873+0000 7f75681ff700 1 -- 172.21.15.49:0/1481247555 shutdown_connections 2023-07-18T03:38:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.873+0000 7f75681ff700 1 --2- 172.21.15.49:0/1481247555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75600d1220 0x7f75600afac0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.873+0000 7f75681ff700 1 --2- 172.21.15.49:0/1481247555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f756015c950 0x7f75600b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.873+0000 7f75681ff700 1 -- 172.21.15.49:0/1481247555 >> 172.21.15.49:0/1481247555 conn(0x7f756009c1d0 msgr2=0x7f75600d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:50.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.873+0000 7f75681ff700 1 -- 172.21.15.49:0/1481247555 shutdown_connections 2023-07-18T03:38:50.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.873+0000 7f75681ff700 1 -- 172.21.15.49:0/1481247555 wait complete. 2023-07-18T03:38:50.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.875+0000 7f75681ff700 1 Processor -- start 2023-07-18T03:38:50.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.875+0000 7f75681ff700 1 -- start start 2023-07-18T03:38:50.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f75681ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75600b1e00 0x7f7560155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:50.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f75681ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75600d1220 0x7f75600b8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:50.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f75681ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f756015c950 0x7f75600b8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:50.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f75681ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7560156450 con 0x7f75600d1220 2023-07-18T03:38:50.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f75681ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75600cb1d0 con 0x7f756015c950 2023-07-18T03:38:50.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f75681ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75600cb4b0 con 0x7f75600b1e00 2023-07-18T03:38:50.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f756579a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75600d1220 0x7f75600b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:50.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f756579a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75600d1220 0x7f75600b8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34656/0 (socket says 172.21.15.49:34656) 2023-07-18T03:38:50.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f756579a700 1 -- 172.21.15.49:0/3248602177 learned_addr learned my addr 172.21.15.49:0/3248602177 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:50.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f7565f9b700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75600b1e00 0x7f7560155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:50.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f756679c700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f756015c950 0x7f75600b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:50.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f756579a700 1 -- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75600b1e00 msgr2=0x7f7560155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:50.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f756579a700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75600b1e00 0x7f7560155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.876+0000 7f756579a700 1 -- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f756015c950 msgr2=0x7f75600b8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:50.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.877+0000 7f756579a700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f756015c950 0x7f75600b8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:50.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.877+0000 7f756579a700 1 -- 172.21.15.49:0/3248602177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7558009b30 con 0x7f75600d1220 2023-07-18T03:38:50.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.877+0000 7f7565f9b700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75600b1e00 0x7f7560155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:50.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.877+0000 7f756579a700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75600d1220 0x7f75600b8230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f75500011e0 tx=0x7f755000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:50.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.877+0000 7f75567fc700 1 -- 172.21.15.49:0/3248602177 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550004c80 con 0x7f75600d1220 2023-07-18T03:38:50.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.877+0000 7f75567fc700 1 -- 172.21.15.49:0/3248602177 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7550004e00 con 0x7f75600d1220 2023-07-18T03:38:50.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.877+0000 7f75567fc700 1 -- 172.21.15.49:0/3248602177 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550007310 con 0x7f75600d1220 2023-07-18T03:38:50.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.877+0000 7f75681ff700 1 -- 172.21.15.49:0/3248602177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75600ce4f0 con 0x7f75600d1220 2023-07-18T03:38:50.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.877+0000 7f75681ff700 1 -- 172.21.15.49:0/3248602177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75600ceab0 con 0x7f75600d1220 2023-07-18T03:38:50.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.878+0000 7f75681ff700 1 -- 172.21.15.49:0/3248602177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75601584c0 con 0x7f75600d1220 2023-07-18T03:38:50.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.879+0000 7f75567fc700 1 -- 172.21.15.49:0/3248602177 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7550007470 con 0x7f75600d1220 2023-07-18T03:38:50.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.880+0000 7f75567fc700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f754406ace0 0x7f754406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:50.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.880+0000 7f75567fc700 1 -- 172.21.15.49:0/3248602177 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7550089260 con 0x7f75600d1220 2023-07-18T03:38:50.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.880+0000 7f7565f9b700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f754406ace0 0x7f754406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:50.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.881+0000 7f7565f9b700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f754406ace0 0x7f754406d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f755c009b70 tx=0x7f755c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:50.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:50.884+0000 7f75567fc700 1 -- 172.21.15.49:0/3248602177 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7550054300 con 0x7f75600d1220 2023-07-18T03:38:51.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.093+0000 7f75681ff700 1 -- 172.21.15.49:0/3248602177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f756004e280 con 0x7f75600d1220 2023-07-18T03:38:51.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.094+0000 7f75567fc700 1 -- 172.21.15.49:0/3248602177 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f7550057920 con 0x7f75600d1220 2023-07-18T03:38:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:51.090 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:37.510006+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:51.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.097+0000 7f75681ff700 1 -- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f754406ace0 msgr2=0x7f754406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.097+0000 7f75681ff700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f754406ace0 0x7f754406d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f755c009b70 tx=0x7f755c008040 comp rx=0 tx=0).stop 2023-07-18T03:38:51.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.097+0000 7f75681ff700 1 -- 172.21.15.49:0/3248602177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75600d1220 msgr2=0x7f75600b8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.097+0000 7f75681ff700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75600d1220 0x7f75600b8230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f75500011e0 tx=0x7f755000cae0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.097+0000 7f75681ff700 1 -- 172.21.15.49:0/3248602177 shutdown_connections 2023-07-18T03:38:51.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.097+0000 7f75681ff700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f756015c950 0x7f75600b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.097+0000 7f75681ff700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75600b1e00 0x7f7560155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.097+0000 7f75681ff700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f754406ace0 0x7f754406d190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.097+0000 7f75681ff700 1 --2- 172.21.15.49:0/3248602177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75600d1220 0x7f75600b8230 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.097+0000 7f75681ff700 1 -- 172.21.15.49:0/3248602177 >> 172.21.15.49:0/3248602177 conn(0x7f756009c1d0 msgr2=0x7f756006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:51.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.097+0000 7f75681ff700 1 -- 172.21.15.49:0/3248602177 shutdown_connections 2023-07-18T03:38:51.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.097+0000 7f75681ff700 1 -- 172.21.15.49:0/3248602177 wait complete. 2023-07-18T03:38:51.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:38:33.773236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:38:35.023692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:38:51.111 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:38:51.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.325+0000 7fbab91ff700 1 Processor -- start 2023-07-18T03:38:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.325+0000 7fbab91ff700 1 -- start start 2023-07-18T03:38:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.327+0000 7fbab91ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbab40d0b10 0x7fbab40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.327+0000 7fbab91ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab40b8e30 con 0x7fbab40b0e20 2023-07-18T03:38:51.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.327+0000 7fbab91ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab40afaa0 con 0x7fbab40b8a60 2023-07-18T03:38:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.327+0000 7fbab91ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab40d14d0 con 0x7fbab40d0b10 2023-07-18T03:38:51.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.327+0000 7fbaabfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbab40d0b10 0x7fbab40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.327+0000 7fbab359e700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fbab40b0e20 0x7fbab40b1260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35172/0 (socket says 172.21.15.49:35172) 2023-07-18T03:38:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.327+0000 7fbab359e700 1 -- 172.21.15.49:0/3514870006 learned_addr learned my addr 172.21.15.49:0/3514870006 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.327+0000 7fbaabfff700 1 -- 172.21.15.49:0/3514870006 >> v1:172.21.15.107:6789/0 conn(0x7fbab40b8a60 legacy=0x7fbab40b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.327+0000 7fbaabfff700 1 -- 172.21.15.49:0/3514870006 >> v1:172.21.15.49:6789/0 conn(0x7fbab40b0e20 legacy=0x7fbab40b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.327+0000 7fbaabfff700 1 -- 172.21.15.49:0/3514870006 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab40afd80 con 0x7fbab40d0b10 2023-07-18T03:38:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.328+0000 7fbaabfff700 1 --2- 172.21.15.49:0/3514870006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbab40d0b10 0x7fbab40d0f90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009840 tx=0x7fba9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8b702384b353f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:51.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.328+0000 7fbab259c700 1 -- 172.21.15.49:0/3514870006 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c011830 con 0x7fbab40d0b10 2023-07-18T03:38:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.328+0000 7fbab259c700 1 -- 172.21.15.49:0/3514870006 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c011990 con 0x7fbab40d0b10 2023-07-18T03:38:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.328+0000 7fbab259c700 1 -- 172.21.15.49:0/3514870006 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c011d20 con 0x7fbab40d0b10 2023-07-18T03:38:51.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.328+0000 7fbab91ff700 1 -- 172.21.15.49:0/3514870006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbab40d0b10 msgr2=0x7fbab40d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.328+0000 7fbab91ff700 1 --2- 172.21.15.49:0/3514870006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbab40d0b10 0x7fbab40d0f90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009840 tx=0x7fba9c009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.329+0000 7fbab91ff700 1 -- 172.21.15.49:0/3514870006 shutdown_connections 2023-07-18T03:38:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.329+0000 7fbab91ff700 1 --2- 172.21.15.49:0/3514870006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbab40d0b10 0x7fbab40d0f90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.329+0000 7fbab91ff700 1 -- 172.21.15.49:0/3514870006 >> 172.21.15.49:0/3514870006 conn(0x7fbab409c1b0 msgr2=0x7fbab40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.329+0000 7fbab91ff700 1 -- 172.21.15.49:0/3514870006 shutdown_connections 2023-07-18T03:38:51.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.329+0000 7fbab91ff700 1 -- 172.21.15.49:0/3514870006 wait complete. 2023-07-18T03:38:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.330+0000 7fbab91ff700 1 Processor -- start 2023-07-18T03:38:51.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.331+0000 7fbab91ff700 1 -- start start 2023-07-18T03:38:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.331+0000 7fbab91ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbab40b0e20 0x7fbab41f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.331+0000 7fbab91ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbab40b8a60 0x7fbab4150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab91ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbab40d0b10 0x7fbab415f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab91ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab41f6130 con 0x7fbab40d0b10 2023-07-18T03:38:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab91ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab41f5890 con 0x7fbab40b0e20 2023-07-18T03:38:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab91ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab41f5b70 con 0x7fbab40b8a60 2023-07-18T03:38:51.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbab40d0b10 0x7fbab415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab2d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbab40b0e20 0x7fbab41f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab2d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbab40b0e20 0x7fbab41f5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:55326/0 (socket says 172.21.15.49:55326) 2023-07-18T03:38:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab2d9d700 1 -- 172.21.15.49:0/574673542 learned_addr learned my addr 172.21.15.49:0/574673542 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:51.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbaabfff700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbab40b8a60 0x7fbab4150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab359e700 1 -- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbab40b8a60 msgr2=0x7fbab4150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab359e700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbab40b8a60 0x7fbab4150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab359e700 1 -- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbab40b0e20 msgr2=0x7fbab41f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab359e700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbab40b0e20 0x7fbab41f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab359e700 1 -- 172.21.15.49:0/574673542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9c0094a0 con 0x7fbab40d0b10 2023-07-18T03:38:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.332+0000 7fbab2d9d700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbab40b0e20 0x7fbab41f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.333+0000 7fbab359e700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbab40d0b10 0x7fbab415f200 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00caa0 tx=0x7fbaac00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.333+0000 7fbaab7fe700 1 -- 172.21.15.49:0/574673542 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac00a9a0 con 0x7fbab40d0b10 2023-07-18T03:38:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.333+0000 7fbaab7fe700 1 -- 172.21.15.49:0/574673542 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac00ab00 con 0x7fbab40d0b10 2023-07-18T03:38:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.333+0000 7fbaab7fe700 1 -- 172.21.15.49:0/574673542 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac00ae70 con 0x7fbab40d0b10 2023-07-18T03:38:51.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.333+0000 7fbab91ff700 1 -- 172.21.15.49:0/574673542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab41f5e50 con 0x7fbab40d0b10 2023-07-18T03:38:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.333+0000 7fbab91ff700 1 -- 172.21.15.49:0/574673542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab4151260 con 0x7fbab40d0b10 2023-07-18T03:38:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.334+0000 7fbab91ff700 1 -- 172.21.15.49:0/574673542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab40ceb20 con 0x7fbab40d0b10 2023-07-18T03:38:51.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.335+0000 7fbaab7fe700 1 -- 172.21.15.49:0/574673542 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbaac01b3b0 con 0x7fbab40d0b10 2023-07-18T03:38:51.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.336+0000 7fbaab7fe700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fba9406ace0 0x7fba9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.336+0000 7fbaab7fe700 1 -- 172.21.15.49:0/574673542 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbaac088eb0 con 0x7fbab40d0b10 2023-07-18T03:38:51.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.336+0000 7fbab2d9d700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fba9406ace0 0x7fba9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.337+0000 7fbab2d9d700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fba9406ace0 0x7fba9406d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000f60 tx=0x7fbaa4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:51.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.340+0000 7fbaab7fe700 1 -- 172.21.15.49:0/574673542 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbab40ceb20 con 0x7fbab40d0b10 2023-07-18T03:38:51.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.546+0000 7fbab91ff700 1 -- 172.21.15.49:0/574673542 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbab40ceb20 con 0x7fbab40d0b10 2023-07-18T03:38:51.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.546+0000 7fbaab7fe700 1 -- 172.21.15.49:0/574673542 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbaac017020 con 0x7fbab40d0b10 2023-07-18T03:38:51.541 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:38:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.549+0000 7fbab91ff700 1 -- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fba9406ace0 msgr2=0x7fba9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.549+0000 7fbab91ff700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fba9406ace0 0x7fba9406d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000f60 tx=0x7fbaa4009040 comp rx=0 tx=0).stop 2023-07-18T03:38:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.549+0000 7fbab91ff700 1 -- 172.21.15.49:0/574673542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbab40d0b10 msgr2=0x7fbab415f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.549+0000 7fbab91ff700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbab40d0b10 0x7fbab415f200 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00caa0 tx=0x7fbaac00cdb0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.549+0000 7fbab91ff700 1 -- 172.21.15.49:0/574673542 shutdown_connections 2023-07-18T03:38:51.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.549+0000 7fbab91ff700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fba9406ace0 0x7fba9406d190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.549+0000 7fbab91ff700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbab40d0b10 0x7fbab415f200 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.549+0000 7fbab91ff700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbab40b0e20 0x7fbab41f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.549+0000 7fbab91ff700 1 --2- 172.21.15.49:0/574673542 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbab40b8a60 0x7fbab4150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.549+0000 7fbab91ff700 1 -- 172.21.15.49:0/574673542 >> 172.21.15.49:0/574673542 conn(0x7fbab409c1b0 msgr2=0x7fbab406b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:51.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.549+0000 7fbab91ff700 1 -- 172.21.15.49:0/574673542 shutdown_connections 2023-07-18T03:38:51.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.549+0000 7fbab91ff700 1 -- 172.21.15.49:0/574673542 wait complete. 2023-07-18T03:38:51.562 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:38:51.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.776+0000 7fe1f17ff700 1 Processor -- start 2023-07-18T03:38:51.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.776+0000 7fe1f17ff700 1 -- start start 2023-07-18T03:38:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.777+0000 7fe1f17ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe1ec0b8a60 0x7fe1ec0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.777+0000 7fe1f17ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1ec0b8e30 con 0x7fe1ec0b4530 2023-07-18T03:38:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.777+0000 7fe1f17ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1ec0afaa0 con 0x7fe1ec0d1220 2023-07-18T03:38:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.777+0000 7fe1f17ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1ec0afd80 con 0x7fe1ec0b8a60 2023-07-18T03:38:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.777+0000 7fe1ea7fc700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe1ec0b4530 0x7fe1ec0d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35186/0 (socket says 172.21.15.49:35186) 2023-07-18T03:38:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.777+0000 7fe1ea7fc700 1 -- 172.21.15.49:0/3526232767 learned_addr learned my addr 172.21.15.49:0/3526232767 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.777+0000 7fe1eaffd700 1 --2- 172.21.15.49:0/3526232767 >> v2:172.21.15.107:3300/0 conn(0x7fe1ec0b8a60 0x7fe1ec0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.778+0000 7fe1e9ffb700 1 -- 172.21.15.49:0/3526232767 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129441482 0 0) 0x7fe1ec0b8e30 con 0x7fe1ec0b4530 2023-07-18T03:38:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.778+0000 7fe1e9ffb700 1 -- 172.21.15.49:0/3526232767 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1d40035b0 con 0x7fe1ec0b4530 2023-07-18T03:38:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.778+0000 7fe1e9ffb700 1 -- 172.21.15.49:0/3526232767 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015023478 0 0) 0x7fe1ec0afaa0 con 0x7fe1ec0d1220 2023-07-18T03:38:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.778+0000 7fe1e9ffb700 1 -- 172.21.15.49:0/3526232767 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1ec0b8e30 con 0x7fe1ec0d1220 2023-07-18T03:38:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.778+0000 7fe1eaffd700 1 -- 172.21.15.49:0/3526232767 >> v1:172.21.15.107:6789/0 conn(0x7fe1ec0d1220 legacy=0x7fe1ec0b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.778+0000 7fe1eaffd700 1 -- 172.21.15.49:0/3526232767 >> v1:172.21.15.49:6789/0 conn(0x7fe1ec0b4530 legacy=0x7fe1ec0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.778+0000 7fe1eaffd700 1 -- 172.21.15.49:0/3526232767 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec0b0060 con 0x7fe1ec0b8a60 2023-07-18T03:38:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.778+0000 7fe1eaffd700 1 --2- 172.21.15.49:0/3526232767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1ec0b8a60 0x7fe1ec0b3ff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009840 tx=0x7fe1dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0dd539041bf89d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.778+0000 7fe1e9ffb700 1 -- 172.21.15.49:0/3526232767 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc011830 con 0x7fe1ec0b8a60 2023-07-18T03:38:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.778+0000 7fe1e9ffb700 1 -- 172.21.15.49:0/3526232767 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc011990 con 0x7fe1ec0b8a60 2023-07-18T03:38:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.779+0000 7fe1e9ffb700 1 -- 172.21.15.49:0/3526232767 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc011d20 con 0x7fe1ec0b8a60 2023-07-18T03:38:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.779+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3526232767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1ec0b8a60 msgr2=0x7fe1ec0b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.779+0000 7fe1f17ff700 1 --2- 172.21.15.49:0/3526232767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1ec0b8a60 0x7fe1ec0b3ff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009840 tx=0x7fe1dc009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.779+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3526232767 shutdown_connections 2023-07-18T03:38:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.779+0000 7fe1f17ff700 1 --2- 172.21.15.49:0/3526232767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1ec0b8a60 0x7fe1ec0b3ff0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.779+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3526232767 >> 172.21.15.49:0/3526232767 conn(0x7fe1ec09c1b0 msgr2=0x7fe1ec0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.779+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3526232767 shutdown_connections 2023-07-18T03:38:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.779+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3526232767 wait complete. 2023-07-18T03:38:51.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.781+0000 7fe1f17ff700 1 Processor -- start 2023-07-18T03:38:51.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.781+0000 7fe1f17ff700 1 -- start start 2023-07-18T03:38:51.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.782+0000 7fe1f17ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1ec0b4530 0x7fe1ec15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.782+0000 7fe1f17ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1ec0b8a60 0x7fe1ec0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.782+0000 7fe1f17ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1ec0d1220 0x7fe1ec0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.782+0000 7fe1f17ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec15dc50 con 0x7fe1ec0b8a60 2023-07-18T03:38:51.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.782+0000 7fe1f17ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec15df30 con 0x7fe1ec0d1220 2023-07-18T03:38:51.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.782+0000 7fe1f17ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec15e210 con 0x7fe1ec0b4530 2023-07-18T03:38:51.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.782+0000 7fe1ea7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1ec0b8a60 0x7fe1ec0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.782+0000 7fe1ea7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1ec0b8a60 0x7fe1ec0cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34694/0 (socket says 172.21.15.49:34694) 2023-07-18T03:38:51.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.782+0000 7fe1ea7fc700 1 -- 172.21.15.49:0/3719570499 learned_addr learned my addr 172.21.15.49:0/3719570499 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.782+0000 7fe1eaffd700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1ec0b4530 0x7fe1ec15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.782+0000 7fe1eb7fe700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1ec0d1220 0x7fe1ec0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.783+0000 7fe1eaffd700 1 -- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1ec0d1220 msgr2=0x7fe1ec0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.783+0000 7fe1eaffd700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1ec0d1220 0x7fe1ec0cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.783+0000 7fe1eaffd700 1 -- 172.21.15.49:0/3719570499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1ec0b8a60 msgr2=0x7fe1ec0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.783+0000 7fe1eaffd700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1ec0b8a60 0x7fe1ec0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.783+0000 7fe1eaffd700 1 -- 172.21.15.49:0/3719570499 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0094a0 con 0x7fe1ec0b4530 2023-07-18T03:38:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.783+0000 7fe1ea7fc700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1ec0b8a60 0x7fe1ec0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.783+0000 7fe1eaffd700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1ec0b4530 0x7fe1ec15ad50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc016000 tx=0x7fe1dc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.783+0000 7fe1d37fe700 1 -- 172.21.15.49:0/3719570499 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc01a6e0 con 0x7fe1ec0b4530 2023-07-18T03:38:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.783+0000 7fe1d37fe700 1 -- 172.21.15.49:0/3719570499 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc01a860 con 0x7fe1ec0b4530 2023-07-18T03:38:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.784+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3719570499 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ec15e4f0 con 0x7fe1ec0b4530 2023-07-18T03:38:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.784+0000 7fe1d37fe700 1 -- 172.21.15.49:0/3719570499 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc01ac10 con 0x7fe1ec0b4530 2023-07-18T03:38:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.784+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3719570499 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ec15b290 con 0x7fe1ec0b4530 2023-07-18T03:38:51.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.785+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3719570499 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ec074350 con 0x7fe1ec0b4530 2023-07-18T03:38:51.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.786+0000 7fe1d37fe700 1 -- 172.21.15.49:0/3719570499 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe1dc022070 con 0x7fe1ec0b4530 2023-07-18T03:38:51.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.787+0000 7fe1d37fe700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1d806ace0 0x7fe1d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.787+0000 7fe1d37fe700 1 -- 172.21.15.49:0/3719570499 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1dc08a770 con 0x7fe1ec0b4530 2023-07-18T03:38:51.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.787+0000 7fe1ea7fc700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1d806ace0 0x7fe1d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.788+0000 7fe1ea7fc700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1d806ace0 0x7fe1d806d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec1f57e0 tx=0x7fe1e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:51.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.791+0000 7fe1d37fe700 1 -- 172.21.15.49:0/3719570499 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe1dc055920 con 0x7fe1ec0b4530 2023-07-18T03:38:51.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.997+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3719570499 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1ec0d23e0 con 0x7fe1ec0b4530 2023-07-18T03:38:51.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:51.998+0000 7fe1d37fe700 1 -- 172.21.15.49:0/3719570499 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1dc058f40 con 0x7fe1ec0b4530 2023-07-18T03:38:51.993 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:38:51.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.000+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1d806ace0 msgr2=0x7fe1d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.000+0000 7fe1f17ff700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1d806ace0 0x7fe1d806d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec1f57e0 tx=0x7fe1e0009040 comp rx=0 tx=0).stop 2023-07-18T03:38:51.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.000+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1ec0b4530 msgr2=0x7fe1ec15ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.000+0000 7fe1f17ff700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1ec0b4530 0x7fe1ec15ad50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc016000 tx=0x7fe1dc004750 comp rx=0 tx=0).stop 2023-07-18T03:38:51.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.000+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3719570499 shutdown_connections 2023-07-18T03:38:51.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.000+0000 7fe1f17ff700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1ec0d1220 0x7fe1ec0cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.000+0000 7fe1f17ff700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1ec0b4530 0x7fe1ec15ad50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.000+0000 7fe1f17ff700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1d806ace0 0x7fe1d806d190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.000+0000 7fe1f17ff700 1 --2- 172.21.15.49:0/3719570499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1ec0b8a60 0x7fe1ec0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.000+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3719570499 >> 172.21.15.49:0/3719570499 conn(0x7fe1ec09c1b0 msgr2=0x7fe1ec0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:51.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.000+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3719570499 shutdown_connections 2023-07-18T03:38:51.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.000+0000 7fe1f17ff700 1 -- 172.21.15.49:0/3719570499 wait complete. 2023-07-18T03:38:52.013 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:38:52.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.227+0000 7f968bbff700 1 Processor -- start 2023-07-18T03:38:52.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.227+0000 7f968bbff700 1 -- start start 2023-07-18T03:38:52.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.228+0000 7f968bbff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f96840b8a60 0x7f96840b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.228+0000 7f968bbff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f96840b4530 0x7f96840d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.228+0000 7f968bbff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96840d1490 con 0x7f96840d1050 2023-07-18T03:38:52.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.228+0000 7f968bbff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96840b8e30 con 0x7f96840b8a60 2023-07-18T03:38:52.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.229+0000 7f968bbff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96840b1e30 con 0x7f96840b4530 2023-07-18T03:38:52.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.229+0000 7f968999b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f96840b8a60 0x7f96840b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.229+0000 7f968a19c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f96840d1050 0x7f96840b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35196/0 (socket says 172.21.15.49:35196) 2023-07-18T03:38:52.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.229+0000 7f968999b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f96840b8a60 0x7f96840b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34700/0 (socket says 172.21.15.49:34700) 2023-07-18T03:38:52.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.229+0000 7f968a19c700 1 -- 172.21.15.49:0/930221443 learned_addr learned my addr 172.21.15.49:0/930221443 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:52.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.229+0000 7f968919a700 1 --2- 172.21.15.49:0/930221443 >> v2:172.21.15.107:3300/0 conn(0x7f96840b4530 0x7f96840d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.229+0000 7f9688999700 1 -- 172.21.15.49:0/930221443 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817046275 0 0) 0x7f96840d1490 con 0x7f96840d1050 2023-07-18T03:38:52.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.229+0000 7f968999b700 1 -- 172.21.15.49:0/930221443 >> v2:172.21.15.107:3300/0 conn(0x7f96840b4530 msgr2=0x7f96840d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:52.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.229+0000 7f968999b700 1 --2- 172.21.15.49:0/930221443 >> v2:172.21.15.107:3300/0 conn(0x7f96840b4530 0x7f96840d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.229+0000 7f968999b700 1 -- 172.21.15.49:0/930221443 >> v1:172.21.15.49:6789/0 conn(0x7f96840d1050 legacy=0x7f96840b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.229+0000 7f968999b700 1 -- 172.21.15.49:0/930221443 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96840afaa0 con 0x7f96840b8a60 2023-07-18T03:38:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.229+0000 7f968919a700 1 --2- 172.21.15.49:0/930221443 >> v2:172.21.15.107:3300/0 conn(0x7f96840b4530 0x7f96840d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.230+0000 7f968999b700 1 --2- 172.21.15.49:0/930221443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96840b8a60 0x7f96840b3ff0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f968409c8e0 tx=0x7f9680009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ab744bd612e93c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.230+0000 7f9688999700 1 -- 172.21.15.49:0/930221443 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680011830 con 0x7f96840b8a60 2023-07-18T03:38:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.230+0000 7f9688999700 1 -- 172.21.15.49:0/930221443 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9680011990 con 0x7f96840b8a60 2023-07-18T03:38:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.230+0000 7f9688999700 1 -- 172.21.15.49:0/930221443 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680011d20 con 0x7f96840b8a60 2023-07-18T03:38:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.230+0000 7f968bbff700 1 -- 172.21.15.49:0/930221443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96840b8a60 msgr2=0x7f96840b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.230+0000 7f968bbff700 1 --2- 172.21.15.49:0/930221443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96840b8a60 0x7f96840b3ff0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f968409c8e0 tx=0x7f9680009b20 comp rx=0 tx=0).stop 2023-07-18T03:38:52.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.230+0000 7f968bbff700 1 -- 172.21.15.49:0/930221443 shutdown_connections 2023-07-18T03:38:52.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.230+0000 7f968bbff700 1 --2- 172.21.15.49:0/930221443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96840b8a60 0x7f96840b3ff0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.230+0000 7f968bbff700 1 --2- 172.21.15.49:0/930221443 >> v2:172.21.15.107:3300/0 conn(0x7f96840b4530 0x7f96840d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.230+0000 7f968bbff700 1 -- 172.21.15.49:0/930221443 >> 172.21.15.49:0/930221443 conn(0x7f968409c1b0 msgr2=0x7f96840d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:52.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.231+0000 7f968bbff700 1 -- 172.21.15.49:0/930221443 shutdown_connections 2023-07-18T03:38:52.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.231+0000 7f968bbff700 1 -- 172.21.15.49:0/930221443 wait complete. 2023-07-18T03:38:52.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.232+0000 7f968bbff700 1 Processor -- start 2023-07-18T03:38:52.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.232+0000 7f968bbff700 1 -- start start 2023-07-18T03:38:52.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.233+0000 7f968bbff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96840b4530 0x7f968415f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.233+0000 7f968bbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96840b8a60 0x7f96841f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.233+0000 7f968bbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96840d1050 0x7f96841f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.233+0000 7f968bbff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96840cf4b0 con 0x7f96840b4530 2023-07-18T03:38:52.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.233+0000 7f968bbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96840cf790 con 0x7f96840b8a60 2023-07-18T03:38:52.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.233+0000 7f968bbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96840cfa70 con 0x7f96840d1050 2023-07-18T03:38:52.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.234+0000 7f968999b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96840b4530 0x7f968415f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.234+0000 7f968999b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96840b4530 0x7f968415f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34708/0 (socket says 172.21.15.49:34708) 2023-07-18T03:38:52.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.234+0000 7f968999b700 1 -- 172.21.15.49:0/3376422461 learned_addr learned my addr 172.21.15.49:0/3376422461 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:52.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.234+0000 7f968a19c700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96840d1050 0x7f96841f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.234+0000 7f968919a700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96840b8a60 0x7f96841f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.234+0000 7f968999b700 1 -- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96840d1050 msgr2=0x7f96841f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.234+0000 7f968999b700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96840d1050 0x7f96841f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.234+0000 7f968999b700 1 -- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96840b8a60 msgr2=0x7f96841f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.234+0000 7f968999b700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96840b8a60 0x7f96841f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.234+0000 7f968999b700 1 -- 172.21.15.49:0/3376422461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96800094a0 con 0x7f96840b4530 2023-07-18T03:38:52.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.234+0000 7f968999b700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96840b4530 0x7f968415f700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9680000c00 tx=0x7f9680004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:52.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.235+0000 7f967a7fc700 1 -- 172.21.15.49:0/3376422461 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96800036a0 con 0x7f96840b4530 2023-07-18T03:38:52.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.235+0000 7f967a7fc700 1 -- 172.21.15.49:0/3376422461 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968001f070 con 0x7f96840b4530 2023-07-18T03:38:52.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.235+0000 7f968bbff700 1 -- 172.21.15.49:0/3376422461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96840cfd50 con 0x7f96840b4530 2023-07-18T03:38:52.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.235+0000 7f967a7fc700 1 -- 172.21.15.49:0/3376422461 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680003a60 con 0x7f96840b4530 2023-07-18T03:38:52.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.235+0000 7f968bbff700 1 -- 172.21.15.49:0/3376422461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968415fc40 con 0x7f96840b4530 2023-07-18T03:38:52.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.236+0000 7f968bbff700 1 -- 172.21.15.49:0/3376422461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96840cfee0 con 0x7f96840b4530 2023-07-18T03:38:52.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.237+0000 7f967a7fc700 1 -- 172.21.15.49:0/3376422461 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9680003bc0 con 0x7f96840b4530 2023-07-18T03:38:52.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.238+0000 7f967a7fc700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f966806ace0 0x7f966806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.238+0000 7f967a7fc700 1 -- 172.21.15.49:0/3376422461 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9680021070 con 0x7f96840b4530 2023-07-18T03:38:52.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.238+0000 7f968919a700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f966806ace0 0x7f966806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.239+0000 7f968919a700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f966806ace0 0x7f966806d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f96840d0640 tx=0x7f9674008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:52.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.242+0000 7f967a7fc700 1 -- 172.21.15.49:0/3376422461 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9680055f90 con 0x7f96840b4530 2023-07-18T03:38:52.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.450+0000 7f968bbff700 1 -- 172.21.15.49:0/3376422461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9684074350 con 0x7f96840b4530 2023-07-18T03:38:52.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.450+0000 7f967a7fc700 1 -- 172.21.15.49:0/3376422461 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96800595b0 con 0x7f96840b4530 2023-07-18T03:38:52.446 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:38:52.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.452+0000 7f968bbff700 1 -- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f966806ace0 msgr2=0x7f966806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.452+0000 7f968bbff700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f966806ace0 0x7f966806d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f96840d0640 tx=0x7f9674008040 comp rx=0 tx=0).stop 2023-07-18T03:38:52.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.452+0000 7f968bbff700 1 -- 172.21.15.49:0/3376422461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96840b4530 msgr2=0x7f968415f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.452+0000 7f968bbff700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96840b4530 0x7f968415f700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9680000c00 tx=0x7f9680004750 comp rx=0 tx=0).stop 2023-07-18T03:38:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.453+0000 7f968bbff700 1 -- 172.21.15.49:0/3376422461 shutdown_connections 2023-07-18T03:38:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.453+0000 7f968bbff700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96840d1050 0x7f96841f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.453+0000 7f968bbff700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f966806ace0 0x7f966806d190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.453+0000 7f968bbff700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96840b4530 0x7f968415f700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.453+0000 7f968bbff700 1 --2- 172.21.15.49:0/3376422461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96840b8a60 0x7f96841f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.453+0000 7f968bbff700 1 -- 172.21.15.49:0/3376422461 >> 172.21.15.49:0/3376422461 conn(0x7f968409c1b0 msgr2=0x7f968406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.453+0000 7f968bbff700 1 -- 172.21.15.49:0/3376422461 shutdown_connections 2023-07-18T03:38:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.453+0000 7f968bbff700 1 -- 172.21.15.49:0/3376422461 wait complete. 2023-07-18T03:38:52.466 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:38:52.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.682+0000 7f45bf5ff700 1 Processor -- start 2023-07-18T03:38:52.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.682+0000 7f45bf5ff700 1 -- start start 2023-07-18T03:38:52.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.683+0000 7f45bf5ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f45b80b8a60 0x7f45b80b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.683+0000 7f45bf5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45b80b4530 0x7f45b80d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.683+0000 7f45bf5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f45b80d1050 0x7f45b80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.683+0000 7f45bf5ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b80d14d0 con 0x7f45b80b8a60 2023-07-18T03:38:52.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.683+0000 7f45bf5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b80b8e30 con 0x7f45b80d1050 2023-07-18T03:38:52.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.683+0000 7f45bf5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80afaa0 con 0x7f45b80b4530 2023-07-18T03:38:52.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bd39b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f45b80b8a60 0x7f45b80b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bd39b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f45b80b8a60 0x7f45b80b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34710/0 (socket says 172.21.15.49:34710) 2023-07-18T03:38:52.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bd39b700 1 -- 172.21.15.49:0/3105422100 learned_addr learned my addr 172.21.15.49:0/3105422100 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:52.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bd39b700 1 -- 172.21.15.49:0/3105422100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45b80b4530 msgr2=0x7f45b80d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:52.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bcb9a700 1 --2- 172.21.15.49:0/3105422100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45b80b4530 0x7f45b80d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bdb9c700 1 --2- 172.21.15.49:0/3105422100 >> v2:172.21.15.107:3300/0 conn(0x7f45b80d1050 0x7f45b80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bd39b700 1 --2- 172.21.15.49:0/3105422100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45b80b4530 0x7f45b80d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bd39b700 1 -- 172.21.15.49:0/3105422100 >> v2:172.21.15.107:3300/0 conn(0x7f45b80d1050 msgr2=0x7f45b80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:52.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bd39b700 1 --2- 172.21.15.49:0/3105422100 >> v2:172.21.15.107:3300/0 conn(0x7f45b80d1050 0x7f45b80b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bd39b700 1 -- 172.21.15.49:0/3105422100 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b80afd80 con 0x7f45b80b8a60 2023-07-18T03:38:52.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bcb9a700 1 --2- 172.21.15.49:0/3105422100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45b80b4530 0x7f45b80d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:52.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bdb9c700 1 --2- 172.21.15.49:0/3105422100 >> v2:172.21.15.107:3300/0 conn(0x7f45b80d1050 0x7f45b80b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:52.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.684+0000 7f45bd39b700 1 --2- 172.21.15.49:0/3105422100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45b80b8a60 0x7f45b80b3ff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f45b808c2e0 tx=0x7f45b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1431934534e8dd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:52.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.685+0000 7f45affff700 1 -- 172.21.15.49:0/3105422100 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400b870 con 0x7f45b80b8a60 2023-07-18T03:38:52.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.685+0000 7f45affff700 1 -- 172.21.15.49:0/3105422100 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b400b9d0 con 0x7f45b80b8a60 2023-07-18T03:38:52.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.685+0000 7f45affff700 1 -- 172.21.15.49:0/3105422100 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400bd60 con 0x7f45b80b8a60 2023-07-18T03:38:52.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.685+0000 7f45bf5ff700 1 -- 172.21.15.49:0/3105422100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45b80b8a60 msgr2=0x7f45b80b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.685+0000 7f45bf5ff700 1 --2- 172.21.15.49:0/3105422100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45b80b8a60 0x7f45b80b3ff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f45b808c2e0 tx=0x7f45b4009bd0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.685+0000 7f45bf5ff700 1 -- 172.21.15.49:0/3105422100 shutdown_connections 2023-07-18T03:38:52.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.685+0000 7f45bf5ff700 1 --2- 172.21.15.49:0/3105422100 >> v2:172.21.15.107:3300/0 conn(0x7f45b80d1050 0x7f45b80b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.685+0000 7f45bf5ff700 1 --2- 172.21.15.49:0/3105422100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45b80b8a60 0x7f45b80b3ff0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.685+0000 7f45bf5ff700 1 --2- 172.21.15.49:0/3105422100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45b80b4530 0x7f45b80d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.685+0000 7f45bf5ff700 1 -- 172.21.15.49:0/3105422100 >> 172.21.15.49:0/3105422100 conn(0x7f45b809c1b0 msgr2=0x7f45b80d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:52.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.685+0000 7f45bf5ff700 1 -- 172.21.15.49:0/3105422100 shutdown_connections 2023-07-18T03:38:52.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.685+0000 7f45bf5ff700 1 -- 172.21.15.49:0/3105422100 wait complete. 2023-07-18T03:38:52.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.687+0000 7f45bf5ff700 1 Processor -- start 2023-07-18T03:38:52.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.687+0000 7f45bf5ff700 1 -- start start 2023-07-18T03:38:52.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.688+0000 7f45bf5ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45b80b4530 0x7f45b80ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.688+0000 7f45bf5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45b80b8a60 0x7f45b81f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.688+0000 7f45bf5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45b80d1050 0x7f45b815f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.688+0000 7f45bf5ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80d0390 con 0x7f45b80b4530 2023-07-18T03:38:52.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.688+0000 7f45bf5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80d0630 con 0x7f45b80d1050 2023-07-18T03:38:52.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.688+0000 7f45bf5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b815d940 con 0x7f45b80b8a60 2023-07-18T03:38:52.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.688+0000 7f45bd39b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45b80b4530 0x7f45b80ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.688+0000 7f45bd39b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45b80b4530 0x7f45b80ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34722/0 (socket says 172.21.15.49:34722) 2023-07-18T03:38:52.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.688+0000 7f45bd39b700 1 -- 172.21.15.49:0/181737596 learned_addr learned my addr 172.21.15.49:0/181737596 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:52.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.689+0000 7f45bd39b700 1 -- 172.21.15.49:0/181737596 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45b80b8a60 msgr2=0x7f45b81f15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.689+0000 7f45bd39b700 1 --2- 172.21.15.49:0/181737596 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45b80b8a60 0x7f45b81f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.689+0000 7f45bd39b700 1 -- 172.21.15.49:0/181737596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45b80d1050 msgr2=0x7f45b815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.689+0000 7f45bd39b700 1 --2- 172.21.15.49:0/181737596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45b80d1050 0x7f45b815f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.689+0000 7f45bd39b700 1 -- 172.21.15.49:0/181737596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b4009550 con 0x7f45b80b4530 2023-07-18T03:38:52.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.689+0000 7f45bd39b700 1 --2- 172.21.15.49:0/181737596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45b80b4530 0x7f45b80ccc00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f45b80681d0 tx=0x7f45b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:52.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.689+0000 7f45adffb700 1 -- 172.21.15.49:0/181737596 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4004310 con 0x7f45b80b4530 2023-07-18T03:38:52.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.689+0000 7f45adffb700 1 -- 172.21.15.49:0/181737596 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b401f070 con 0x7f45b80b4530 2023-07-18T03:38:52.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.689+0000 7f45bf5ff700 1 -- 172.21.15.49:0/181737596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b815dc20 con 0x7f45b80b4530 2023-07-18T03:38:52.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.689+0000 7f45adffb700 1 -- 172.21.15.49:0/181737596 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b40036a0 con 0x7f45b80b4530 2023-07-18T03:38:52.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.689+0000 7f45bf5ff700 1 -- 172.21.15.49:0/181737596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b81f1b20 con 0x7f45b80b4530 2023-07-18T03:38:52.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.690+0000 7f45bf5ff700 1 -- 172.21.15.49:0/181737596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b815e140 con 0x7f45b80b4530 2023-07-18T03:38:52.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.691+0000 7f45adffb700 1 -- 172.21.15.49:0/181737596 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f45b4021030 con 0x7f45b80b4530 2023-07-18T03:38:52.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.692+0000 7f45adffb700 1 --2- 172.21.15.49:0/181737596 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f459c06ace0 0x7f459c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.692+0000 7f45adffb700 1 -- 172.21.15.49:0/181737596 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f45b408a760 con 0x7f45b80b4530 2023-07-18T03:38:52.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.693+0000 7f45bcb9a700 1 --2- 172.21.15.49:0/181737596 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f459c06ace0 0x7f459c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.693+0000 7f45bcb9a700 1 --2- 172.21.15.49:0/181737596 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f459c06ace0 0x7f459c06d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f45b80b83c0 tx=0x7f45a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:52.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.696+0000 7f45adffb700 1 -- 172.21.15.49:0/181737596 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f45b4055800 con 0x7f45b80b4530 2023-07-18T03:38:52.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.904+0000 7f45bf5ff700 1 -- 172.21.15.49:0/181737596 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45b8074350 con 0x7f45b80b4530 2023-07-18T03:38:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.905+0000 7f45adffb700 1 -- 172.21.15.49:0/181737596 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45b40150a0 con 0x7f45b80b4530 2023-07-18T03:38:52.900 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:38:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.907+0000 7f45bf5ff700 1 -- 172.21.15.49:0/181737596 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f459c06ace0 msgr2=0x7f459c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.907+0000 7f45bf5ff700 1 --2- 172.21.15.49:0/181737596 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f459c06ace0 0x7f459c06d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f45b80b83c0 tx=0x7f45a8008040 comp rx=0 tx=0).stop 2023-07-18T03:38:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.907+0000 7f45bf5ff700 1 -- 172.21.15.49:0/181737596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45b80b4530 msgr2=0x7f45b80ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.907+0000 7f45bf5ff700 1 --2- 172.21.15.49:0/181737596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45b80b4530 0x7f45b80ccc00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f45b80681d0 tx=0x7f45b4004750 comp rx=0 tx=0).stop 2023-07-18T03:38:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.907+0000 7f45bf5ff700 1 -- 172.21.15.49:0/181737596 shutdown_connections 2023-07-18T03:38:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.907+0000 7f45bf5ff700 1 --2- 172.21.15.49:0/181737596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45b80d1050 0x7f45b815f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.907+0000 7f45bf5ff700 1 --2- 172.21.15.49:0/181737596 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f459c06ace0 0x7f459c06d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.907+0000 7f45bf5ff700 1 --2- 172.21.15.49:0/181737596 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45b80b4530 0x7f45b80ccc00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.907+0000 7f45bf5ff700 1 --2- 172.21.15.49:0/181737596 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45b80b8a60 0x7f45b81f15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.907+0000 7f45bf5ff700 1 -- 172.21.15.49:0/181737596 >> 172.21.15.49:0/181737596 conn(0x7f45b809c1b0 msgr2=0x7f45b815c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.907+0000 7f45bf5ff700 1 -- 172.21.15.49:0/181737596 shutdown_connections 2023-07-18T03:38:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:52.908+0000 7f45bf5ff700 1 -- 172.21.15.49:0/181737596 wait complete. 2023-07-18T03:38:52.921 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:38:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.131+0000 7fb1ebbff700 1 Processor -- start 2023-07-18T03:38:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.131+0000 7fb1ebbff700 1 -- start start 2023-07-18T03:38:53.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.132+0000 7fb1ebbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1e415c950 0x7fb1e40b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.132+0000 7fb1ebbff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb1e40b1e00 0x7fb1e40d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.132+0000 7fb1ebbff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e40d2ce0 con 0x7fb1e40b2350 2023-07-18T03:38:53.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.132+0000 7fb1ebbff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e40b3350 con 0x7fb1e40b1e00 2023-07-18T03:38:53.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.132+0000 7fb1ebbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40b3630 con 0x7fb1e415c950 2023-07-18T03:38:53.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.132+0000 7fb1ea19c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb1e40b2350 0x7fb1e40afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35210/0 (socket says 172.21.15.49:35210) 2023-07-18T03:38:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.132+0000 7fb1ea19c700 1 -- 172.21.15.49:0/1717723476 learned_addr learned my addr 172.21.15.49:0/1717723476 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.133+0000 7fb1e919a700 1 --2- 172.21.15.49:0/1717723476 >> v2:172.21.15.107:3300/0 conn(0x7fb1e40b1e00 0x7fb1e40d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.133+0000 7fb1e999b700 1 --2- 172.21.15.49:0/1717723476 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1e415c950 0x7fb1e40b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.133+0000 7fb1e8999700 1 -- 172.21.15.49:0/1717723476 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586119206 0 0) 0x7fb1e40d2ce0 con 0x7fb1e40b2350 2023-07-18T03:38:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.133+0000 7fb1e8999700 1 -- 172.21.15.49:0/1717723476 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1d00035b0 con 0x7fb1e40b2350 2023-07-18T03:38:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.133+0000 7fb1e919a700 1 -- 172.21.15.49:0/1717723476 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1e415c950 msgr2=0x7fb1e40b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.133+0000 7fb1e919a700 1 --2- 172.21.15.49:0/1717723476 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1e415c950 0x7fb1e40b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.133+0000 7fb1e919a700 1 -- 172.21.15.49:0/1717723476 >> v1:172.21.15.49:6789/0 conn(0x7fb1e40b2350 legacy=0x7fb1e40afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.133+0000 7fb1e919a700 1 -- 172.21.15.49:0/1717723476 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e40b3910 con 0x7fb1e40b1e00 2023-07-18T03:38:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.133+0000 7fb1e919a700 1 --2- 172.21.15.49:0/1717723476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1e40b1e00 0x7fb1e40d0b10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009840 tx=0x7fb1d4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d151366913179c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.134+0000 7fb1e8999700 1 -- 172.21.15.49:0/1717723476 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4011830 con 0x7fb1e40b1e00 2023-07-18T03:38:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.134+0000 7fb1e8999700 1 -- 172.21.15.49:0/1717723476 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d4011990 con 0x7fb1e40b1e00 2023-07-18T03:38:53.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.134+0000 7fb1e8999700 1 -- 172.21.15.49:0/1717723476 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4011d20 con 0x7fb1e40b1e00 2023-07-18T03:38:53.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.134+0000 7fb1ebbff700 1 -- 172.21.15.49:0/1717723476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1e40b1e00 msgr2=0x7fb1e40d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.134+0000 7fb1ebbff700 1 --2- 172.21.15.49:0/1717723476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1e40b1e00 0x7fb1e40d0b10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009840 tx=0x7fb1d4009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:53.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.134+0000 7fb1ebbff700 1 -- 172.21.15.49:0/1717723476 shutdown_connections 2023-07-18T03:38:53.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.134+0000 7fb1ebbff700 1 --2- 172.21.15.49:0/1717723476 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1e415c950 0x7fb1e40b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.134+0000 7fb1ebbff700 1 --2- 172.21.15.49:0/1717723476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1e40b1e00 0x7fb1e40d0b10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.134+0000 7fb1ebbff700 1 -- 172.21.15.49:0/1717723476 >> 172.21.15.49:0/1717723476 conn(0x7fb1e409c1d0 msgr2=0x7fb1e40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:53.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.134+0000 7fb1ebbff700 1 -- 172.21.15.49:0/1717723476 shutdown_connections 2023-07-18T03:38:53.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.134+0000 7fb1ebbff700 1 -- 172.21.15.49:0/1717723476 wait complete. 2023-07-18T03:38:53.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.136+0000 7fb1ebbff700 1 Processor -- start 2023-07-18T03:38:53.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.136+0000 7fb1ebbff700 1 -- start start 2023-07-18T03:38:53.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.137+0000 7fb1ebbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1e40b1e00 0x7fb1e41542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.137+0000 7fb1ebbff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1e40b2350 0x7fb1e4153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.137+0000 7fb1ebbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1e415c950 0x7fb1e40cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.137+0000 7fb1ebbff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4154100 con 0x7fb1e40b2350 2023-07-18T03:38:53.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.137+0000 7fb1ebbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40d03e0 con 0x7fb1e40b1e00 2023-07-18T03:38:53.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.137+0000 7fb1ebbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40cee00 con 0x7fb1e415c950 2023-07-18T03:38:53.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.138+0000 7fb1e919a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1e40b2350 0x7fb1e4153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.138+0000 7fb1e919a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1e40b2350 0x7fb1e4153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34736/0 (socket says 172.21.15.49:34736) 2023-07-18T03:38:53.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.138+0000 7fb1e919a700 1 -- 172.21.15.49:0/2080695204 learned_addr learned my addr 172.21.15.49:0/2080695204 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:53.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.138+0000 7fb1ea19c700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1e415c950 0x7fb1e40cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.138+0000 7fb1e999b700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1e40b1e00 0x7fb1e41542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.138+0000 7fb1e919a700 1 -- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1e415c950 msgr2=0x7fb1e40cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.138+0000 7fb1e919a700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1e415c950 0x7fb1e40cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.138+0000 7fb1e919a700 1 -- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1e40b1e00 msgr2=0x7fb1e41542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.138+0000 7fb1e919a700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1e40b1e00 0x7fb1e41542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.138+0000 7fb1e919a700 1 -- 172.21.15.49:0/2080695204 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d40094a0 con 0x7fb1e40b2350 2023-07-18T03:38:53.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.138+0000 7fb1e919a700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1e40b2350 0x7fb1e4153bc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009790 tx=0x7fb1d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:53.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.139+0000 7fb1da7fc700 1 -- 172.21.15.49:0/2080695204 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d40036a0 con 0x7fb1e40b2350 2023-07-18T03:38:53.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.139+0000 7fb1da7fc700 1 -- 172.21.15.49:0/2080695204 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d401f070 con 0x7fb1e40b2350 2023-07-18T03:38:53.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.139+0000 7fb1ebbff700 1 -- 172.21.15.49:0/2080695204 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e40cf0e0 con 0x7fb1e40b2350 2023-07-18T03:38:53.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.139+0000 7fb1da7fc700 1 -- 172.21.15.49:0/2080695204 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4003a20 con 0x7fb1e40b2350 2023-07-18T03:38:53.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.139+0000 7fb1ebbff700 1 -- 172.21.15.49:0/2080695204 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e40cf6a0 con 0x7fb1e40b2350 2023-07-18T03:38:53.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.140+0000 7fb1ebbff700 1 -- 172.21.15.49:0/2080695204 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e404e280 con 0x7fb1e40b2350 2023-07-18T03:38:53.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.141+0000 7fb1da7fc700 1 -- 172.21.15.49:0/2080695204 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb1d4003b80 con 0x7fb1e40b2350 2023-07-18T03:38:53.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.142+0000 7fb1da7fc700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb1c806ace0 0x7fb1c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.142+0000 7fb1da7fc700 1 -- 172.21.15.49:0/2080695204 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb1d4021070 con 0x7fb1e40b2350 2023-07-18T03:38:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.143+0000 7fb1e999b700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb1c806ace0 0x7fb1c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.143+0000 7fb1e999b700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb1c806ace0 0x7fb1c806d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40c9be0 tx=0x7fb1e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.147+0000 7fb1da7fc700 1 -- 172.21.15.49:0/2080695204 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb1d4055d70 con 0x7fb1e40b2350 2023-07-18T03:38:53.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.401+0000 7fb1ebbff700 1 -- 172.21.15.49:0/2080695204 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1e408c5a0 con 0x7fb1e40b2350 2023-07-18T03:38:53.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.402+0000 7fb1da7fc700 1 -- 172.21.15.49:0/2080695204 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4709 (secure 0 0 0) 0x7fb1d402b090 con 0x7fb1e40b2350 2023-07-18T03:38:53.397 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:53.397 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1477016053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1477016053},{"type":"v1","addr":"172.21.15.107:6835","nonce":1477016053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2998451500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2998451500},{"type":"v1","addr":"172.21.15.49:6835","nonce":2998451500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4247281109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4247281109},{"type":"v1","addr":"172.21.15.49:6833","nonce":4247281109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:38:36.025442+0000","modified":"2023-07-18T03:38:39.535929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1127901907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1127901907},{"type":"v1","addr":"172.21.15.107:6837","nonce":1127901907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:38:53.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.404+0000 7fb1ebbff700 1 -- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb1c806ace0 msgr2=0x7fb1c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.404+0000 7fb1ebbff700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb1c806ace0 0x7fb1c806d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40c9be0 tx=0x7fb1e0008040 comp rx=0 tx=0).stop 2023-07-18T03:38:53.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.405+0000 7fb1ebbff700 1 -- 172.21.15.49:0/2080695204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1e40b2350 msgr2=0x7fb1e4153bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.405+0000 7fb1ebbff700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1e40b2350 0x7fb1e4153bc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009790 tx=0x7fb1d4004750 comp rx=0 tx=0).stop 2023-07-18T03:38:53.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.405+0000 7fb1ebbff700 1 -- 172.21.15.49:0/2080695204 shutdown_connections 2023-07-18T03:38:53.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.405+0000 7fb1ebbff700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1e415c950 0x7fb1e40cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.405+0000 7fb1ebbff700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1e40b1e00 0x7fb1e41542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.405+0000 7fb1ebbff700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb1c806ace0 0x7fb1c806d190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.405+0000 7fb1ebbff700 1 --2- 172.21.15.49:0/2080695204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1e40b2350 0x7fb1e4153bc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.405+0000 7fb1ebbff700 1 -- 172.21.15.49:0/2080695204 >> 172.21.15.49:0/2080695204 conn(0x7fb1e409c1d0 msgr2=0x7fb1e40d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:53.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.405+0000 7fb1ebbff700 1 -- 172.21.15.49:0/2080695204 shutdown_connections 2023-07-18T03:38:53.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.405+0000 7fb1ebbff700 1 -- 172.21.15.49:0/2080695204 wait complete. 2023-07-18T03:38:53.404 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2023-07-18T03:38:53.419 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:38:53.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.628+0000 7f45f9bff700 1 Processor -- start 2023-07-18T03:38:53.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.629+0000 7f45f9bff700 1 -- start start 2023-07-18T03:38:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.630+0000 7f45f9bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f415c950 0x7f45f40b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.630+0000 7f45f9bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f40d2ce0 con 0x7f45f40b1e00 2023-07-18T03:38:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.630+0000 7f45f9bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f40b3350 con 0x7f45f40d1220 2023-07-18T03:38:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.630+0000 7f45f9bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40b3630 con 0x7f45f415c950 2023-07-18T03:38:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.630+0000 7f45f2ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f45f40b1e00 0x7f45f40d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35226/0 (socket says 172.21.15.49:35226) 2023-07-18T03:38:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.630+0000 7f45f2ffd700 1 -- 172.21.15.49:0/4170011759 learned_addr learned my addr 172.21.15.49:0/4170011759 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:53.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.630+0000 7f45f37fe700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f415c950 0x7f45f40b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.630+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473608346 0 0) 0x7f45f40d2ce0 con 0x7f45f40b1e00 2023-07-18T03:38:53.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.630+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45dc0035b0 con 0x7f45f40b1e00 2023-07-18T03:38:53.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266021876 0 0) 0x7f45f40b3350 con 0x7f45f40d1220 2023-07-18T03:38:53.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f40d2ce0 con 0x7f45f40d1220 2023-07-18T03:38:53.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3431534612 0 0) 0x7f45dc0035b0 con 0x7f45f40b1e00 2023-07-18T03:38:53.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f415c950 msgr2=0x7f45f40b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:53.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f415c950 0x7f45f40b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 >> v1:172.21.15.107:6789/0 conn(0x7f45f40d1220 legacy=0x7f45f40afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40b3910 con 0x7f45f40b1e00 2023-07-18T03:38:53.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f45e8001670 con 0x7f45f40b1e00 2023-07-18T03:38:53.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 >> v1:172.21.15.49:6789/0 conn(0x7f45f40b1e00 legacy=0x7f45f40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f415c950 0x7f45dc003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45dc004170 0x7f45dc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45dc004d90 0x7f45dc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40b3630 con 0x7f45dc004d90 2023-07-18T03:38:53.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc00bba0 con 0x7f45dc004170 2023-07-18T03:38:53.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc00bd60 con 0x7f45f415c950 2023-07-18T03:38:53.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f3fff700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45dc004d90 0x7f45dc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f2ffd700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45dc004170 0x7f45dc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.631+0000 7f45f37fe700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f415c950 0x7f45dc003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.632+0000 7f45f3fff700 1 -- 172.21.15.49:0/4170011759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f415c950 msgr2=0x7f45dc003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:53.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.632+0000 7f45f3fff700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f415c950 0x7f45dc003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.632+0000 7f45f3fff700 1 -- 172.21.15.49:0/4170011759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45dc004170 msgr2=0x7f45dc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:53.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.632+0000 7f45f3fff700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45dc004170 0x7f45dc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.632+0000 7f45f3fff700 1 -- 172.21.15.49:0/4170011759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc0035b0 con 0x7f45dc004d90 2023-07-18T03:38:53.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.632+0000 7f45f2ffd700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45dc004170 0x7f45dc004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:53.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.632+0000 7f45f3fff700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45dc004d90 0x7f45dc005200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00c490 tx=0x7f45dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632b068caecb7814 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:53.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.632+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec003a00 con 0x7f45dc004d90 2023-07-18T03:38:53.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.632+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec003b60 con 0x7f45dc004d90 2023-07-18T03:38:53.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.632+0000 7f45f27fc700 1 -- 172.21.15.49:0/4170011759 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0050d0 con 0x7f45dc004d90 2023-07-18T03:38:53.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.633+0000 7f45f9bff700 1 -- 172.21.15.49:0/4170011759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45dc004d90 msgr2=0x7f45dc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.633+0000 7f45f9bff700 1 --2- 172.21.15.49:0/4170011759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45dc004d90 0x7f45dc005200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00c490 tx=0x7f45dc005880 comp rx=0 tx=0).stop 2023-07-18T03:38:53.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.633+0000 7f45f3fff700 1 -- 172.21.15.49:0/4170011759 reap_dead start 2023-07-18T03:38:53.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.633+0000 7f45f9bff700 1 -- 172.21.15.49:0/4170011759 shutdown_connections 2023-07-18T03:38:53.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.633+0000 7f45f9bff700 1 -- 172.21.15.49:0/4170011759 >> 172.21.15.49:0/4170011759 conn(0x7f45f409c1d0 msgr2=0x7f45f40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:53.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.633+0000 7f45f9bff700 1 -- 172.21.15.49:0/4170011759 shutdown_connections 2023-07-18T03:38:53.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.633+0000 7f45f9bff700 1 -- 172.21.15.49:0/4170011759 wait complete. 2023-07-18T03:38:53.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.635+0000 7f45f9bff700 1 Processor -- start 2023-07-18T03:38:53.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.636+0000 7f45f9bff700 1 -- start start 2023-07-18T03:38:53.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f9bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45dc004d90 0x7f45f40b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f9bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f40b6780 0x7f45f41f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f9bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f41570c0 0x7f45f4157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f9bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4157aa0 con 0x7f45f41570c0 2023-07-18T03:38:53.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f9bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40b7d10 con 0x7f45dc004d90 2023-07-18T03:38:53.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f9bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40b7ff0 con 0x7f45f40b6780 2023-07-18T03:38:53.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f41570c0 0x7f45f4157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f41570c0 0x7f45f4157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34756/0 (socket says 172.21.15.49:34756) 2023-07-18T03:38:53.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f3fff700 1 -- 172.21.15.49:0/2339296295 learned_addr learned my addr 172.21.15.49:0/2339296295 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:53.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f37fe700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45dc004d90 0x7f45f40b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f2ffd700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f40b6780 0x7f45f41f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f3fff700 1 -- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f40b6780 msgr2=0x7f45f41f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.637+0000 7f45f3fff700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f40b6780 0x7f45f41f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.638+0000 7f45f3fff700 1 -- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45dc004d90 msgr2=0x7f45f40b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.638+0000 7f45f3fff700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45dc004d90 0x7f45f40b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.638+0000 7f45f3fff700 1 -- 172.21.15.49:0/2339296295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40b82d0 con 0x7f45f41570c0 2023-07-18T03:38:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.638+0000 7f45f2ffd700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f40b6780 0x7f45f41f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.638+0000 7f45f3fff700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f41570c0 0x7f45f4157560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f45ec004000 tx=0x7f45ec001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:53.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.638+0000 7f45dbfff700 1 -- 172.21.15.49:0/2339296295 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec003a00 con 0x7f45f41570c0 2023-07-18T03:38:53.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.638+0000 7f45dbfff700 1 -- 172.21.15.49:0/2339296295 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec001e90 con 0x7f45f41570c0 2023-07-18T03:38:53.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.638+0000 7f45f9bff700 1 -- 172.21.15.49:0/2339296295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f40b85b0 con 0x7f45f41570c0 2023-07-18T03:38:53.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.638+0000 7f45dbfff700 1 -- 172.21.15.49:0/2339296295 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec005520 con 0x7f45f41570c0 2023-07-18T03:38:53.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.638+0000 7f45f9bff700 1 -- 172.21.15.49:0/2339296295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f41f22e0 con 0x7f45f41570c0 2023-07-18T03:38:53.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.639+0000 7f45f9bff700 1 -- 172.21.15.49:0/2339296295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f40b8ad0 con 0x7f45f41570c0 2023-07-18T03:38:53.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.640+0000 7f45dbfff700 1 -- 172.21.15.49:0/2339296295 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f45ec019070 con 0x7f45f41570c0 2023-07-18T03:38:53.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.641+0000 7f45dbfff700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f45e006ace0 0x7f45e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.641+0000 7f45dbfff700 1 -- 172.21.15.49:0/2339296295 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f45ec08a140 con 0x7f45f41570c0 2023-07-18T03:38:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.641+0000 7f45f37fe700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f45e006ace0 0x7f45e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.642+0000 7f45f37fe700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f45e006ace0 0x7f45e006d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009c40 tx=0x7f45e40091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:53.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.645+0000 7f45dbfff700 1 -- 172.21.15.49:0/2339296295 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f45f40b8ad0 con 0x7f45f41570c0 2023-07-18T03:38:53.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.852+0000 7f45f9bff700 1 -- 172.21.15.49:0/2339296295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45f40b8ad0 con 0x7f45f41570c0 2023-07-18T03:38:53.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.853+0000 7f45dbfff700 1 -- 172.21.15.49:0/2339296295 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f45f40b8ad0 con 0x7f45f41570c0 2023-07-18T03:38:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:53.849 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:37.510006+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:38:53.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.856+0000 7f45f9bff700 1 -- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f45e006ace0 msgr2=0x7f45e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.856+0000 7f45f9bff700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f45e006ace0 0x7f45e006d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009c40 tx=0x7f45e40091c0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.856+0000 7f45f9bff700 1 -- 172.21.15.49:0/2339296295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f41570c0 msgr2=0x7f45f4157560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.856+0000 7f45f9bff700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f41570c0 0x7f45f4157560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f45ec004000 tx=0x7f45ec001690 comp rx=0 tx=0).stop 2023-07-18T03:38:53.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.856+0000 7f45f9bff700 1 -- 172.21.15.49:0/2339296295 shutdown_connections 2023-07-18T03:38:53.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.856+0000 7f45f9bff700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f45e006ace0 0x7f45e006d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.856+0000 7f45f9bff700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f41570c0 0x7f45f4157560 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.856+0000 7f45f9bff700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45dc004d90 0x7f45f40b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.856+0000 7f45f9bff700 1 --2- 172.21.15.49:0/2339296295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f40b6780 0x7f45f41f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.856+0000 7f45f9bff700 1 -- 172.21.15.49:0/2339296295 >> 172.21.15.49:0/2339296295 conn(0x7f45f409c1d0 msgr2=0x7f45f40b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:53.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.856+0000 7f45f9bff700 1 -- 172.21.15.49:0/2339296295 shutdown_connections 2023-07-18T03:38:53.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:53.856+0000 7f45f9bff700 1 -- 172.21.15.49:0/2339296295 wait complete. 2023-07-18T03:38:53.869 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:38:54.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.078+0000 7f771e7ff700 1 Processor -- start 2023-07-18T03:38:54.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.078+0000 7f771e7ff700 1 -- start start 2023-07-18T03:38:54.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f771e7ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f77180d1220 0x7f77180b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f771e7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f77180b1e00 0x7f77180afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f771e7ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77180d2ce0 con 0x7f771815c950 2023-07-18T03:38:54.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f771e7ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77180b3350 con 0x7f77180d1220 2023-07-18T03:38:54.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f771e7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77180b3630 con 0x7f77180b1e00 2023-07-18T03:38:54.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f77177fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f77180d1220 0x7f77180b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f7717fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f771815c950 0x7f77180d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35240/0 (socket says 172.21.15.49:35240) 2023-07-18T03:38:54.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f7717fff700 1 -- 172.21.15.49:0/1719723651 learned_addr learned my addr 172.21.15.49:0/1719723651 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:54.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f771cd9c700 1 --2- 172.21.15.49:0/1719723651 >> v2:172.21.15.107:3300/0 conn(0x7f77180b1e00 0x7f77180afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f77177fe700 1 -- 172.21.15.49:0/1719723651 >> v2:172.21.15.107:3300/0 conn(0x7f77180b1e00 msgr2=0x7f77180afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:54.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f77177fe700 1 --2- 172.21.15.49:0/1719723651 >> v2:172.21.15.107:3300/0 conn(0x7f77180b1e00 0x7f77180afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f77177fe700 1 -- 172.21.15.49:0/1719723651 >> v1:172.21.15.49:6789/0 conn(0x7f771815c950 legacy=0x7f77180d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:54.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f77177fe700 1 -- 172.21.15.49:0/1719723651 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180b3910 con 0x7f77180d1220 2023-07-18T03:38:54.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.080+0000 7f7716ffd700 1 -- 172.21.15.49:0/1719723651 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962313274 0 0) 0x7f77180d2ce0 con 0x7f771815c950 2023-07-18T03:38:54.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.081+0000 7f77177fe700 1 --2- 172.21.15.49:0/1719723651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77180d1220 0x7f77180b43f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f771808c300 tx=0x7f770c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda67de3e48fd5d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:54.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.081+0000 7f7716ffd700 1 -- 172.21.15.49:0/1719723651 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c011830 con 0x7f77180d1220 2023-07-18T03:38:54.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.081+0000 7f7716ffd700 1 -- 172.21.15.49:0/1719723651 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c011990 con 0x7f77180d1220 2023-07-18T03:38:54.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.081+0000 7f7716ffd700 1 -- 172.21.15.49:0/1719723651 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c011d20 con 0x7f77180d1220 2023-07-18T03:38:54.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.081+0000 7f771e7ff700 1 -- 172.21.15.49:0/1719723651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77180d1220 msgr2=0x7f77180b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:54.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.081+0000 7f771e7ff700 1 --2- 172.21.15.49:0/1719723651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77180d1220 0x7f77180b43f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f771808c300 tx=0x7f770c009b20 comp rx=0 tx=0).stop 2023-07-18T03:38:54.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.082+0000 7f771e7ff700 1 -- 172.21.15.49:0/1719723651 shutdown_connections 2023-07-18T03:38:54.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.082+0000 7f771e7ff700 1 --2- 172.21.15.49:0/1719723651 >> v2:172.21.15.107:3300/0 conn(0x7f77180b1e00 0x7f77180afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.082+0000 7f771e7ff700 1 --2- 172.21.15.49:0/1719723651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77180d1220 0x7f77180b43f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.082+0000 7f771e7ff700 1 -- 172.21.15.49:0/1719723651 >> 172.21.15.49:0/1719723651 conn(0x7f771809c1d0 msgr2=0x7f77180d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:54.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.082+0000 7f771e7ff700 1 -- 172.21.15.49:0/1719723651 shutdown_connections 2023-07-18T03:38:54.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.082+0000 7f771e7ff700 1 -- 172.21.15.49:0/1719723651 wait complete. 2023-07-18T03:38:54.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.083+0000 7f771e7ff700 1 Processor -- start 2023-07-18T03:38:54.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.084+0000 7f771e7ff700 1 -- start start 2023-07-18T03:38:54.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.084+0000 7f771e7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77180b1e00 0x7f77180c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.084+0000 7f771e7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77180d1220 0x7f77180c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f771e7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f771815c950 0x7f77180ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f771e7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77180c9950 con 0x7f77180d1220 2023-07-18T03:38:54.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f771e7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771814fba0 con 0x7f771815c950 2023-07-18T03:38:54.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f771e7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771815cd20 con 0x7f77180b1e00 2023-07-18T03:38:54.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f77177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77180d1220 0x7f77180c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f77177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77180d1220 0x7f77180c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34776/0 (socket says 172.21.15.49:34776) 2023-07-18T03:38:54.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f77177fe700 1 -- 172.21.15.49:0/142150634 learned_addr learned my addr 172.21.15.49:0/142150634 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:54.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f77177fe700 1 -- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77180b1e00 msgr2=0x7f77180c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:54.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f771cd9c700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f771815c950 0x7f77180ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f77177fe700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77180b1e00 0x7f77180c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f77177fe700 1 -- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f771815c950 msgr2=0x7f77180ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:54.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f77177fe700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f771815c950 0x7f77180ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f77177fe700 1 -- 172.21.15.49:0/142150634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c0094a0 con 0x7f77180d1220 2023-07-18T03:38:54.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f771cd9c700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f771815c950 0x7f77180ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:38:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.085+0000 7f77177fe700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77180d1220 0x7f77180c9410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f770c0090b0 tx=0x7f770c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.086+0000 7f7714ff9700 1 -- 172.21.15.49:0/142150634 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c0038a0 con 0x7f77180d1220 2023-07-18T03:38:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.086+0000 7f7714ff9700 1 -- 172.21.15.49:0/142150634 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c01f070 con 0x7f77180d1220 2023-07-18T03:38:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.086+0000 7f7714ff9700 1 -- 172.21.15.49:0/142150634 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c003c00 con 0x7f77180d1220 2023-07-18T03:38:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.086+0000 7f771e7ff700 1 -- 172.21.15.49:0/142150634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77180c7b60 con 0x7f77180d1220 2023-07-18T03:38:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.086+0000 7f771e7ff700 1 -- 172.21.15.49:0/142150634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77180c8120 con 0x7f77180d1220 2023-07-18T03:38:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.087+0000 7f771e7ff700 1 -- 172.21.15.49:0/142150634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771815cee0 con 0x7f77180d1220 2023-07-18T03:38:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.088+0000 7f7714ff9700 1 -- 172.21.15.49:0/142150634 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f770c01a430 con 0x7f77180d1220 2023-07-18T03:38:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.089+0000 7f7714ff9700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f770406ace0 0x7f770406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.089+0000 7f7714ff9700 1 -- 172.21.15.49:0/142150634 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f770c021070 con 0x7f77180d1220 2023-07-18T03:38:54.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.089+0000 7f7717fff700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f770406ace0 0x7f770406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.090+0000 7f7717fff700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f770406ace0 0x7f770406d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7708007fd0 tx=0x7f77080079a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:54.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.093+0000 7f7714ff9700 1 -- 172.21.15.49:0/142150634 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f770c055f60 con 0x7f77180d1220 2023-07-18T03:38:54.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.344+0000 7f771e7ff700 1 -- 172.21.15.49:0/142150634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f771804e280 con 0x7f77180d1220 2023-07-18T03:38:54.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.345+0000 7f7714ff9700 1 -- 172.21.15.49:0/142150634 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4709 (secure 0 0 0) 0x7f770c059580 con 0x7f77180d1220 2023-07-18T03:38:54.342 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:38:54.342 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1477016053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1477016053},{"type":"v1","addr":"172.21.15.107:6835","nonce":1477016053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2998451500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2998451500},{"type":"v1","addr":"172.21.15.49:6835","nonce":2998451500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4247281109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4247281109},{"type":"v1","addr":"172.21.15.49:6833","nonce":4247281109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:38:36.025442+0000","modified":"2023-07-18T03:38:39.535929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1127901907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1127901907},{"type":"v1","addr":"172.21.15.107:6837","nonce":1127901907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:38:54.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.347+0000 7f771e7ff700 1 -- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f770406ace0 msgr2=0x7f770406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:54.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.347+0000 7f771e7ff700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f770406ace0 0x7f770406d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7708007fd0 tx=0x7f77080079a0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.347+0000 7f771e7ff700 1 -- 172.21.15.49:0/142150634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77180d1220 msgr2=0x7f77180c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:54.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.347+0000 7f771e7ff700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77180d1220 0x7f77180c9410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f770c0090b0 tx=0x7f770c004750 comp rx=0 tx=0).stop 2023-07-18T03:38:54.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.347+0000 7f771e7ff700 1 -- 172.21.15.49:0/142150634 shutdown_connections 2023-07-18T03:38:54.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.347+0000 7f771e7ff700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f771815c950 0x7f77180ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.347+0000 7f771e7ff700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77180b1e00 0x7f77180c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.347+0000 7f771e7ff700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f770406ace0 0x7f770406d190 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.347+0000 7f771e7ff700 1 --2- 172.21.15.49:0/142150634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77180d1220 0x7f77180c9410 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.347+0000 7f771e7ff700 1 -- 172.21.15.49:0/142150634 >> 172.21.15.49:0/142150634 conn(0x7f771809c1d0 msgr2=0x7f77180b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:54.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.347+0000 7f771e7ff700 1 -- 172.21.15.49:0/142150634 shutdown_connections 2023-07-18T03:38:54.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.347+0000 7f771e7ff700 1 -- 172.21.15.49:0/142150634 wait complete. 2023-07-18T03:38:54.346 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2023-07-18T03:38:54.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-18T03:38:36.025442+0000', 'modified': '2023-07-18T03:38:39.535929+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.107:6837/1127901907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 1127901907}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 1127901907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-18T03:38:54.764 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-18T03:38:54.764 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-07-18T03:38:54.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.975+0000 7fea3dbff700 1 Processor -- start 2023-07-18T03:38:54.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.975+0000 7fea3dbff700 1 -- start start 2023-07-18T03:38:54.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.976+0000 7fea3dbff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fea380d0b10 0x7fea380d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.976+0000 7fea3dbff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea380b8e30 con 0x7fea380b8a60 2023-07-18T03:38:54.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.976+0000 7fea3dbff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea380afaa0 con 0x7fea380b0e20 2023-07-18T03:38:54.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.976+0000 7fea3dbff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea380d14d0 con 0x7fea380d0b10 2023-07-18T03:38:54.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.977+0000 7fea377fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fea380b8a60 0x7fea380b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35254/0 (socket says 172.21.15.49:35254) 2023-07-18T03:38:54.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.977+0000 7fea377fe700 1 -- 172.21.15.49:0/202646346 learned_addr learned my addr 172.21.15.49:0/202646346 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:38:54.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.977+0000 7fea36ffd700 1 --2- 172.21.15.49:0/202646346 >> v2:172.21.15.107:3300/0 conn(0x7fea380d0b10 0x7fea380d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.977+0000 7fea367fc700 1 -- 172.21.15.49:0/202646346 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327260099 0 0) 0x7fea380b8e30 con 0x7fea380b8a60 2023-07-18T03:38:54.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.977+0000 7fea367fc700 1 -- 172.21.15.49:0/202646346 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea200035b0 con 0x7fea380b8a60 2023-07-18T03:38:54.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.977+0000 7fea367fc700 1 -- 172.21.15.49:0/202646346 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688111474 0 0) 0x7fea380afaa0 con 0x7fea380b0e20 2023-07-18T03:38:54.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.977+0000 7fea367fc700 1 -- 172.21.15.49:0/202646346 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea380b8e30 con 0x7fea380b0e20 2023-07-18T03:38:54.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.977+0000 7fea36ffd700 1 -- 172.21.15.49:0/202646346 >> v1:172.21.15.107:6789/0 conn(0x7fea380b0e20 legacy=0x7fea380b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:54.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.977+0000 7fea36ffd700 1 -- 172.21.15.49:0/202646346 >> v1:172.21.15.49:6789/0 conn(0x7fea380b8a60 legacy=0x7fea380b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:54.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.977+0000 7fea36ffd700 1 -- 172.21.15.49:0/202646346 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea380afd80 con 0x7fea380d0b10 2023-07-18T03:38:54.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.978+0000 7fea36ffd700 1 --2- 172.21.15.49:0/202646346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea380d0b10 0x7fea380d0f90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009840 tx=0x7fea2c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=210c11a0ba82df17 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:54.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.978+0000 7fea367fc700 1 -- 172.21.15.49:0/202646346 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c011830 con 0x7fea380d0b10 2023-07-18T03:38:54.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.978+0000 7fea367fc700 1 -- 172.21.15.49:0/202646346 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c011990 con 0x7fea380d0b10 2023-07-18T03:38:54.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.978+0000 7fea367fc700 1 -- 172.21.15.49:0/202646346 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c011d20 con 0x7fea380d0b10 2023-07-18T03:38:54.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.978+0000 7fea3dbff700 1 -- 172.21.15.49:0/202646346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea380d0b10 msgr2=0x7fea380d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:54.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.978+0000 7fea3dbff700 1 --2- 172.21.15.49:0/202646346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea380d0b10 0x7fea380d0f90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009840 tx=0x7fea2c009b50 comp rx=0 tx=0).stop 2023-07-18T03:38:54.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.978+0000 7fea3dbff700 1 -- 172.21.15.49:0/202646346 shutdown_connections 2023-07-18T03:38:54.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.978+0000 7fea3dbff700 1 --2- 172.21.15.49:0/202646346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea380d0b10 0x7fea380d0f90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.978+0000 7fea3dbff700 1 -- 172.21.15.49:0/202646346 >> 172.21.15.49:0/202646346 conn(0x7fea3809c1b0 msgr2=0x7fea380d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:54.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.978+0000 7fea3dbff700 1 -- 172.21.15.49:0/202646346 shutdown_connections 2023-07-18T03:38:54.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.978+0000 7fea3dbff700 1 -- 172.21.15.49:0/202646346 wait complete. 2023-07-18T03:38:54.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.980+0000 7fea3dbff700 1 Processor -- start 2023-07-18T03:38:54.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.980+0000 7fea3dbff700 1 -- start start 2023-07-18T03:38:54.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.981+0000 7fea3dbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea380b0e20 0x7fea38149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.981+0000 7fea3dbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea380b8a60 0x7fea380cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.981+0000 7fea3dbff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea380d0b10 0x7fea380cf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.981+0000 7fea3dbff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea380cdf10 con 0x7fea380d0b10 2023-07-18T03:38:54.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.981+0000 7fea3dbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea380cfba0 con 0x7fea380b8a60 2023-07-18T03:38:54.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.981+0000 7fea3dbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea380cfe80 con 0x7fea380b0e20 2023-07-18T03:38:54.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.981+0000 7fea37fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea380d0b10 0x7fea380cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.982+0000 7fea37fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea380d0b10 0x7fea380cf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34788/0 (socket says 172.21.15.49:34788) 2023-07-18T03:38:54.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.982+0000 7fea37fff700 1 -- 172.21.15.49:0/1585072706 learned_addr learned my addr 172.21.15.49:0/1585072706 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:54.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.982+0000 7fea377fe700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea380b0e20 0x7fea38149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.982+0000 7fea36ffd700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea380b8a60 0x7fea380cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.982+0000 7fea37fff700 1 -- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea380b0e20 msgr2=0x7fea38149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:54.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.982+0000 7fea37fff700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea380b0e20 0x7fea38149b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.982+0000 7fea37fff700 1 -- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea380b8a60 msgr2=0x7fea380cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:54.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.982+0000 7fea37fff700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea380b8a60 0x7fea380cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.982+0000 7fea37fff700 1 -- 172.21.15.49:0/1585072706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0094a0 con 0x7fea380d0b10 2023-07-18T03:38:54.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.982+0000 7fea36ffd700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea380b8a60 0x7fea380cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:54.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.982+0000 7fea37fff700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea380d0b10 0x7fea380cf300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fea3000cc20 tx=0x7fea3000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.982+0000 7fea1ffff700 1 -- 172.21.15.49:0/1585072706 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30003c00 con 0x7fea380d0b10 2023-07-18T03:38:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.983+0000 7fea1ffff700 1 -- 172.21.15.49:0/1585072706 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea30003d60 con 0x7fea380d0b10 2023-07-18T03:38:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.983+0000 7fea3dbff700 1 -- 172.21.15.49:0/1585072706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea380d0160 con 0x7fea380d0b10 2023-07-18T03:38:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.983+0000 7fea1ffff700 1 -- 172.21.15.49:0/1585072706 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3000ab90 con 0x7fea380d0b10 2023-07-18T03:38:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.983+0000 7fea3dbff700 1 -- 172.21.15.49:0/1585072706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea380d0720 con 0x7fea380d0b10 2023-07-18T03:38:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.985+0000 7fea1ffff700 1 -- 172.21.15.49:0/1585072706 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fea3000acf0 con 0x7fea380d0b10 2023-07-18T03:38:54.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.986+0000 7fea1ffff700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fea2406ace0 0x7fea2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.986+0000 7fea1ffff700 1 -- 172.21.15.49:0/1585072706 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea30089690 con 0x7fea380d0b10 2023-07-18T03:38:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.986+0000 7fea377fe700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fea2406ace0 0x7fea2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.987+0000 7fea377fe700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fea2406ace0 0x7fea2406d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fea28007fd0 tx=0x7fea28007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.988+0000 7fea5157ab80 1 Processor -- start 2023-07-18T03:38:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.988+0000 7fea5157ab80 1 -- start start 2023-07-18T03:38:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.989+0000 7fea5157ab80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402cf00 0x55d41402d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.989+0000 7fea5157ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402d2d0 0x55d414031960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.989+0000 7fea5157ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d414031ea0 0x55d41404c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.989+0000 7fea5157ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d413f0bc60 con 0x55d41402d2d0 2023-07-18T03:38:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.989+0000 7fea5157ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d413f5e9f0 con 0x55d414031ea0 2023-07-18T03:38:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.989+0000 7fea5157ab80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d413f57e80 con 0x55d41402cf00 2023-07-18T03:38:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.989+0000 7fea36ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402d2d0 0x55d414031960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.989+0000 7fea36ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402d2d0 0x55d414031960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34794/0 (socket says 172.21.15.49:34794) 2023-07-18T03:38:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.989+0000 7fea36ffd700 1 -- 172.21.15.49:0/2672192884 learned_addr learned my addr 172.21.15.49:0/2672192884 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.989+0000 7fea377fe700 1 --2- 172.21.15.49:0/2672192884 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402cf00 0x55d41402d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.989+0000 7fea37fff700 1 --2- 172.21.15.49:0/2672192884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d414031ea0 0x55d41404c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.990+0000 7fea36ffd700 1 -- 172.21.15.49:0/2672192884 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402cf00 msgr2=0x55d41402d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:54.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.990+0000 7fea36ffd700 1 --2- 172.21.15.49:0/2672192884 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402cf00 0x55d41402d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.990+0000 7fea36ffd700 1 -- 172.21.15.49:0/2672192884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d414031ea0 msgr2=0x55d41404c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.990+0000 7fea36ffd700 1 --2- 172.21.15.49:0/2672192884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d414031ea0 0x55d41404c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.990+0000 7fea36ffd700 1 -- 172.21.15.49:0/2672192884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d413d85d90 con 0x55d41402d2d0 2023-07-18T03:38:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.990+0000 7fea37fff700 1 --2- 172.21.15.49:0/2672192884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d414031ea0 0x55d41404c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:54.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.990+0000 7fea377fe700 1 --2- 172.21.15.49:0/2672192884 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402cf00 0x55d41402d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:54.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.990+0000 7fea36ffd700 1 --2- 172.21.15.49:0/2672192884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402d2d0 0x55d414031960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fea2c000c00 tx=0x7fea2c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3376b6131f7d1294 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:54.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.990+0000 7fea1d7fa700 1 -- 172.21.15.49:0/2672192884 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c011b00 con 0x55d41402d2d0 2023-07-18T03:38:54.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.990+0000 7fea1d7fa700 1 -- 172.21.15.49:0/2672192884 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c011c60 con 0x55d41402d2d0 2023-07-18T03:38:54.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.990+0000 7fea1d7fa700 1 -- 172.21.15.49:0/2672192884 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c01a660 con 0x55d41402d2d0 2023-07-18T03:38:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.991+0000 7fea5157ab80 1 -- 172.21.15.49:0/2672192884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402d2d0 msgr2=0x55d414031960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.991+0000 7fea5157ab80 1 --2- 172.21.15.49:0/2672192884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402d2d0 0x55d414031960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fea2c000c00 tx=0x7fea2c008030 comp rx=0 tx=0).stop 2023-07-18T03:38:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.991+0000 7fea5157ab80 1 -- 172.21.15.49:0/2672192884 shutdown_connections 2023-07-18T03:38:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.991+0000 7fea5157ab80 1 --2- 172.21.15.49:0/2672192884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d414031ea0 0x55d41404c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.991+0000 7fea5157ab80 1 --2- 172.21.15.49:0/2672192884 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402cf00 0x55d41402d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.991+0000 7fea5157ab80 1 --2- 172.21.15.49:0/2672192884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402d2d0 0x55d414031960 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:54.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.991+0000 7fea5157ab80 1 -- 172.21.15.49:0/2672192884 >> 172.21.15.49:0/2672192884 conn(0x55d413ffbb90 msgr2=0x55d414001130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:54.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.991+0000 7fea5157ab80 1 -- 172.21.15.49:0/2672192884 shutdown_connections 2023-07-18T03:38:54.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.991+0000 7fea5157ab80 1 -- 172.21.15.49:0/2672192884 wait complete. 2023-07-18T03:38:54.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.992+0000 7fea5157ab80 1 -- start start 2023-07-18T03:38:54.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.992+0000 7fea5157ab80 1 Processor -- start 2023-07-18T03:38:54.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.993+0000 7fea5157ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402cf00 0x55d41404d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.993+0000 7fea5157ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402d2d0 0x55d414030c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.993+0000 7fea5157ab80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d414031ea0 0x55d41401aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.993+0000 7fea5157ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d413ef7a70 con 0x55d41402cf00 2023-07-18T03:38:55.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.993+0000 7fea5157ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d413ef7770 con 0x55d41402d2d0 2023-07-18T03:38:55.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.993+0000 7fea5157ab80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d413ddb8d0 con 0x55d414031ea0 2023-07-18T03:38:55.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.993+0000 7fea5157ab80 10 client.0 _ll_register_callbacks cb 0x55d413ef5710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:55.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.993+0000 7fea377fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402cf00 0x55d41404d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.993+0000 7fea377fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402cf00 0x55d41404d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34808/0 (socket says 172.21.15.49:34808) 2023-07-18T03:38:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.993+0000 7fea377fe700 1 -- 172.21.15.49:0/1606350457 learned_addr learned my addr 172.21.15.49:0/1606350457 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.993+0000 7fea37fff700 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d414031ea0 0x55d41401aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.993+0000 7fea36ffd700 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402d2d0 0x55d414030c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.994+0000 7fea377fe700 1 -- 172.21.15.49:0/1606350457 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d414031ea0 msgr2=0x55d41401aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.994+0000 7fea377fe700 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d414031ea0 0x55d41401aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.994+0000 7fea377fe700 1 -- 172.21.15.49:0/1606350457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402d2d0 msgr2=0x55d414030c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.994+0000 7fea377fe700 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402d2d0 0x55d414030c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.994+0000 7fea377fe700 1 -- 172.21.15.49:0/1606350457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea380afd80 con 0x55d41402cf00 2023-07-18T03:38:55.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.994+0000 7fea377fe700 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402cf00 0x55d41404d5e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea28006630 tx=0x7fea28001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e72c6dc2928a9f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:55.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.994+0000 7fea167fc700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-07-18T03:38:55.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.994+0000 7fea167fc700 1 -- 172.21.15.49:0/1606350457 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28026070 con 0x55d41402cf00 2023-07-18T03:38:55.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.994+0000 7fea167fc700 1 -- 172.21.15.49:0/1606350457 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea2801b070 con 0x55d41402cf00 2023-07-18T03:38:55.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.994+0000 7fea5157ab80 1 -- 172.21.15.49:0/1606350457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d413ff91a0 con 0x55d41402cf00 2023-07-18T03:38:55.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.995+0000 7fea167fc700 1 -- 172.21.15.49:0/1606350457 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea28001580 con 0x55d41402cf00 2023-07-18T03:38:55.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.995+0000 7fea5157ab80 10 client.4689 fetch_fsmap learned FSMap version 8 2023-07-18T03:38:55.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.995+0000 7fea5157ab80 1 -- 172.21.15.49:0/1606350457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55d413d6ec10 con 0x55d41402cf00 2023-07-18T03:38:55.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.995+0000 7fea167fc700 1 -- 172.21.15.49:0/1606350457 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea28030020 con 0x55d41402cf00 2023-07-18T03:38:55.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.995+0000 7fea167fc700 1 -- 172.21.15.49:0/1606350457 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fea28029020 con 0x55d41402cf00 2023-07-18T03:38:55.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.995+0000 7fea5157ab80 10 client.4689 fetch_fsmap finished waiting for FSMap version 8 2023-07-18T03:38:55.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.995+0000 7fea5157ab80 10 client.4689 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-18T03:38:55.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.995+0000 7fea5157ab80 20 client.4689 populate_metadata read hostname 'smithi049' 2023-07-18T03:38:55.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.995+0000 7fea5157ab80 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.107:6836/1127901907,v1:172.21.15.107:6837/1127901907] conn(0x55d414001150 0x55d414001520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.995+0000 7fea5157ab80 4 client.4689 mds_command: new command op to 4288 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:38:55.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.995+0000 7fea5157ab80 1 -- 172.21.15.49:0/1606350457 --> [v2:172.21.15.107:6836/1127901907,v1:172.21.15.107:6837/1127901907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d413f50e60 con 0x55d414001150 2023-07-18T03:38:55.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.995+0000 7fea36ffd700 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.107:6836/1127901907,v1:172.21.15.107:6837/1127901907] conn(0x55d414001150 0x55d414001520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.996+0000 7fea36ffd700 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.107:6836/1127901907,v1:172.21.15.107:6837/1127901907] conn(0x55d414001150 0x55d414001520 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8800faedd10709 server_cookie=e9653ac8c45b58ff in_seq=0 out_seq=0 2023-07-18T03:38:55.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.996+0000 7fea167fc700 10 client.4689 ms_handle_connect on v2:172.21.15.107:6836/1127901907 2023-07-18T03:38:55.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea167fc700 1 -- 172.21.15.49:0/1606350457 <== mds.0 v2:172.21.15.107:6836/1127901907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d413f50e60 con 0x55d414001150 2023-07-18T03:38:55.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea167fc700 10 client.4689 handle_command_reply: tid=0 2023-07-18T03:38:55.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea5157ab80 1 client.4689 shutdown 2023-07-18T03:38:55.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea5157ab80 1 -- 172.21.15.49:0/1606350457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402cf00 msgr2=0x55d41404d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea5157ab80 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402cf00 0x55d41404d5e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea28006630 tx=0x7fea28001150 comp rx=0 tx=0).stop 2023-07-18T03:38:55.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea5157ab80 1 -- 172.21.15.49:0/1606350457 shutdown_connections 2023-07-18T03:38:55.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea5157ab80 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.107:6836/1127901907,v1:172.21.15.107:6837/1127901907] conn(0x55d414001150 0x55d414001520 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea5157ab80 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d414031ea0 0x55d41401aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea5157ab80 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d41402cf00 0x55d41404d5e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea5157ab80 1 --2- 172.21.15.49:0/1606350457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402d2d0 0x55d414030c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea167fc700 0 client.4689 ms_handle_reset on v2:172.21.15.107:6836/1127901907 2023-07-18T03:38:55.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea5157ab80 1 -- 172.21.15.49:0/1606350457 >> 172.21.15.49:0/1606350457 conn(0x55d413ffbb90 msgr2=0x55d414005810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:55.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea5157ab80 1 -- 172.21.15.49:0/1606350457 shutdown_connections 2023-07-18T03:38:55.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.997+0000 7fea5157ab80 1 -- 172.21.15.49:0/1606350457 wait complete. 2023-07-18T03:38:55.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:54.998+0000 7fea5157ab80 20 client.4689 trim_cache size 0 max 16384 2023-07-18T03:38:55.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.016+0000 7fea5157ab80 1 Processor -- start 2023-07-18T03:38:55.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.016+0000 7fea5157ab80 1 -- start start 2023-07-18T03:38:55.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea5157ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402cf00 0x55d4140ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea5157ab80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402d2d0 0x55d413fffe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea5157ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 0x55d414000360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea5157ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d414005b00 con 0x55d414031ea0 2023-07-18T03:38:55.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea5157ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4140020c0 con 0x55d41402cf00 2023-07-18T03:38:55.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea5157ab80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d41401b5b0 con 0x55d41402d2d0 2023-07-18T03:38:55.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea37fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 0x55d414000360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea37fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 0x55d414000360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34810/0 (socket says 172.21.15.49:34810) 2023-07-18T03:38:55.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea37fff700 1 -- 172.21.15.49:0/2808834147 learned_addr learned my addr 172.21.15.49:0/2808834147 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:55.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea377fe700 1 --2- 172.21.15.49:0/2808834147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402cf00 0x55d4140ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea36ffd700 1 --2- 172.21.15.49:0/2808834147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402d2d0 0x55d413fffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea37fff700 1 -- 172.21.15.49:0/2808834147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402d2d0 msgr2=0x55d413fffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea37fff700 1 --2- 172.21.15.49:0/2808834147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402d2d0 0x55d413fffe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea37fff700 1 -- 172.21.15.49:0/2808834147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402cf00 msgr2=0x55d4140ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea37fff700 1 --2- 172.21.15.49:0/2808834147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402cf00 0x55d4140ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.017+0000 7fea37fff700 1 -- 172.21.15.49:0/2808834147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d413dac200 con 0x55d414031ea0 2023-07-18T03:38:55.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.018+0000 7fea37fff700 1 --2- 172.21.15.49:0/2808834147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 0x55d414000360 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea30060180 tx=0x7fea30089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b80454a168984e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:55.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.018+0000 7fea1dffb700 1 -- 172.21.15.49:0/2808834147 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300564a0 con 0x55d414031ea0 2023-07-18T03:38:55.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.018+0000 7fea1dffb700 1 -- 172.21.15.49:0/2808834147 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea30056700 con 0x55d414031ea0 2023-07-18T03:38:55.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.018+0000 7fea1dffb700 1 -- 172.21.15.49:0/2808834147 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30057850 con 0x55d414031ea0 2023-07-18T03:38:55.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.018+0000 7fea5157ab80 1 -- 172.21.15.49:0/2808834147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 msgr2=0x55d414000360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.018+0000 7fea5157ab80 1 --2- 172.21.15.49:0/2808834147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 0x55d414000360 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea30060180 tx=0x7fea30089040 comp rx=0 tx=0).stop 2023-07-18T03:38:55.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.018+0000 7fea5157ab80 1 -- 172.21.15.49:0/2808834147 shutdown_connections 2023-07-18T03:38:55.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.018+0000 7fea5157ab80 1 --2- 172.21.15.49:0/2808834147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 0x55d414000360 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.018+0000 7fea5157ab80 1 --2- 172.21.15.49:0/2808834147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402cf00 0x55d4140ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.018+0000 7fea5157ab80 1 --2- 172.21.15.49:0/2808834147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402d2d0 0x55d413fffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.018+0000 7fea5157ab80 1 -- 172.21.15.49:0/2808834147 >> 172.21.15.49:0/2808834147 conn(0x55d413ffbb90 msgr2=0x55d41402c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:55.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.018+0000 7fea5157ab80 1 -- 172.21.15.49:0/2808834147 shutdown_connections 2023-07-18T03:38:55.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.019+0000 7fea5157ab80 1 -- 172.21.15.49:0/2808834147 wait complete. 2023-07-18T03:38:55.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.019+0000 7fea5157ab80 1 -- start start 2023-07-18T03:38:55.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.019+0000 7fea5157ab80 1 Processor -- start 2023-07-18T03:38:55.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.020+0000 7fea5157ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402cf00 0x55d414004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.020+0000 7fea5157ab80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402d2d0 0x55d4140ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.020+0000 7fea5157ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 0x55d4140b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.020+0000 7fea5157ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d41404ec70 con 0x55d414031ea0 2023-07-18T03:38:55.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.020+0000 7fea5157ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4140b6ed0 con 0x55d41402cf00 2023-07-18T03:38:55.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.020+0000 7fea5157ab80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d414004b60 con 0x55d41402d2d0 2023-07-18T03:38:55.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.020+0000 7fea5157ab80 10 client.0 _ll_register_callbacks cb 0x55d413ef5710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:55.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea37fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 0x55d4140b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea37fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 0x55d4140b66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34826/0 (socket says 172.21.15.49:34826) 2023-07-18T03:38:55.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea37fff700 1 -- 172.21.15.49:0/1535639517 learned_addr learned my addr 172.21.15.49:0/1535639517 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:55.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea37fff700 1 -- 172.21.15.49:0/1535639517 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402d2d0 msgr2=0x55d4140ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea37fff700 1 --2- 172.21.15.49:0/1535639517 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402d2d0 0x55d4140ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea37fff700 1 -- 172.21.15.49:0/1535639517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402cf00 msgr2=0x55d414004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea37fff700 1 --2- 172.21.15.49:0/1535639517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402cf00 0x55d414004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea37fff700 1 -- 172.21.15.49:0/1535639517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea380d0720 con 0x55d414031ea0 2023-07-18T03:38:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea37fff700 1 --2- 172.21.15.49:0/1535639517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 0x55d4140b66a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fea300582f0 tx=0x7fea30001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ff88beacda696f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea167fc700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-07-18T03:38:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea167fc700 1 -- 172.21.15.49:0/1535639517 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30057a50 con 0x55d414031ea0 2023-07-18T03:38:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea167fc700 1 -- 172.21.15.49:0/1535639517 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea30060220 con 0x55d414031ea0 2023-07-18T03:38:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.021+0000 7fea5157ab80 1 -- 172.21.15.49:0/1535639517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea2c011c60 con 0x55d414031ea0 2023-07-18T03:38:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.022+0000 7fea167fc700 1 -- 172.21.15.49:0/1535639517 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea30060400 con 0x55d414031ea0 2023-07-18T03:38:55.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.022+0000 7fea5157ab80 10 client.4695 fetch_fsmap learned FSMap version 8 2023-07-18T03:38:55.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.022+0000 7fea5157ab80 1 -- 172.21.15.49:0/1535639517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55d413d84260 con 0x55d414031ea0 2023-07-18T03:38:55.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.022+0000 7fea167fc700 1 -- 172.21.15.49:0/1535639517 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c011c60 con 0x55d414031ea0 2023-07-18T03:38:55.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.022+0000 7fea167fc700 1 -- 172.21.15.49:0/1535639517 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fea30081b00 con 0x55d414031ea0 2023-07-18T03:38:55.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.022+0000 7fea5157ab80 10 client.4695 fetch_fsmap finished waiting for FSMap version 8 2023-07-18T03:38:55.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.022+0000 7fea5157ab80 10 client.4695 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-18T03:38:55.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.022+0000 7fea5157ab80 20 client.4695 populate_metadata read hostname 'smithi049' 2023-07-18T03:38:55.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.022+0000 7fea5157ab80 1 --2- 172.21.15.49:0/1535639517 >> [v2:172.21.15.107:6836/1127901907,v1:172.21.15.107:6837/1127901907] conn(0x55d41404e5a0 0x55d414002fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.022+0000 7fea5157ab80 4 client.4695 mds_command: new command op to 4288 tid=0[{"prefix": "status"}] 2023-07-18T03:38:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.022+0000 7fea5157ab80 1 -- 172.21.15.49:0/1535639517 --> [v2:172.21.15.107:6836/1127901907,v1:172.21.15.107:6837/1127901907] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d413f13000 con 0x55d41404e5a0 2023-07-18T03:38:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.022+0000 7fea377fe700 1 --2- 172.21.15.49:0/1535639517 >> [v2:172.21.15.107:6836/1127901907,v1:172.21.15.107:6837/1127901907] conn(0x55d41404e5a0 0x55d414002fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea377fe700 1 --2- 172.21.15.49:0/1535639517 >> [v2:172.21.15.107:6836/1127901907,v1:172.21.15.107:6837/1127901907] conn(0x55d41404e5a0 0x55d414002fd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d62f61b228da36f server_cookie=53ee169d18663d12 in_seq=0 out_seq=0 2023-07-18T03:38:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea167fc700 10 client.4695 ms_handle_connect on v2:172.21.15.107:6836/1127901907 2023-07-18T03:38:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea167fc700 1 -- 172.21.15.49:0/1535639517 <== mds.0 v2:172.21.15.107:6836/1127901907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x7fea28030150 con 0x55d41404e5a0 2023-07-18T03:38:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea167fc700 10 client.4695 handle_command_reply: tid=0 2023-07-18T03:38:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea5157ab80 1 client.4695 shutdown 2023-07-18T03:38:55.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea5157ab80 1 -- 172.21.15.49:0/1535639517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 msgr2=0x55d4140b66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea5157ab80 1 --2- 172.21.15.49:0/1535639517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 0x55d4140b66a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fea300582f0 tx=0x7fea30001510 comp rx=0 tx=0).stop 2023-07-18T03:38:55.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea5157ab80 1 -- 172.21.15.49:0/1535639517 shutdown_connections 2023-07-18T03:38:55.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea5157ab80 1 --2- 172.21.15.49:0/1535639517 >> [v2:172.21.15.107:6836/1127901907,v1:172.21.15.107:6837/1127901907] conn(0x55d41404e5a0 0x55d414002fd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea5157ab80 1 --2- 172.21.15.49:0/1535639517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d414031ea0 0x55d4140b66a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea5157ab80 1 --2- 172.21.15.49:0/1535639517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d41402cf00 0x55d414004ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea5157ab80 1 --2- 172.21.15.49:0/1535639517 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d41402d2d0 0x55d4140ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea167fc700 0 client.4695 ms_handle_reset on v2:172.21.15.107:6836/1127901907 2023-07-18T03:38:55.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.023+0000 7fea5157ab80 1 -- 172.21.15.49:0/1535639517 >> 172.21.15.49:0/1535639517 conn(0x55d413ffbb90 msgr2=0x55d41404c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:55.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea5157ab80 1 -- 172.21.15.49:0/1535639517 shutdown_connections 2023-07-18T03:38:55.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea5157ab80 1 -- 172.21.15.49:0/1535639517 wait complete. 2023-07-18T03:38:55.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea5157ab80 20 client.4695 trim_cache size 0 max 16384 2023-07-18T03:38:55.027 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:38:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "e859a751-7613-40d7-973c-3e47a5e301cb", 2023-07-18T03:38:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-07-18T03:38:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4288, 2023-07-18T03:38:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-07-18T03:38:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-07-18T03:38:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-07-18T03:38:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 18.494681376999999, 2023-07-18T03:38:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 8, 2023-07-18T03:38:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 22, 2023-07-18T03:38:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 22, 2023-07-18T03:38:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 25.485147627 2023-07-18T03:38:55.029 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:38:55.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea3dbff700 1 -- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fea2406ace0 msgr2=0x7fea2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea3dbff700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fea2406ace0 0x7fea2406d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fea28007fd0 tx=0x7fea28007dc0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea3dbff700 1 -- 172.21.15.49:0/1585072706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea380d0b10 msgr2=0x7fea380cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea3dbff700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea380d0b10 0x7fea380cf300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fea3000cc20 tx=0x7fea3000cf30 comp rx=0 tx=0).stop 2023-07-18T03:38:55.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea3dbff700 1 -- 172.21.15.49:0/1585072706 shutdown_connections 2023-07-18T03:38:55.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea3dbff700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fea2406ace0 0x7fea2406d190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea3dbff700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea380d0b10 0x7fea380cf300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea3dbff700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea380b0e20 0x7fea38149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea3dbff700 1 --2- 172.21.15.49:0/1585072706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea380b8a60 0x7fea380cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.024+0000 7fea3dbff700 1 -- 172.21.15.49:0/1585072706 >> 172.21.15.49:0/1585072706 conn(0x7fea3809c1b0 msgr2=0x7fea380d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:55.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.025+0000 7fea3dbff700 1 -- 172.21.15.49:0/1585072706 shutdown_connections 2023-07-18T03:38:55.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.025+0000 7fea3dbff700 1 -- 172.21.15.49:0/1585072706 wait complete. 2023-07-18T03:38:55.041 INFO:teuthology.run_tasks:Running task kclient... 2023-07-18T03:38:55.064 INFO:tasks.kclient:Mounting kernel clients... 2023-07-18T03:38:55.064 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-07-18T03:38:55.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:55.065 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-07-18T03:38:55.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:55.110 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2023-07-18T03:38:55.224 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2023-07-18T03:38:55.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:38:55.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:55.286 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-07-18T03:38:55.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:55.334 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2023-07-18T03:38:55.456 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2023-07-18T03:38:55.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:38:55.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:55.459 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-18T03:38:55.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:55.511 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2023-07-18T03:38:55.586 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2023-07-18T03:38:55.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:38:55.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:55.591 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-18T03:38:55.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:55.635 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2023-07-18T03:38:55.746 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2023-07-18T03:38:55.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:38:55.749 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-07-18T03:38:55.750 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-18T03:38:55.750 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-18T03:38:55.751 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-18T03:38:55.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.961+0000 7fb4297ff700 1 Processor -- start 2023-07-18T03:38:55.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.962+0000 7fb4297ff700 1 -- start start 2023-07-18T03:38:55.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.963+0000 7fb4297ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb4240d1220 0x7fb4240b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.963+0000 7fb4297ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4240b1e00 0x7fb4240afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.963+0000 7fb4297ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4240d2ce0 con 0x7fb42415c950 2023-07-18T03:38:55.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.963+0000 7fb4297ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4240b3350 con 0x7fb4240d1220 2023-07-18T03:38:55.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.963+0000 7fb4297ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4240b3630 con 0x7fb4240b1e00 2023-07-18T03:38:55.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.963+0000 7fb4237fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4240b1e00 0x7fb4240afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.963+0000 7fb4237fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4240b1e00 0x7fb4240afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:41904/0 (socket says 172.21.15.49:41904) 2023-07-18T03:38:55.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.963+0000 7fb4237fe700 1 -- 172.21.15.49:0/1335580272 learned_addr learned my addr 172.21.15.49:0/1335580272 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:55.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.963+0000 7fb41bfff700 1 --2- 172.21.15.49:0/1335580272 >> v2:172.21.15.107:3300/0 conn(0x7fb4240d1220 0x7fb4240b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.964+0000 7fb4237fe700 1 -- 172.21.15.49:0/1335580272 >> v2:172.21.15.107:3300/0 conn(0x7fb4240d1220 msgr2=0x7fb4240b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.964+0000 7fb4237fe700 1 --2- 172.21.15.49:0/1335580272 >> v2:172.21.15.107:3300/0 conn(0x7fb4240d1220 0x7fb4240b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.964+0000 7fb4227fc700 1 -- 172.21.15.49:0/1335580272 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805291102 0 0) 0x7fb4240d2ce0 con 0x7fb42415c950 2023-07-18T03:38:55.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.964+0000 7fb4237fe700 1 -- 172.21.15.49:0/1335580272 >> v1:172.21.15.107:6789/0 conn(0x7fb42415c950 legacy=0x7fb4240d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.964+0000 7fb4237fe700 1 -- 172.21.15.49:0/1335580272 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4240b3910 con 0x7fb4240b1e00 2023-07-18T03:38:55.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.964+0000 7fb4237fe700 1 --2- 172.21.15.49:0/1335580272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4240b1e00 0x7fb4240afac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009e50 tx=0x7fb41c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2bd7dab22189f8e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:55.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.964+0000 7fb4227fc700 1 -- 172.21.15.49:0/1335580272 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00e070 con 0x7fb4240b1e00 2023-07-18T03:38:55.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.964+0000 7fb4227fc700 1 -- 172.21.15.49:0/1335580272 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c00b910 con 0x7fb4240b1e00 2023-07-18T03:38:55.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.964+0000 7fb4227fc700 1 -- 172.21.15.49:0/1335580272 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00bcc0 con 0x7fb4240b1e00 2023-07-18T03:38:55.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.965+0000 7fb4297ff700 1 -- 172.21.15.49:0/1335580272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4240b1e00 msgr2=0x7fb4240afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.965+0000 7fb4297ff700 1 --2- 172.21.15.49:0/1335580272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4240b1e00 0x7fb4240afac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009e50 tx=0x7fb41c00b330 comp rx=0 tx=0).stop 2023-07-18T03:38:55.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.965+0000 7fb4297ff700 1 -- 172.21.15.49:0/1335580272 shutdown_connections 2023-07-18T03:38:55.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.965+0000 7fb4297ff700 1 --2- 172.21.15.49:0/1335580272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4240b1e00 0x7fb4240afac0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.965+0000 7fb4297ff700 1 --2- 172.21.15.49:0/1335580272 >> v2:172.21.15.107:3300/0 conn(0x7fb4240d1220 0x7fb4240b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.965+0000 7fb4297ff700 1 -- 172.21.15.49:0/1335580272 >> 172.21.15.49:0/1335580272 conn(0x7fb42409c1d0 msgr2=0x7fb4240d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:55.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.965+0000 7fb4297ff700 1 -- 172.21.15.49:0/1335580272 shutdown_connections 2023-07-18T03:38:55.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.965+0000 7fb4297ff700 1 -- 172.21.15.49:0/1335580272 wait complete. 2023-07-18T03:38:55.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.967+0000 7fb4297ff700 1 Processor -- start 2023-07-18T03:38:55.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.967+0000 7fb4297ff700 1 -- start start 2023-07-18T03:38:55.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.968+0000 7fb4297ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4240b1e00 0x7fb4241527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.968+0000 7fb4297ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4240d1220 0x7fb4240cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.968+0000 7fb4297ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42415c950 0x7fb4240cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.968+0000 7fb4297ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4240c7a60 con 0x7fb4240d1220 2023-07-18T03:38:55.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.968+0000 7fb4297ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4240d1f10 con 0x7fb4240b1e00 2023-07-18T03:38:55.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.968+0000 7fb4297ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4240c6e50 con 0x7fb42415c950 2023-07-18T03:38:55.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.968+0000 7fb41bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4240d1220 0x7fb4240cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.968+0000 7fb41bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4240d1220 0x7fb4240cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34830/0 (socket says 172.21.15.49:34830) 2023-07-18T03:38:55.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.968+0000 7fb41bfff700 1 -- 172.21.15.49:0/350932338 learned_addr learned my addr 172.21.15.49:0/350932338 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:38:55.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.968+0000 7fb422ffd700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4240b1e00 0x7fb4241527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.969+0000 7fb4237fe700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42415c950 0x7fb4240cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.969+0000 7fb4237fe700 1 -- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4240b1e00 msgr2=0x7fb4241527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.969+0000 7fb4237fe700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4240b1e00 0x7fb4241527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.969+0000 7fb4237fe700 1 -- 172.21.15.49:0/350932338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4240d1220 msgr2=0x7fb4240cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.969+0000 7fb4237fe700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4240d1220 0x7fb4240cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.969+0000 7fb4237fe700 1 -- 172.21.15.49:0/350932338 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c009b30 con 0x7fb42415c950 2023-07-18T03:38:55.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.969+0000 7fb41bfff700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4240d1220 0x7fb4240cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:55.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.969+0000 7fb4237fe700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42415c950 0x7fb4240cd890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb41c015000 tx=0x7fb41c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:55.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.969+0000 7fb41b7fe700 1 -- 172.21.15.49:0/350932338 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c004880 con 0x7fb42415c950 2023-07-18T03:38:55.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.969+0000 7fb41b7fe700 1 -- 172.21.15.49:0/350932338 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c0099b0 con 0x7fb42415c950 2023-07-18T03:38:55.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.969+0000 7fb4297ff700 1 -- 172.21.15.49:0/350932338 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4240c7130 con 0x7fb42415c950 2023-07-18T03:38:55.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.969+0000 7fb41b7fe700 1 -- 172.21.15.49:0/350932338 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c01c680 con 0x7fb42415c950 2023-07-18T03:38:55.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.970+0000 7fb4297ff700 1 -- 172.21.15.49:0/350932338 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4240c76f0 con 0x7fb42415c950 2023-07-18T03:38:55.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.970+0000 7fb4297ff700 1 -- 172.21.15.49:0/350932338 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4240c7880 con 0x7fb42415c950 2023-07-18T03:38:55.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.972+0000 7fb41b7fe700 1 -- 172.21.15.49:0/350932338 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb41c01c7e0 con 0x7fb42415c950 2023-07-18T03:38:55.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.973+0000 7fb41b7fe700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb40406ace0 0x7fb40406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.973+0000 7fb41b7fe700 1 -- 172.21.15.49:0/350932338 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb41c08af00 con 0x7fb42415c950 2023-07-18T03:38:55.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.973+0000 7fb422ffd700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb40406ace0 0x7fb40406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.973+0000 7fb422ffd700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb40406ace0 0x7fb40406d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb414000fb0 tx=0x7fb414009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:55.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:55.976+0000 7fb41b7fe700 1 -- 172.21.15.49:0/350932338 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb41c056020 con 0x7fb42415c950 2023-07-18T03:38:56.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.231+0000 7fb4297ff700 1 -- 172.21.15.49:0/350932338 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb42404e280 con 0x7fb42415c950 2023-07-18T03:38:56.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.232+0000 7fb41b7fe700 1 -- 172.21.15.49:0/350932338 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb41c059640 con 0x7fb42415c950 2023-07-18T03:38:56.228 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-18T03:38:56.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.234+0000 7fb4297ff700 1 -- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb40406ace0 msgr2=0x7fb40406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.234+0000 7fb4297ff700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb40406ace0 0x7fb40406d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb414000fb0 tx=0x7fb414009040 comp rx=0 tx=0).stop 2023-07-18T03:38:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.234+0000 7fb4297ff700 1 -- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42415c950 msgr2=0x7fb4240cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.234+0000 7fb4297ff700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42415c950 0x7fb4240cd890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb41c015000 tx=0x7fb41c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:38:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.235+0000 7fb4297ff700 1 -- 172.21.15.49:0/350932338 shutdown_connections 2023-07-18T03:38:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.235+0000 7fb4297ff700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42415c950 0x7fb4240cd890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.235+0000 7fb4297ff700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4240b1e00 0x7fb4241527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.235+0000 7fb4297ff700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb40406ace0 0x7fb40406d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:56.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.235+0000 7fb4297ff700 1 --2- 172.21.15.49:0/350932338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4240d1220 0x7fb4240cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:56.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.235+0000 7fb4297ff700 1 -- 172.21.15.49:0/350932338 >> 172.21.15.49:0/350932338 conn(0x7fb42409c1d0 msgr2=0x7fb4240b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:56.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.235+0000 7fb4297ff700 1 -- 172.21.15.49:0/350932338 shutdown_connections 2023-07-18T03:38:56.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:38:56.235+0000 7fb4297ff700 1 -- 172.21.15.49:0/350932338 wait complete. 2023-07-18T03:38:56.249 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-18T03:38:56.249 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-18T03:38:56.249 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-07-18T03:38:56.250 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-18T03:38:56.250 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-18T03:38:56.250 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-18T03:38:56.250 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-18T03:38:56.250 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-18T03:38:56.251 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-18T03:38:56.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:56.251 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-07-18T03:38:56.295 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:38:56.296 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:38:56.296 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:38:56.296 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:38:56.296 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-07-18T03:38:56.296 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:38:56.297 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:38:56.297 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:38:56.297 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-07-18T03:38:56.297 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:38:56.297 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:38:56.297 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-07-18T03:38:56.297 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-18T03:38:56.297 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:38:56.298 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-07-18T03:38:56.298 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-18T03:38:56.298 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 41971sec preferred_lft 41971sec 2023-07-18T03:38:56.298 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-07-18T03:38:56.298 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:38:56.298 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:38:56.298 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:38:56.299 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-07-18T03:38:56.299 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-18T03:38:56.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:56.300 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-07-18T03:38:56.300 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-07-18T03:38:56.300 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2023-07-18T03:38:56.301 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2023-07-18T03:38:56.301 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2023-07-18T03:38:56.301 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-18T03:38:56.301 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-07-18T03:38:56.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:56.768 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-18T03:38:56.842 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-07-18T03:38:56.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:56.846 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-07-18T03:38:56.912 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-07-18T03:38:56.913 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-18T03:38:56.913 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-18T03:38:56.913 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-18T03:38:56.914 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-18T03:38:56.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:56.915 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-07-18T03:38:56.916 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-07-18T03:38:56.916 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-18T03:38:56.917 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-18T03:38:56.917 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-18T03:38:56.917 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-07-18T03:38:57.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:57.651 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-07-18T03:38:57.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:57.697 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2023-07-18T03:38:57.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:57.782 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-07-18T03:38:57.783 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-07-18T03:38:57.783 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-18T03:38:57.783 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-18T03:38:57.784 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-07-18T03:38:57.968 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-18T03:38:57.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:57.968 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-07-18T03:38:57.969 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-07-18T03:38:57.969 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-18T03:38:57.969 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-18T03:38:57.969 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-18T03:38:57.970 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-18T03:38:57.970 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-18T03:38:57.970 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-07-18T03:38:58.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:58.262 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-07-18T03:38:58.262 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-07-18T03:38:58.263 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-07-18T03:38:58.263 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2023-07-18T03:38:58.264 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-07-18T03:38:58.441 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:38:58.442 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-18T03:38:58.486 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-18T03:38:58.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:38:58.487 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-18T03:38:58.570 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-07-18T03:38:58.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:58.570 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:38:58.740 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:38:58.740 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:38:59.199 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:38:59.200 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:38:59.200 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2023-07-18T03:38:59.200 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with new device syntax: 0@e859a751-7613-40d7-973c-3e47a5e301cb.cephfs=/ 2023-07-18T03:38:59.201 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.49:3300/172.21.15.107:3300/172.21.15.107:3301" will pass to kernel 2023-07-18T03:38:59.205 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-18T03:38:59.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:59.205 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-18T03:38:59.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:59.278 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:38:59.412 INFO:teuthology.orchestra.run.smithi049.stdout:client4469 2023-07-18T03:38:59.413 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:38:59.414 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:38:59.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:59.459 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:38:59.587 INFO:teuthology.orchestra.run.smithi049.stdout:client4469 2023-07-18T03:38:59.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:59.588 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:38:59.676 INFO:teuthology.orchestra.run.smithi049.stdout:client4469 2023-07-18T03:38:59.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:38:59.677 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e859a751-7613-40d7-973c-3e47a5e301cb.client4469/status') 2023-07-18T03:38:59.787 INFO:teuthology.orchestra.run.smithi049.stdout:instance: client.4469 (3)172.21.15.49:0/510021139 2023-07-18T03:38:59.787 INFO:teuthology.orchestra.run.smithi049.stdout:blocklisted: false 2023-07-18T03:38:59.787 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records in 2023-07-18T03:38:59.788 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records out 2023-07-18T03:38:59.788 INFO:teuthology.orchestra.run.smithi049.stderr:69 bytes copied, 5.8022e-05 s, 1.2 MB/s 2023-07-18T03:38:59.790 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-07-18T03:38:59.790 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-18T03:38:59.791 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-18T03:38:59.791 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-18T03:38:59.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.002+0000 7f835adff700 1 Processor -- start 2023-07-18T03:38:59.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.003+0000 7f835adff700 1 -- start start 2023-07-18T03:39:00.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.004+0000 7f835adff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835415c950 0x7f83540b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:00.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.004+0000 7f835adff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f83540b1e00 0x7f83540d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:00.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.004+0000 7f835adff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f83540b2350 0x7f83540d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:00.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.004+0000 7f835adff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83540d2ce0 con 0x7f83540b2350 2023-07-18T03:39:00.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.004+0000 7f835adff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83540b3350 con 0x7f83540b1e00 2023-07-18T03:39:00.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.004+0000 7f835adff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83540b3630 con 0x7f835415c950 2023-07-18T03:39:00.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.004+0000 7f835939c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f83540b2350 0x7f83540d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:00.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.004+0000 7f835939c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f83540b2350 0x7f83540d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60802/0 (socket says 172.21.15.49:60802) 2023-07-18T03:39:00.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.004+0000 7f835939c700 1 -- 172.21.15.49:0/544012747 learned_addr learned my addr 172.21.15.49:0/544012747 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:00.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.004+0000 7f8358b9b700 1 --2- 172.21.15.49:0/544012747 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835415c950 0x7f83540b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:00.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.004+0000 7f8353fff700 1 --2- 172.21.15.49:0/544012747 >> v2:172.21.15.107:3300/0 conn(0x7f83540b1e00 0x7f83540d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:00.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.005+0000 7f835939c700 1 -- 172.21.15.49:0/544012747 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835415c950 msgr2=0x7f83540b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:00.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.005+0000 7f835939c700 1 --2- 172.21.15.49:0/544012747 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835415c950 0x7f83540b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:00.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.005+0000 7f835939c700 1 -- 172.21.15.49:0/544012747 >> v2:172.21.15.107:3300/0 conn(0x7f83540b1e00 msgr2=0x7f83540d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:00.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.005+0000 7f835939c700 1 --2- 172.21.15.49:0/544012747 >> v2:172.21.15.107:3300/0 conn(0x7f83540b1e00 0x7f83540d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:00.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.005+0000 7f835939c700 1 -- 172.21.15.49:0/544012747 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83540b3910 con 0x7f83540b2350 2023-07-18T03:39:00.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.005+0000 7f8358b9b700 1 --2- 172.21.15.49:0/544012747 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835415c950 0x7f83540b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:00.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.005+0000 7f8353fff700 1 --2- 172.21.15.49:0/544012747 >> v2:172.21.15.107:3300/0 conn(0x7f83540b1e00 0x7f83540d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:00.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.005+0000 7f835939c700 1 --2- 172.21.15.49:0/544012747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83540b2350 0x7f83540d1050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f835408baa0 tx=0x7f834c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc9d1c1f8af4409 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:00.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.005+0000 7f83537fe700 1 -- 172.21.15.49:0/544012747 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c00e070 con 0x7f83540b2350 2023-07-18T03:39:00.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.005+0000 7f83537fe700 1 -- 172.21.15.49:0/544012747 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c00b9c0 con 0x7f83540b2350 2023-07-18T03:39:00.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.005+0000 7f83537fe700 1 -- 172.21.15.49:0/544012747 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c00bd70 con 0x7f83540b2350 2023-07-18T03:39:00.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.006+0000 7f835adff700 1 -- 172.21.15.49:0/544012747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83540b2350 msgr2=0x7f83540d1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:00.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.006+0000 7f835adff700 1 --2- 172.21.15.49:0/544012747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83540b2350 0x7f83540d1050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f835408baa0 tx=0x7f834c00b330 comp rx=0 tx=0).stop 2023-07-18T03:39:00.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.006+0000 7f835adff700 1 -- 172.21.15.49:0/544012747 shutdown_connections 2023-07-18T03:39:00.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.006+0000 7f835adff700 1 --2- 172.21.15.49:0/544012747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83540b2350 0x7f83540d1050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:00.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.006+0000 7f835adff700 1 --2- 172.21.15.49:0/544012747 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835415c950 0x7f83540b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:00.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.006+0000 7f835adff700 1 --2- 172.21.15.49:0/544012747 >> v2:172.21.15.107:3300/0 conn(0x7f83540b1e00 0x7f83540d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:00.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.006+0000 7f835adff700 1 -- 172.21.15.49:0/544012747 >> 172.21.15.49:0/544012747 conn(0x7f835409c1d0 msgr2=0x7f83540d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:00.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.006+0000 7f835adff700 1 -- 172.21.15.49:0/544012747 shutdown_connections 2023-07-18T03:39:00.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.006+0000 7f835adff700 1 -- 172.21.15.49:0/544012747 wait complete. 2023-07-18T03:39:00.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.009+0000 7f835adff700 1 Processor -- start 2023-07-18T03:39:00.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.009+0000 7f835adff700 1 -- start start 2023-07-18T03:39:00.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.010+0000 7f835adff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83540b1e00 0x7f83540cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:00.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f835adff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83540b2350 0x7f83540afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:00.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f835adff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835415c950 0x7f83540b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:00.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f835adff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83540b67b0 con 0x7f83540b2350 2023-07-18T03:39:00.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f835adff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83540b5e10 con 0x7f83540b1e00 2023-07-18T03:39:00.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f835adff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83540b60f0 con 0x7f835415c950 2023-07-18T03:39:00.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f8353fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83540b2350 0x7f83540afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:00.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f8358b9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83540b1e00 0x7f83540cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:00.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f8353fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83540b2350 0x7f83540afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60814/0 (socket says 172.21.15.49:60814) 2023-07-18T03:39:00.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f8353fff700 1 -- 172.21.15.49:0/2570781646 learned_addr learned my addr 172.21.15.49:0/2570781646 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:00.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f835939c700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835415c950 0x7f83540b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:00.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f8353fff700 1 -- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835415c950 msgr2=0x7f83540b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:00.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f8353fff700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835415c950 0x7f83540b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:00.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f8353fff700 1 -- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83540b1e00 msgr2=0x7f83540cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:00.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f8353fff700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83540b1e00 0x7f83540cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:00.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.011+0000 7f8353fff700 1 -- 172.21.15.49:0/2570781646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c009b30 con 0x7f83540b2350 2023-07-18T03:39:00.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.012+0000 7f8353fff700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83540b2350 0x7f83540afe20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f834000d800 tx=0x7f834000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:00.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.012+0000 7f83517fa700 1 -- 172.21.15.49:0/2570781646 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340004030 con 0x7f83540b2350 2023-07-18T03:39:00.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.012+0000 7f83517fa700 1 -- 172.21.15.49:0/2570781646 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8340004190 con 0x7f83540b2350 2023-07-18T03:39:00.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.012+0000 7f835adff700 1 -- 172.21.15.49:0/2570781646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83540b63d0 con 0x7f83540b2350 2023-07-18T03:39:00.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.012+0000 7f83517fa700 1 -- 172.21.15.49:0/2570781646 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340004520 con 0x7f83540b2350 2023-07-18T03:39:00.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.012+0000 7f835adff700 1 -- 172.21.15.49:0/2570781646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83540b7d80 con 0x7f83540b2350 2023-07-18T03:39:00.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.013+0000 7f835adff700 1 -- 172.21.15.49:0/2570781646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83540b6560 con 0x7f83540b2350 2023-07-18T03:39:00.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.014+0000 7f83517fa700 1 -- 172.21.15.49:0/2570781646 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83400036a0 con 0x7f83540b2350 2023-07-18T03:39:00.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.015+0000 7f83517fa700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f833806ace0 0x7f833806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:00.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.015+0000 7f83517fa700 1 -- 172.21.15.49:0/2570781646 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8340088d20 con 0x7f83540b2350 2023-07-18T03:39:00.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.016+0000 7f8358b9b700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f833806ace0 0x7f833806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:00.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.017+0000 7f8358b9b700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f833806ace0 0x7f833806d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f834800c960 tx=0x7f8348008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:00.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.020+0000 7f83517fa700 1 -- 172.21.15.49:0/2570781646 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8340053de0 con 0x7f83540b2350 2023-07-18T03:39:00.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.273+0000 7f835adff700 1 -- 172.21.15.49:0/2570781646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f835404e280 con 0x7f83540b2350 2023-07-18T03:39:00.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.274+0000 7f83517fa700 1 -- 172.21.15.49:0/2570781646 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8340019020 con 0x7f83540b2350 2023-07-18T03:39:00.270 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-18T03:39:00.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.276+0000 7f835adff700 1 -- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f833806ace0 msgr2=0x7f833806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:00.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.276+0000 7f835adff700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f833806ace0 0x7f833806d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f834800c960 tx=0x7f8348008040 comp rx=0 tx=0).stop 2023-07-18T03:39:00.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.276+0000 7f835adff700 1 -- 172.21.15.49:0/2570781646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83540b2350 msgr2=0x7f83540afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:00.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.276+0000 7f835adff700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83540b2350 0x7f83540afe20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f834000d800 tx=0x7f834000db10 comp rx=0 tx=0).stop 2023-07-18T03:39:00.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.276+0000 7f835adff700 1 -- 172.21.15.49:0/2570781646 shutdown_connections 2023-07-18T03:39:00.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.276+0000 7f835adff700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835415c950 0x7f83540b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:00.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.276+0000 7f835adff700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83540b1e00 0x7f83540cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:00.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.276+0000 7f835adff700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f833806ace0 0x7f833806d190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:00.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.276+0000 7f835adff700 1 --2- 172.21.15.49:0/2570781646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83540b2350 0x7f83540afe20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:00.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.276+0000 7f835adff700 1 -- 172.21.15.49:0/2570781646 >> 172.21.15.49:0/2570781646 conn(0x7f835409c1d0 msgr2=0x7f83540cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:00.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.276+0000 7f835adff700 1 -- 172.21.15.49:0/2570781646 shutdown_connections 2023-07-18T03:39:00.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:00.276+0000 7f835adff700 1 -- 172.21.15.49:0/2570781646 wait complete. 2023-07-18T03:39:00.285 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:39:00.290 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:39:00.291 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-18T03:39:00.291 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-18T03:39:00.292 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-07-18T03:39:00.292 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-07-18T03:39:00.292 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-18T03:39:00.292 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-18T03:39:00.292 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-18T03:39:00.293 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-18T03:39:00.293 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-07-18T03:39:00.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:00.293 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-07-18T03:39:00.375 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:39:00.376 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:39:00.376 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:39:00.376 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:00.377 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-07-18T03:39:00.377 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:00.377 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:39:00.377 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:00.378 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-07-18T03:39:00.378 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:39:00.378 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:00.378 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-07-18T03:39:00.379 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-18T03:39:00.379 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:00.379 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-07-18T03:39:00.380 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-18T03:39:00.380 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 41967sec preferred_lft 41967sec 2023-07-18T03:39:00.380 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-07-18T03:39:00.381 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:00.381 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:39:00.381 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:00.381 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-07-18T03:39:00.382 INFO:teuthology.orchestra.run.smithi049.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-18T03:39:00.382 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 16:94:d3:1e:3e:91 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:00.382 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-18T03:39:00.383 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:00.383 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::463:84ff:fe79:c51d/64 scope link 2023-07-18T03:39:00.383 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:00.384 INFO:teuthology.orchestra.run.smithi049.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-18T03:39:00.384 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 16:94:d3:1e:3e:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-18T03:39:00.384 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::1494:d3ff:fe1e:3e91/64 scope link 2023-07-18T03:39:00.385 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:00.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:00.387 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-18T03:39:00.505 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-07-18T03:39:00.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:00.509 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-07-18T03:39:00.558 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-07-18T03:39:00.559 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-18T03:39:00.559 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-18T03:39:00.559 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-18T03:39:00.560 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-18T03:39:00.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:00.561 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-07-18T03:39:00.561 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-07-18T03:39:00.562 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-18T03:39:00.562 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-18T03:39:00.563 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-18T03:39:00.563 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-07-18T03:39:00.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:00.739 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-07-18T03:39:00.784 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-18T03:39:00.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:00.785 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2023-07-18T03:39:00.871 INFO:teuthology.orchestra.run.smithi049.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-07-18T03:39:00.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:00.872 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-07-18T03:39:00.872 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-07-18T03:39:00.873 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-18T03:39:00.873 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-07-18T03:39:00.873 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-07-18T03:39:01.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:01.064 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-18T03:39:01.136 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:39:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:39:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:39:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-07-18T03:39:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:01.137 INFO:teuthology.orchestra.run.smithi049.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-18T03:39:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 5a:b6:23:fa:6e:bd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-18T03:39:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-18T03:39:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::58b6:23ff:fefa:6ebd/64 scope link 2023-07-18T03:39:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:01.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:01.140 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-18T03:39:01.211 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:39:01.211 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:39:01.212 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:39:01.212 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:01.212 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-07-18T03:39:01.212 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:01.212 INFO:teuthology.orchestra.run.smithi049.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-18T03:39:01.212 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 5a:b6:23:fa:6e:bd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-18T03:39:01.212 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-18T03:39:01.213 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:01.213 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::58b6:23ff:fefa:6ebd/64 scope link 2023-07-18T03:39:01.213 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:01.214 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-07-18T03:39:01.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:01.215 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-07-18T03:39:01.215 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-07-18T03:39:01.215 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-07-18T03:39:01.215 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-18T03:39:01.216 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-07-18T03:39:01.216 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-07-18T03:39:01.216 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-07-18T03:39:01.216 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-07-18T03:39:01.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:01.473 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-07-18T03:39:01.474 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-07-18T03:39:01.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.1 up 2023-07-18T03:39:01.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.1 master ceph-brx 2023-07-18T03:39:01.474 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-07-18T03:39:01.651 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:39:01.652 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-07-18T03:39:01.696 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-07-18T03:39:01.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:39:01.698 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-07-18T03:39:01.780 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-07-18T03:39:01.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:01.781 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:39:01.928 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:39:01.928 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:39:01.959 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:39:01.960 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:39:01.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2023-07-18T03:39:01.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with new device syntax: 2@e859a751-7613-40d7-973c-3e47a5e301cb.cephfs=/ 2023-07-18T03:39:01.961 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.49:3300/172.21.15.107:3300/172.21.15.107:3301" will pass to kernel 2023-07-18T03:39:01.966 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-18T03:39:01.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:01.967 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-07-18T03:39:02.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:02.040 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:39:02.128 INFO:teuthology.orchestra.run.smithi049.stdout:client4663 2023-07-18T03:39:02.129 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:39:02.130 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:02.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:02.178 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:39:02.305 INFO:teuthology.orchestra.run.smithi049.stdout:client4663 2023-07-18T03:39:02.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:02.306 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:39:02.393 INFO:teuthology.orchestra.run.smithi049.stdout:client4663 2023-07-18T03:39:02.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:02.394 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e859a751-7613-40d7-973c-3e47a5e301cb.client4663/status') 2023-07-18T03:39:02.503 INFO:teuthology.orchestra.run.smithi049.stdout:instance: client.4663 (3)172.21.15.49:0/2503398976 2023-07-18T03:39:02.504 INFO:teuthology.orchestra.run.smithi049.stdout:blocklisted: false 2023-07-18T03:39:02.504 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records in 2023-07-18T03:39:02.504 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records out 2023-07-18T03:39:02.504 INFO:teuthology.orchestra.run.smithi049.stderr:70 bytes copied, 5.6896e-05 s, 1.2 MB/s 2023-07-18T03:39:02.506 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-07-18T03:39:02.507 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-18T03:39:02.507 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-18T03:39:02.507 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-18T03:39:02.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.717+0000 7f21ac7ff700 1 Processor -- start 2023-07-18T03:39:02.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.717+0000 7f21ac7ff700 1 -- start start 2023-07-18T03:39:02.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.718+0000 7f21ac7ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f21a40b4510 0x7f21a40b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:02.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.718+0000 7f21ac7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f21a40b5d90 0x7f21a40cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:02.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21ac7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21a40d0b10 0x7f21a40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:02.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21ac7ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a40b48e0 con 0x7f21a40b4510 2023-07-18T03:39:02.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21ac7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a40d14d0 con 0x7f21a40b5d90 2023-07-18T03:39:02.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21ac7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a40b7f50 con 0x7f21a40d0b10 2023-07-18T03:39:02.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21aa59b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f21a40b4510 0x7f21a40b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:02.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21aa59b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f21a40b4510 0x7f21a40b6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60820/0 (socket says 172.21.15.49:60820) 2023-07-18T03:39:02.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21aa59b700 1 -- 172.21.15.49:0/3201050159 learned_addr learned my addr 172.21.15.49:0/3201050159 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:02.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21aad9c700 1 --2- 172.21.15.49:0/3201050159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21a40d0b10 0x7f21a40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:02.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21aa59b700 1 -- 172.21.15.49:0/3201050159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21a40d0b10 msgr2=0x7f21a40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:02.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21a9d9a700 1 --2- 172.21.15.49:0/3201050159 >> v2:172.21.15.107:3300/0 conn(0x7f21a40b5d90 0x7f21a40cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:02.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21aa59b700 1 --2- 172.21.15.49:0/3201050159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21a40d0b10 0x7f21a40d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:02.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21aa59b700 1 -- 172.21.15.49:0/3201050159 >> v2:172.21.15.107:3300/0 conn(0x7f21a40b5d90 msgr2=0x7f21a40cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:02.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21aa59b700 1 --2- 172.21.15.49:0/3201050159 >> v2:172.21.15.107:3300/0 conn(0x7f21a40b5d90 0x7f21a40cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:02.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21aa59b700 1 -- 172.21.15.49:0/3201050159 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a40b7340 con 0x7f21a40b4510 2023-07-18T03:39:02.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21aad9c700 1 --2- 172.21.15.49:0/3201050159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21a40d0b10 0x7f21a40d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:02.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.719+0000 7f21a9d9a700 1 --2- 172.21.15.49:0/3201050159 >> v2:172.21.15.107:3300/0 conn(0x7f21a40b5d90 0x7f21a40cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:02.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.720+0000 7f21aa59b700 1 --2- 172.21.15.49:0/3201050159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21a40b4510 0x7f21a40b6c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f21a40b5b30 tx=0x7f21a0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faad9c9a4d56276e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:02.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.720+0000 7f21a9599700 1 -- 172.21.15.49:0/3201050159 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0005e30 con 0x7f21a40b4510 2023-07-18T03:39:02.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.720+0000 7f21a9599700 1 -- 172.21.15.49:0/3201050159 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a0004030 con 0x7f21a40b4510 2023-07-18T03:39:02.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.720+0000 7f21a9599700 1 -- 172.21.15.49:0/3201050159 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a00043e0 con 0x7f21a40b4510 2023-07-18T03:39:02.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.720+0000 7f21ac7ff700 1 -- 172.21.15.49:0/3201050159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21a40b4510 msgr2=0x7f21a40b6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:02.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.720+0000 7f21ac7ff700 1 --2- 172.21.15.49:0/3201050159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21a40b4510 0x7f21a40b6c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f21a40b5b30 tx=0x7f21a0009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:02.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.721+0000 7f21ac7ff700 1 -- 172.21.15.49:0/3201050159 shutdown_connections 2023-07-18T03:39:02.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.721+0000 7f21ac7ff700 1 --2- 172.21.15.49:0/3201050159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21a40d0b10 0x7f21a40d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:02.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.721+0000 7f21ac7ff700 1 --2- 172.21.15.49:0/3201050159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21a40b4510 0x7f21a40b6c80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:02.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.721+0000 7f21ac7ff700 1 --2- 172.21.15.49:0/3201050159 >> v2:172.21.15.107:3300/0 conn(0x7f21a40b5d90 0x7f21a40cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:02.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.721+0000 7f21ac7ff700 1 -- 172.21.15.49:0/3201050159 >> 172.21.15.49:0/3201050159 conn(0x7f21a409c1d0 msgr2=0x7f21a40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:02.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.721+0000 7f21ac7ff700 1 -- 172.21.15.49:0/3201050159 shutdown_connections 2023-07-18T03:39:02.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.721+0000 7f21ac7ff700 1 -- 172.21.15.49:0/3201050159 wait complete. 2023-07-18T03:39:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.722+0000 7f21ac7ff700 1 Processor -- start 2023-07-18T03:39:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.723+0000 7f21ac7ff700 1 -- start start 2023-07-18T03:39:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21ac7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21a40b4510 0x7f21a41513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21ac7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21a40b5d90 0x7f21a4152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21ac7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21a40d0b10 0x7f21a4157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:02.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21ac7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a41f3b00 con 0x7f21a40b4510 2023-07-18T03:39:02.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21ac7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4153960 con 0x7f21a40b5d90 2023-07-18T03:39:02.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21ac7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a41f34b0 con 0x7f21a40d0b10 2023-07-18T03:39:02.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21aa59b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21a40b4510 0x7f21a41513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21aa59b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21a40b4510 0x7f21a41513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60832/0 (socket says 172.21.15.49:60832) 2023-07-18T03:39:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21aa59b700 1 -- 172.21.15.49:0/472500975 learned_addr learned my addr 172.21.15.49:0/472500975 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21aad9c700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21a40d0b10 0x7f21a4157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21a9d9a700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21a40b5d90 0x7f21a4152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21aa59b700 1 -- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21a40d0b10 msgr2=0x7f21a4157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21aa59b700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21a40d0b10 0x7f21a4157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21aa59b700 1 -- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21a40b5d90 msgr2=0x7f21a4152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21aa59b700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21a40b5d90 0x7f21a4152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21aa59b700 1 -- 172.21.15.49:0/472500975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a0009500 con 0x7f21a40b4510 2023-07-18T03:39:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.724+0000 7f21aad9c700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21a40d0b10 0x7f21a4157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.725+0000 7f21aa59b700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21a40b4510 0x7f21a41513c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f21a0000c00 tx=0x7f21a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.725+0000 7f219affd700 1 -- 172.21.15.49:0/472500975 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0003e70 con 0x7f21a40b4510 2023-07-18T03:39:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.725+0000 7f219affd700 1 -- 172.21.15.49:0/472500975 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a0011420 con 0x7f21a40b4510 2023-07-18T03:39:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.725+0000 7f21ac7ff700 1 -- 172.21.15.49:0/472500975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a41f3790 con 0x7f21a40b4510 2023-07-18T03:39:02.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.725+0000 7f219affd700 1 -- 172.21.15.49:0/472500975 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a00117f0 con 0x7f21a40b4510 2023-07-18T03:39:02.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.725+0000 7f21ac7ff700 1 -- 172.21.15.49:0/472500975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a41f31b0 con 0x7f21a40b4510 2023-07-18T03:39:02.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.726+0000 7f21ac7ff700 1 -- 172.21.15.49:0/472500975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a404e280 con 0x7f21a40b4510 2023-07-18T03:39:02.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.727+0000 7f219affd700 1 -- 172.21.15.49:0/472500975 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f21a0011950 con 0x7f21a40b4510 2023-07-18T03:39:02.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.728+0000 7f219affd700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f218806ace0 0x7f218806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:02.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.728+0000 7f219affd700 1 -- 172.21.15.49:0/472500975 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f21a008a7f0 con 0x7f21a40b4510 2023-07-18T03:39:02.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.728+0000 7f21a9d9a700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f218806ace0 0x7f218806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:02.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.729+0000 7f21a9d9a700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f218806ace0 0x7f218806d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f21a41548f0 tx=0x7f2194008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:02.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.732+0000 7f219affd700 1 -- 172.21.15.49:0/472500975 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f21a0055930 con 0x7f21a40b4510 2023-07-18T03:39:02.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.986+0000 7f21ac7ff700 1 -- 172.21.15.49:0/472500975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f21a41f3920 con 0x7f21a40b4510 2023-07-18T03:39:02.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.986+0000 7f219affd700 1 -- 172.21.15.49:0/472500975 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f21a0058f50 con 0x7f21a40b4510 2023-07-18T03:39:02.982 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-18T03:39:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.988+0000 7f21ac7ff700 1 -- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f218806ace0 msgr2=0x7f218806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.988+0000 7f21ac7ff700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f218806ace0 0x7f218806d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f21a41548f0 tx=0x7f2194008040 comp rx=0 tx=0).stop 2023-07-18T03:39:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.988+0000 7f21ac7ff700 1 -- 172.21.15.49:0/472500975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21a40b4510 msgr2=0x7f21a41513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.988+0000 7f21ac7ff700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21a40b4510 0x7f21a41513c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f21a0000c00 tx=0x7f21a0004750 comp rx=0 tx=0).stop 2023-07-18T03:39:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.988+0000 7f21ac7ff700 1 -- 172.21.15.49:0/472500975 shutdown_connections 2023-07-18T03:39:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.988+0000 7f21ac7ff700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21a40d0b10 0x7f21a4157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.988+0000 7f21ac7ff700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f218806ace0 0x7f218806d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.988+0000 7f21ac7ff700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21a40b4510 0x7f21a41513c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.988+0000 7f21ac7ff700 1 --2- 172.21.15.49:0/472500975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21a40b5d90 0x7f21a4152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.988+0000 7f21ac7ff700 1 -- 172.21.15.49:0/472500975 >> 172.21.15.49:0/472500975 conn(0x7f21a409c1d0 msgr2=0x7f21a415b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.989+0000 7f21ac7ff700 1 -- 172.21.15.49:0/472500975 shutdown_connections 2023-07-18T03:39:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:02.989+0000 7f21ac7ff700 1 -- 172.21.15.49:0/472500975 wait complete. 2023-07-18T03:39:03.003 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-18T03:39:03.003 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-18T03:39:03.003 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2023-07-18T03:39:03.003 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-18T03:39:03.004 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-18T03:39:03.004 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-18T03:39:03.004 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-18T03:39:03.004 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-18T03:39:03.004 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-18T03:39:03.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:03.005 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2023-07-18T03:39:03.054 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:39:03.054 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:39:03.054 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:39:03.054 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:03.054 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2023-07-18T03:39:03.055 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:03.055 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:39:03.055 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:03.055 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2023-07-18T03:39:03.055 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:39:03.055 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:03.055 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2023-07-18T03:39:03.056 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:39:03.056 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:03.056 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-18T03:39:03.056 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:03.056 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-18T03:39:03.056 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 41950sec preferred_lft 41950sec 2023-07-18T03:39:03.056 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::9059:9611:e3b2:9f28/64 scope link noprefixroute 2023-07-18T03:39:03.057 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:03.057 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-18T03:39:03.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:03.058 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-18T03:39:03.058 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-18T03:39:03.058 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2023-07-18T03:39:03.059 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2023-07-18T03:39:03.059 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2023-07-18T03:39:03.059 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-18T03:39:03.059 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-18T03:39:03.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:03.490 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-18T03:39:03.559 INFO:teuthology.orchestra.run.smithi107.stdout:1 2023-07-18T03:39:03.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:03.562 DEBUG:teuthology.orchestra.run.smithi107:> route 2023-07-18T03:39:03.618 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2023-07-18T03:39:03.618 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-18T03:39:03.618 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-18T03:39:03.619 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-18T03:39:03.619 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-18T03:39:03.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:03.620 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-18T03:39:03.620 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-18T03:39:03.621 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-18T03:39:03.621 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-18T03:39:03.621 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-18T03:39:03.621 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-18T03:39:04.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:04.334 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-18T03:39:04.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:04.379 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2023-07-18T03:39:04.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:04.463 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-18T03:39:04.464 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-18T03:39:04.464 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-18T03:39:04.464 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-07-18T03:39:04.464 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-18T03:39:04.645 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-07-18T03:39:04.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:04.646 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-18T03:39:04.646 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-18T03:39:04.647 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-07-18T03:39:04.647 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-18T03:39:04.647 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-07-18T03:39:04.648 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-07-18T03:39:04.648 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-07-18T03:39:04.648 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-18T03:39:04.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:04.923 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-18T03:39:04.924 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-18T03:39:04.924 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2023-07-18T03:39:04.924 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2023-07-18T03:39:04.925 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-18T03:39:05.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:39:05.097 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-18T03:39:05.141 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-18T03:39:05.141 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:39:05.142 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-18T03:39:05.223 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-07-18T03:39:05.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:05.224 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:39:05.402 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:39:05.402 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:39:05.858 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:39:05.858 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:39:05.858 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2023-07-18T03:39:05.858 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with new device syntax: 1@e859a751-7613-40d7-973c-3e47a5e301cb.cephfs=/ 2023-07-18T03:39:05.858 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.49:3300/172.21.15.107:3300/172.21.15.107:3301" will pass to kernel 2023-07-18T03:39:05.863 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-18T03:39:05.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:05.864 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-18T03:39:05.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:05.933 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:39:06.025 INFO:teuthology.orchestra.run.smithi107.stdout:client4713 2023-07-18T03:39:06.026 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:39:06.026 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:06.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:06.070 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:39:06.195 INFO:teuthology.orchestra.run.smithi107.stdout:client4713 2023-07-18T03:39:06.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:06.196 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:39:06.280 INFO:teuthology.orchestra.run.smithi107.stdout:client4713 2023-07-18T03:39:06.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:06.281 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e859a751-7613-40d7-973c-3e47a5e301cb.client4713/status') 2023-07-18T03:39:06.389 INFO:teuthology.orchestra.run.smithi107.stdout:instance: client.4713 (3)172.21.15.107:0/3096958099 2023-07-18T03:39:06.390 INFO:teuthology.orchestra.run.smithi107.stdout:blocklisted: false 2023-07-18T03:39:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records in 2023-07-18T03:39:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records out 2023-07-18T03:39:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:71 bytes copied, 5.3224e-05 s, 1.3 MB/s 2023-07-18T03:39:06.392 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-07-18T03:39:06.392 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-18T03:39:06.392 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-18T03:39:06.393 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-18T03:39:06.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.613+0000 7f44d01ff700 1 Processor -- start 2023-07-18T03:39:06.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.613+0000 7f44d01ff700 1 -- start start 2023-07-18T03:39:06.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.614+0000 7f44d01ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44c815c950 0x7f44c80b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:06.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44d01ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44c80b1e00 0x7f44c80d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:06.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44d01ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f44c80b2350 0x7f44c80d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:06.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44d01ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c80d2ce0 con 0x7f44c80b1e00 2023-07-18T03:39:06.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44d01ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c80b3350 con 0x7f44c80b2350 2023-07-18T03:39:06.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44d01ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c80b3630 con 0x7f44c815c950 2023-07-18T03:39:06.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44cd79a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44c80b1e00 0x7f44c80d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:06.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44cd79a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44c80b1e00 0x7f44c80d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60838/0 (socket says 172.21.15.49:60838) 2023-07-18T03:39:06.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44cd79a700 1 -- 172.21.15.49:0/3448128064 learned_addr learned my addr 172.21.15.49:0/3448128064 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:06.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44ce79c700 1 --2- 172.21.15.49:0/3448128064 >> v2:172.21.15.107:3300/0 conn(0x7f44c80b2350 0x7f44c80d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:06.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44cdf9b700 1 --2- 172.21.15.49:0/3448128064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44c815c950 0x7f44c80b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:06.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44cd79a700 1 -- 172.21.15.49:0/3448128064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44c815c950 msgr2=0x7f44c80b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:06.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44cd79a700 1 --2- 172.21.15.49:0/3448128064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44c815c950 0x7f44c80b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44cd79a700 1 -- 172.21.15.49:0/3448128064 >> v2:172.21.15.107:3300/0 conn(0x7f44c80b2350 msgr2=0x7f44c80d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:06.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44cd79a700 1 --2- 172.21.15.49:0/3448128064 >> v2:172.21.15.107:3300/0 conn(0x7f44c80b2350 0x7f44c80d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.615+0000 7f44cd79a700 1 -- 172.21.15.49:0/3448128064 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c80b3910 con 0x7f44c80b1e00 2023-07-18T03:39:06.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.616+0000 7f44cd79a700 1 --2- 172.21.15.49:0/3448128064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c80b1e00 0x7f44c80d0b10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f44c80b3280 tx=0x7f44b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293db8a40153f902 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:06.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.616+0000 7f44ccf99700 1 -- 172.21.15.49:0/3448128064 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b800b870 con 0x7f44c80b1e00 2023-07-18T03:39:06.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.616+0000 7f44ccf99700 1 -- 172.21.15.49:0/3448128064 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b800b9d0 con 0x7f44c80b1e00 2023-07-18T03:39:06.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.616+0000 7f44ccf99700 1 -- 172.21.15.49:0/3448128064 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b800bd60 con 0x7f44c80b1e00 2023-07-18T03:39:06.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.616+0000 7f44d01ff700 1 -- 172.21.15.49:0/3448128064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c80b1e00 msgr2=0x7f44c80d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:06.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.616+0000 7f44d01ff700 1 --2- 172.21.15.49:0/3448128064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c80b1e00 0x7f44c80d0b10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f44c80b3280 tx=0x7f44b8009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.617+0000 7f44d01ff700 1 -- 172.21.15.49:0/3448128064 shutdown_connections 2023-07-18T03:39:06.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.617+0000 7f44d01ff700 1 --2- 172.21.15.49:0/3448128064 >> v2:172.21.15.107:3300/0 conn(0x7f44c80b2350 0x7f44c80d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.617+0000 7f44d01ff700 1 --2- 172.21.15.49:0/3448128064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44c815c950 0x7f44c80b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.617+0000 7f44d01ff700 1 --2- 172.21.15.49:0/3448128064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c80b1e00 0x7f44c80d0b10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.617+0000 7f44d01ff700 1 -- 172.21.15.49:0/3448128064 >> 172.21.15.49:0/3448128064 conn(0x7f44c809c1d0 msgr2=0x7f44c80d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:06.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.617+0000 7f44d01ff700 1 -- 172.21.15.49:0/3448128064 shutdown_connections 2023-07-18T03:39:06.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.617+0000 7f44d01ff700 1 -- 172.21.15.49:0/3448128064 wait complete. 2023-07-18T03:39:06.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.618+0000 7f44d01ff700 1 Processor -- start 2023-07-18T03:39:06.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.619+0000 7f44d01ff700 1 -- start start 2023-07-18T03:39:06.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.619+0000 7f44d01ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44c80b1e00 0x7f44c80b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:06.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.619+0000 7f44d01ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44c80b2350 0x7f44c81524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:06.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44d01ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c815c950 0x7f44c80b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:06.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44d01ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c80c6e30 con 0x7f44c815c950 2023-07-18T03:39:06.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44d01ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c80c99d0 con 0x7f44c80b2350 2023-07-18T03:39:06.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44d01ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c80b62b0 con 0x7f44c80b1e00 2023-07-18T03:39:06.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44ce79c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c815c950 0x7f44c80b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:06.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44ce79c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c815c950 0x7f44c80b7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60842/0 (socket says 172.21.15.49:60842) 2023-07-18T03:39:06.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44ce79c700 1 -- 172.21.15.49:0/1553948324 learned_addr learned my addr 172.21.15.49:0/1553948324 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:06.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44cd79a700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44c80b2350 0x7f44c81524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:06.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44cdf9b700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44c80b1e00 0x7f44c80b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:06.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44ce79c700 1 -- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44c80b1e00 msgr2=0x7f44c80b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:06.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44ce79c700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44c80b1e00 0x7f44c80b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44ce79c700 1 -- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44c80b2350 msgr2=0x7f44c81524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:06.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44ce79c700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44c80b2350 0x7f44c81524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44ce79c700 1 -- 172.21.15.49:0/1553948324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b8009550 con 0x7f44c815c950 2023-07-18T03:39:06.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.620+0000 7f44cd79a700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44c80b2350 0x7f44c81524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:06.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.621+0000 7f44ce79c700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c815c950 0x7f44c80b7c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f44c0007e90 tx=0x7f44c000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:06.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.621+0000 7f44be7fc700 1 -- 172.21.15.49:0/1553948324 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c000eea0 con 0x7f44c815c950 2023-07-18T03:39:06.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.621+0000 7f44be7fc700 1 -- 172.21.15.49:0/1553948324 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c0014420 con 0x7f44c815c950 2023-07-18T03:39:06.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.621+0000 7f44be7fc700 1 -- 172.21.15.49:0/1553948324 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c00147f0 con 0x7f44c815c950 2023-07-18T03:39:06.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.621+0000 7f44d01ff700 1 -- 172.21.15.49:0/1553948324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c80c8dc0 con 0x7f44c815c950 2023-07-18T03:39:06.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.621+0000 7f44d01ff700 1 -- 172.21.15.49:0/1553948324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c80c9380 con 0x7f44c815c950 2023-07-18T03:39:06.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.622+0000 7f44d01ff700 1 -- 172.21.15.49:0/1553948324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c804e280 con 0x7f44c815c950 2023-07-18T03:39:06.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.623+0000 7f44be7fc700 1 -- 172.21.15.49:0/1553948324 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f44c0014950 con 0x7f44c815c950 2023-07-18T03:39:06.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.624+0000 7f44be7fc700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f44ac06ace0 0x7f44ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:06.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.624+0000 7f44be7fc700 1 -- 172.21.15.49:0/1553948324 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f44c0089cf0 con 0x7f44c815c950 2023-07-18T03:39:06.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.624+0000 7f44cdf9b700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f44ac06ace0 0x7f44ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:06.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.625+0000 7f44cdf9b700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f44ac06ace0 0x7f44ac06d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f44c4006fd0 tx=0x7f44c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:06.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.629+0000 7f44be7fc700 1 -- 172.21.15.49:0/1553948324 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f44c008e070 con 0x7f44c815c950 2023-07-18T03:39:06.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.901+0000 7f44d01ff700 1 -- 172.21.15.49:0/1553948324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f44c80b1be0 con 0x7f44c815c950 2023-07-18T03:39:06.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.901+0000 7f44be7fc700 1 -- 172.21.15.49:0/1553948324 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f44c0054d10 con 0x7f44c815c950 2023-07-18T03:39:06.897 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-18T03:39:06.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.903+0000 7f44d01ff700 1 -- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f44ac06ace0 msgr2=0x7f44ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:06.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.903+0000 7f44d01ff700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f44ac06ace0 0x7f44ac06d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f44c4006fd0 tx=0x7f44c4008040 comp rx=0 tx=0).stop 2023-07-18T03:39:06.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.903+0000 7f44d01ff700 1 -- 172.21.15.49:0/1553948324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c815c950 msgr2=0x7f44c80b7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:06.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.903+0000 7f44d01ff700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c815c950 0x7f44c80b7c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f44c0007e90 tx=0x7f44c000e3c0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.904+0000 7f44d01ff700 1 -- 172.21.15.49:0/1553948324 shutdown_connections 2023-07-18T03:39:06.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.904+0000 7f44d01ff700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f44ac06ace0 0x7f44ac06d190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.904+0000 7f44d01ff700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44c815c950 0x7f44c80b7c70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.904+0000 7f44d01ff700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44c80b1e00 0x7f44c80b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.904+0000 7f44d01ff700 1 --2- 172.21.15.49:0/1553948324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44c80b2350 0x7f44c81524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:06.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.904+0000 7f44d01ff700 1 -- 172.21.15.49:0/1553948324 >> 172.21.15.49:0/1553948324 conn(0x7f44c809c1d0 msgr2=0x7f44c80b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:06.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.904+0000 7f44d01ff700 1 -- 172.21.15.49:0/1553948324 shutdown_connections 2023-07-18T03:39:06.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:06.904+0000 7f44d01ff700 1 -- 172.21.15.49:0/1553948324 wait complete. 2023-07-18T03:39:06.917 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-18T03:39:06.918 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-18T03:39:06.918 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2023-07-18T03:39:06.918 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-07-18T03:39:06.918 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-18T03:39:06.918 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-18T03:39:06.919 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-18T03:39:06.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-18T03:39:06.919 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-07-18T03:39:06.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:06.920 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2023-07-18T03:39:06.965 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:39:06.965 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:39:06.965 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:39:06.965 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:06.966 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2023-07-18T03:39:06.966 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:39:06.966 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:06.966 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2023-07-18T03:39:06.966 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:39:06.966 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:06.966 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2023-07-18T03:39:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:39:06.967 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:06.967 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-18T03:39:06.967 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:06.967 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-18T03:39:06.967 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 41946sec preferred_lft 41946sec 2023-07-18T03:39:06.967 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::9059:9611:e3b2:9f28/64 scope link noprefixroute 2023-07-18T03:39:06.968 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:06.968 INFO:teuthology.orchestra.run.smithi107.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-18T03:39:06.968 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether d2:89:6b:8e:5e:af brd ff:ff:ff:ff:ff:ff 2023-07-18T03:39:06.968 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-18T03:39:06.968 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:06.968 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::60b5:e0ff:fe5c:981b/64 scope link 2023-07-18T03:39:06.968 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:06.969 INFO:teuthology.orchestra.run.smithi107.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-18T03:39:06.969 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether d2:89:6b:8e:5e:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-18T03:39:06.969 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::d089:6bff:fe8e:5eaf/64 scope link tentative 2023-07-18T03:39:06.969 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:06.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:06.970 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-18T03:39:07.077 INFO:teuthology.orchestra.run.smithi107.stdout:1 2023-07-18T03:39:07.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:07.081 DEBUG:teuthology.orchestra.run.smithi107:> route 2023-07-18T03:39:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2023-07-18T03:39:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-18T03:39:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-18T03:39:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-18T03:39:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-18T03:39:07.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:07.129 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-18T03:39:07.129 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-18T03:39:07.130 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-18T03:39:07.130 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-18T03:39:07.131 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-18T03:39:07.131 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-18T03:39:07.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:07.300 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-18T03:39:07.343 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-18T03:39:07.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:07.344 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2023-07-18T03:39:07.426 INFO:teuthology.orchestra.run.smithi107.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-07-18T03:39:07.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:07.427 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-18T03:39:07.428 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-18T03:39:07.428 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-18T03:39:07.428 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-07-18T03:39:07.428 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-18T03:39:07.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:07.609 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-18T03:39:07.677 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:39:07.677 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:39:07.677 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:39:07.678 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:07.678 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2023-07-18T03:39:07.678 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:07.678 INFO:teuthology.orchestra.run.smithi107.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-18T03:39:07.678 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 96:e9:26:de:a9:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-18T03:39:07.679 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-18T03:39:07.679 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:07.679 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::94e9:26ff:fede:a921/64 scope link 2023-07-18T03:39:07.679 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:07.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:07.680 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-18T03:39:07.751 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:39:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:39:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:39:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2023-07-18T03:39:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:07.752 INFO:teuthology.orchestra.run.smithi107.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-18T03:39:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 96:e9:26:de:a9:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-18T03:39:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-18T03:39:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::94e9:26ff:fede:a921/64 scope link 2023-07-18T03:39:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:39:07.754 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-07-18T03:39:07.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:07.755 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-18T03:39:07.755 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-18T03:39:07.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-07-18T03:39:07.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-18T03:39:07.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-07-18T03:39:07.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-07-18T03:39:07.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-07-18T03:39:07.757 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-18T03:39:08.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:08.003 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-18T03:39:08.003 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-18T03:39:08.004 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.1 up 2023-07-18T03:39:08.004 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.1 master ceph-brx 2023-07-18T03:39:08.005 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-18T03:39:08.177 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:39:08.177 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-07-18T03:39:08.222 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-07-18T03:39:08.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:39:08.223 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-07-18T03:39:08.305 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-07-18T03:39:08.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:08.306 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:39:08.448 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:39:08.448 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:39:08.478 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:39:08.478 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:39:08.478 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2023-07-18T03:39:08.478 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with new device syntax: 3@e859a751-7613-40d7-973c-3e47a5e301cb.cephfs=/ 2023-07-18T03:39:08.478 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.49:3300/172.21.15.107:3300/172.21.15.107:3301" will pass to kernel 2023-07-18T03:39:08.483 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-18T03:39:08.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:08.484 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-07-18T03:39:08.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:08.554 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:39:08.639 INFO:teuthology.orchestra.run.smithi107.stdout:client4675 2023-07-18T03:39:08.639 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:39:08.639 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:08.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:08.682 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:39:08.808 INFO:teuthology.orchestra.run.smithi107.stdout:client4675 2023-07-18T03:39:08.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:08.809 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:39:08.893 INFO:teuthology.orchestra.run.smithi107.stdout:client4675 2023-07-18T03:39:08.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:08.894 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e859a751-7613-40d7-973c-3e47a5e301cb.client4675/status') 2023-07-18T03:39:09.000 INFO:teuthology.orchestra.run.smithi107.stdout:instance: client.4675 (3)192.168.0.2:0/2888580235 2023-07-18T03:39:09.000 INFO:teuthology.orchestra.run.smithi107.stdout:blocklisted: false 2023-07-18T03:39:09.001 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records in 2023-07-18T03:39:09.001 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records out 2023-07-18T03:39:09.001 INFO:teuthology.orchestra.run.smithi107.stderr:69 bytes copied, 5.4307e-05 s, 1.3 MB/s 2023-07-18T03:39:09.003 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-07-18T03:39:09.021 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:09.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.244+0000 7fc2ac3ff700 1 Processor -- start 2023-07-18T03:39:09.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.244+0000 7fc2ac3ff700 1 -- start start 2023-07-18T03:39:09.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.245+0000 7fc2ac3ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2a415c950 0x7fc2a40b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:09.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.245+0000 7fc2ac3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc2a40d1220 0x7fc2a40afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:09.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.245+0000 7fc2ac3ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a40d2ce0 con 0x7fc2a40b1e00 2023-07-18T03:39:09.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.245+0000 7fc2ac3ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a40b3350 con 0x7fc2a415c950 2023-07-18T03:39:09.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.245+0000 7fc2ac3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a40b3630 con 0x7fc2a40d1220 2023-07-18T03:39:09.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.246+0000 7fc2aa19b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2a415c950 0x7fc2a40b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:09.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.246+0000 7fc2aa19b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2a415c950 0x7fc2a40b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54482/0 (socket says 172.21.15.49:54482) 2023-07-18T03:39:09.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.246+0000 7fc2aa19b700 1 -- 172.21.15.49:0/316024067 learned_addr learned my addr 172.21.15.49:0/316024067 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:09.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.246+0000 7fc2aa99c700 1 --2- 172.21.15.49:0/316024067 >> v2:172.21.15.107:3300/0 conn(0x7fc2a40d1220 0x7fc2a40afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:09.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.246+0000 7fc2aa19b700 1 -- 172.21.15.49:0/316024067 >> v2:172.21.15.107:3300/0 conn(0x7fc2a40d1220 msgr2=0x7fc2a40afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:09.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.246+0000 7fc2aa19b700 1 --2- 172.21.15.49:0/316024067 >> v2:172.21.15.107:3300/0 conn(0x7fc2a40d1220 0x7fc2a40afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.246+0000 7fc2aa19b700 1 -- 172.21.15.49:0/316024067 >> v1:172.21.15.49:6789/0 conn(0x7fc2a40b1e00 legacy=0x7fc2a40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.246+0000 7fc2aa19b700 1 -- 172.21.15.49:0/316024067 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a40b3910 con 0x7fc2a415c950 2023-07-18T03:39:09.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.246+0000 7fc2a9199700 1 -- 172.21.15.49:0/316024067 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745481717 0 0) 0x7fc2a40d2ce0 con 0x7fc2a40b1e00 2023-07-18T03:39:09.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.246+0000 7fc2aa19b700 1 --2- 172.21.15.49:0/316024067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a415c950 0x7fc2a40b43f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2a409c8e0 tx=0x7fc2a0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12eb1fd649424c90 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:09.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.247+0000 7fc2a9199700 1 -- 172.21.15.49:0/316024067 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0005e30 con 0x7fc2a415c950 2023-07-18T03:39:09.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.247+0000 7fc2a9199700 1 -- 172.21.15.49:0/316024067 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a0004030 con 0x7fc2a415c950 2023-07-18T03:39:09.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.247+0000 7fc2a9199700 1 -- 172.21.15.49:0/316024067 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a00043e0 con 0x7fc2a415c950 2023-07-18T03:39:09.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.247+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/316024067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a415c950 msgr2=0x7fc2a40b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.247+0000 7fc2ac3ff700 1 --2- 172.21.15.49:0/316024067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a415c950 0x7fc2a40b43f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2a409c8e0 tx=0x7fc2a0009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.247+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/316024067 shutdown_connections 2023-07-18T03:39:09.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.247+0000 7fc2ac3ff700 1 --2- 172.21.15.49:0/316024067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a415c950 0x7fc2a40b43f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.247+0000 7fc2ac3ff700 1 --2- 172.21.15.49:0/316024067 >> v2:172.21.15.107:3300/0 conn(0x7fc2a40d1220 0x7fc2a40afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.247+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/316024067 >> 172.21.15.49:0/316024067 conn(0x7fc2a409c1d0 msgr2=0x7fc2a40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:09.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.247+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/316024067 shutdown_connections 2023-07-18T03:39:09.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.247+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/316024067 wait complete. 2023-07-18T03:39:09.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.249+0000 7fc2ac3ff700 1 Processor -- start 2023-07-18T03:39:09.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.249+0000 7fc2ac3ff700 1 -- start start 2023-07-18T03:39:09.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.250+0000 7fc2ac3ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a40b1e00 0x7fc2a4152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:09.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.250+0000 7fc2ac3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2a40d1220 0x7fc2a40cf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:09.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.250+0000 7fc2ac3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2a415c950 0x7fc2a40b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:09.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.250+0000 7fc2ac3ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40cf0c0 con 0x7fc2a40b1e00 2023-07-18T03:39:09.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.250+0000 7fc2ac3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40b8920 con 0x7fc2a40d1220 2023-07-18T03:39:09.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.250+0000 7fc2ac3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40c8c50 con 0x7fc2a415c950 2023-07-18T03:39:09.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.250+0000 7fc2aa19b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a40b1e00 0x7fc2a4152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:09.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.250+0000 7fc2aa19b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a40b1e00 0x7fc2a4152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54486/0 (socket says 172.21.15.49:54486) 2023-07-18T03:39:09.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.250+0000 7fc2aa19b700 1 -- 172.21.15.49:0/357495627 learned_addr learned my addr 172.21.15.49:0/357495627 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:09.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc2a999a700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2a40d1220 0x7fc2a40cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:09.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc2aa99c700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2a415c950 0x7fc2a40b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:09.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc2aa19b700 1 -- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2a415c950 msgr2=0x7fc2a40b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc2aa19b700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2a415c950 0x7fc2a40b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc2aa19b700 1 -- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2a40d1220 msgr2=0x7fc2a40cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc2aa19b700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2a40d1220 0x7fc2a40cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc2aa19b700 1 -- 172.21.15.49:0/357495627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a0009500 con 0x7fc2a40b1e00 2023-07-18T03:39:09.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc2a999a700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2a40d1220 0x7fc2a40cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:09.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc2aa19b700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a40b1e00 0x7fc2a4152460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0004000 tx=0x7fc2a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:09.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc29affd700 1 -- 172.21.15.49:0/357495627 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0003e10 con 0x7fc2a40b1e00 2023-07-18T03:39:09.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc29affd700 1 -- 172.21.15.49:0/357495627 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a001d430 con 0x7fc2a40b1e00 2023-07-18T03:39:09.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc29affd700 1 -- 172.21.15.49:0/357495627 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a001d7e0 con 0x7fc2a40b1e00 2023-07-18T03:39:09.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.251+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/357495627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a40c8f30 con 0x7fc2a40b1e00 2023-07-18T03:39:09.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.252+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/357495627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a41529a0 con 0x7fc2a40b1e00 2023-07-18T03:39:09.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.253+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/357495627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a404e280 con 0x7fc2a40b1e00 2023-07-18T03:39:09.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.254+0000 7fc29affd700 1 -- 172.21.15.49:0/357495627 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2a001d940 con 0x7fc2a40b1e00 2023-07-18T03:39:09.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.255+0000 7fc29affd700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc28806ace0 0x7fc28806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:09.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.255+0000 7fc29affd700 1 -- 172.21.15.49:0/357495627 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc2a008a8c0 con 0x7fc2a40b1e00 2023-07-18T03:39:09.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.255+0000 7fc2a999a700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc28806ace0 0x7fc28806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:09.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.256+0000 7fc2a999a700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc28806ace0 0x7fc28806d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40cd240 tx=0x7fc294009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:09.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.259+0000 7fc29affd700 1 -- 172.21.15.49:0/357495627 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc2a0055960 con 0x7fc2a40b1e00 2023-07-18T03:39:09.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.470+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/357495627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2a40c9450 con 0x7fc2a40b1e00 2023-07-18T03:39:09.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.472+0000 7fc29affd700 1 -- 172.21.15.49:0/357495627 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18185 (secure 0 0 0) 0x7fc2a0058f80 con 0x7fc2a40b1e00 2023-07-18T03:39:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:09.467 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:37.510006+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:09.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.474+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc28806ace0 msgr2=0x7fc28806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.474+0000 7fc2ac3ff700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc28806ace0 0x7fc28806d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40cd240 tx=0x7fc294009040 comp rx=0 tx=0).stop 2023-07-18T03:39:09.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.474+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/357495627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a40b1e00 msgr2=0x7fc2a4152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.474+0000 7fc2ac3ff700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a40b1e00 0x7fc2a4152460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0004000 tx=0x7fc2a0004750 comp rx=0 tx=0).stop 2023-07-18T03:39:09.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.474+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/357495627 shutdown_connections 2023-07-18T03:39:09.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.474+0000 7fc2ac3ff700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2a415c950 0x7fc2a40b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.474+0000 7fc2ac3ff700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc28806ace0 0x7fc28806d190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.474+0000 7fc2ac3ff700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2a40b1e00 0x7fc2a4152460 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.474+0000 7fc2ac3ff700 1 --2- 172.21.15.49:0/357495627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2a40d1220 0x7fc2a40cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.474+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/357495627 >> 172.21.15.49:0/357495627 conn(0x7fc2a409c1d0 msgr2=0x7fc2a40b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:09.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.474+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/357495627 shutdown_connections 2023-07-18T03:39:09.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.474+0000 7fc2ac3ff700 1 -- 172.21.15.49:0/357495627 wait complete. 2023-07-18T03:39:09.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:38:33.773236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:38:35.023692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:39:09.489 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:39:09.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.701+0000 7fa8309ff700 1 Processor -- start 2023-07-18T03:39:09.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.701+0000 7fa8309ff700 1 -- start start 2023-07-18T03:39:09.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.702+0000 7fa8309ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa82c0d0b10 0x7fa82c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:09.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.702+0000 7fa8309ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c0b8e30 con 0x7fa82c0b0e20 2023-07-18T03:39:09.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.702+0000 7fa8309ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c0afaa0 con 0x7fa82c0b8a60 2023-07-18T03:39:09.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.702+0000 7fa8309ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c0d14d0 con 0x7fa82c0d0b10 2023-07-18T03:39:09.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.703+0000 7fa82ad9d700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa82c0b0e20 0x7fa82c0b1260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42528/0 (socket says 172.21.15.49:42528) 2023-07-18T03:39:09.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.703+0000 7fa82ad9d700 1 -- 172.21.15.49:0/409825601 learned_addr learned my addr 172.21.15.49:0/409825601 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:09.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.703+0000 7fa823fff700 1 --2- 172.21.15.49:0/409825601 >> v2:172.21.15.107:3300/0 conn(0x7fa82c0d0b10 0x7fa82c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:09.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.703+0000 7fa829d9b700 1 -- 172.21.15.49:0/409825601 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638244628 0 0) 0x7fa82c0b8e30 con 0x7fa82c0b0e20 2023-07-18T03:39:09.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.703+0000 7fa829d9b700 1 -- 172.21.15.49:0/409825601 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8180035b0 con 0x7fa82c0b0e20 2023-07-18T03:39:09.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.703+0000 7fa829d9b700 1 -- 172.21.15.49:0/409825601 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524444874 0 0) 0x7fa82c0afaa0 con 0x7fa82c0b8a60 2023-07-18T03:39:09.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.703+0000 7fa829d9b700 1 -- 172.21.15.49:0/409825601 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa82c0b8e30 con 0x7fa82c0b8a60 2023-07-18T03:39:09.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.703+0000 7fa823fff700 1 -- 172.21.15.49:0/409825601 >> v1:172.21.15.107:6789/0 conn(0x7fa82c0b8a60 legacy=0x7fa82c0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.703+0000 7fa823fff700 1 -- 172.21.15.49:0/409825601 >> v1:172.21.15.49:6789/0 conn(0x7fa82c0b0e20 legacy=0x7fa82c0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.703+0000 7fa823fff700 1 -- 172.21.15.49:0/409825601 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c0afd80 con 0x7fa82c0d0b10 2023-07-18T03:39:09.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.704+0000 7fa823fff700 1 --2- 172.21.15.49:0/409825601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa82c0d0b10 0x7fa82c0d0f90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa814009840 tx=0x7fa814009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7961a1240bafc528 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:09.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.704+0000 7fa829d9b700 1 -- 172.21.15.49:0/409825601 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814011830 con 0x7fa82c0d0b10 2023-07-18T03:39:09.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.704+0000 7fa829d9b700 1 -- 172.21.15.49:0/409825601 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa814011990 con 0x7fa82c0d0b10 2023-07-18T03:39:09.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.704+0000 7fa829d9b700 1 -- 172.21.15.49:0/409825601 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814011d00 con 0x7fa82c0d0b10 2023-07-18T03:39:09.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.704+0000 7fa8309ff700 1 -- 172.21.15.49:0/409825601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa82c0d0b10 msgr2=0x7fa82c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.704+0000 7fa8309ff700 1 --2- 172.21.15.49:0/409825601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa82c0d0b10 0x7fa82c0d0f90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa814009840 tx=0x7fa814009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:09.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.705+0000 7fa8309ff700 1 -- 172.21.15.49:0/409825601 shutdown_connections 2023-07-18T03:39:09.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.705+0000 7fa8309ff700 1 --2- 172.21.15.49:0/409825601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa82c0d0b10 0x7fa82c0d0f90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.705+0000 7fa8309ff700 1 -- 172.21.15.49:0/409825601 >> 172.21.15.49:0/409825601 conn(0x7fa82c09c1b0 msgr2=0x7fa82c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:09.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.705+0000 7fa8309ff700 1 -- 172.21.15.49:0/409825601 shutdown_connections 2023-07-18T03:39:09.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.705+0000 7fa8309ff700 1 -- 172.21.15.49:0/409825601 wait complete. 2023-07-18T03:39:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.706+0000 7fa8309ff700 1 Processor -- start 2023-07-18T03:39:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.707+0000 7fa8309ff700 1 -- start start 2023-07-18T03:39:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.707+0000 7fa8309ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa82c0b0e20 0x7fa82c0cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.707+0000 7fa8309ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa82c0b8a60 0x7fa82c0cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.707+0000 7fa8309ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c0d0b10 0x7fa82c153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.707+0000 7fa8309ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0cb960 con 0x7fa82c0d0b10 2023-07-18T03:39:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.707+0000 7fa8309ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0cfa30 con 0x7fa82c0b8a60 2023-07-18T03:39:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.707+0000 7fa8309ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0cbc00 con 0x7fa82c0b0e20 2023-07-18T03:39:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.708+0000 7fa82ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c0d0b10 0x7fa82c153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.708+0000 7fa82ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c0d0b10 0x7fa82c153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54502/0 (socket says 172.21.15.49:54502) 2023-07-18T03:39:09.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.708+0000 7fa82ad9d700 1 -- 172.21.15.49:0/1618657638 learned_addr learned my addr 172.21.15.49:0/1618657638 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:09.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.708+0000 7fa823fff700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa82c0b8a60 0x7fa82c0cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:09.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.708+0000 7fa82a59c700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa82c0b0e20 0x7fa82c0cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:09.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.708+0000 7fa82ad9d700 1 -- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa82c0b0e20 msgr2=0x7fa82c0cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.708+0000 7fa82ad9d700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa82c0b0e20 0x7fa82c0cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.708+0000 7fa82ad9d700 1 -- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa82c0b8a60 msgr2=0x7fa82c0cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.708+0000 7fa82ad9d700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa82c0b8a60 0x7fa82c0cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.708+0000 7fa82ad9d700 1 -- 172.21.15.49:0/1618657638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140094a0 con 0x7fa82c0d0b10 2023-07-18T03:39:09.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.709+0000 7fa82ad9d700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c0d0b10 0x7fa82c153b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa82400cba0 tx=0x7fa82400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:09.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.709+0000 7fa822ffd700 1 -- 172.21.15.49:0/1618657638 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82400a980 con 0x7fa82c0d0b10 2023-07-18T03:39:09.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.709+0000 7fa822ffd700 1 -- 172.21.15.49:0/1618657638 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82400aae0 con 0x7fa82c0d0b10 2023-07-18T03:39:09.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.709+0000 7fa8309ff700 1 -- 172.21.15.49:0/1618657638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c0cbee0 con 0x7fa82c0d0b10 2023-07-18T03:39:09.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.709+0000 7fa822ffd700 1 -- 172.21.15.49:0/1618657638 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82400ae50 con 0x7fa82c0d0b10 2023-07-18T03:39:09.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.709+0000 7fa8309ff700 1 -- 172.21.15.49:0/1618657638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c0cc4a0 con 0x7fa82c0d0b10 2023-07-18T03:39:09.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.710+0000 7fa8309ff700 1 -- 172.21.15.49:0/1618657638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c074350 con 0x7fa82c0d0b10 2023-07-18T03:39:09.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.711+0000 7fa822ffd700 1 -- 172.21.15.49:0/1618657638 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa82401b3b0 con 0x7fa82c0d0b10 2023-07-18T03:39:09.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.712+0000 7fa822ffd700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa80c06ace0 0x7fa80c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:09.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.712+0000 7fa822ffd700 1 -- 172.21.15.49:0/1618657638 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa824088fb0 con 0x7fa82c0d0b10 2023-07-18T03:39:09.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.713+0000 7fa82a59c700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa80c06ace0 0x7fa80c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:09.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.713+0000 7fa82a59c700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa80c06ace0 0x7fa80c06d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa81c001220 tx=0x7fa81c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:09.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.716+0000 7fa822ffd700 1 -- 172.21.15.49:0/1618657638 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa824057b90 con 0x7fa82c0d0b10 2023-07-18T03:39:09.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.920+0000 7fa8309ff700 1 -- 172.21.15.49:0/1618657638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa82c0d23e0 con 0x7fa82c0d0b10 2023-07-18T03:39:09.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.921+0000 7fa822ffd700 1 -- 172.21.15.49:0/1618657638 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa824017020 con 0x7fa82c0d0b10 2023-07-18T03:39:09.917 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:39:09.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.923+0000 7fa8309ff700 1 -- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa80c06ace0 msgr2=0x7fa80c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.923+0000 7fa8309ff700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa80c06ace0 0x7fa80c06d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa81c001220 tx=0x7fa81c009040 comp rx=0 tx=0).stop 2023-07-18T03:39:09.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.923+0000 7fa8309ff700 1 -- 172.21.15.49:0/1618657638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c0d0b10 msgr2=0x7fa82c153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.923+0000 7fa8309ff700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c0d0b10 0x7fa82c153b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa82400cba0 tx=0x7fa82400ceb0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.923+0000 7fa8309ff700 1 -- 172.21.15.49:0/1618657638 shutdown_connections 2023-07-18T03:39:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.923+0000 7fa8309ff700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa80c06ace0 0x7fa80c06d190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.923+0000 7fa8309ff700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa82c0d0b10 0x7fa82c153b00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.923+0000 7fa8309ff700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa82c0b0e20 0x7fa82c0cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.923+0000 7fa8309ff700 1 --2- 172.21.15.49:0/1618657638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa82c0b8a60 0x7fa82c0cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.923+0000 7fa8309ff700 1 -- 172.21.15.49:0/1618657638 >> 172.21.15.49:0/1618657638 conn(0x7fa82c09c1b0 msgr2=0x7fa82c0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.923+0000 7fa8309ff700 1 -- 172.21.15.49:0/1618657638 shutdown_connections 2023-07-18T03:39:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:09.923+0000 7fa8309ff700 1 -- 172.21.15.49:0/1618657638 wait complete. 2023-07-18T03:39:09.937 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:39:10.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.150+0000 7fadab1ff700 1 Processor -- start 2023-07-18T03:39:10.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.150+0000 7fadab1ff700 1 -- start start 2023-07-18T03:39:10.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fadab1ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fada40d0b10 0x7fada40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:10.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fadab1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fada40b0e20 0x7fada40b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fadab1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada40b8e30 con 0x7fada40b8a60 2023-07-18T03:39:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fadab1ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada40d14d0 con 0x7fada40d0b10 2023-07-18T03:39:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fadab1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada40afaa0 con 0x7fada40b0e20 2023-07-18T03:39:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fada3fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fada40d0b10 0x7fada40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fada3fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fada40d0b10 0x7fada40d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54514/0 (socket says 172.21.15.49:54514) 2023-07-18T03:39:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fada3fff700 1 -- 172.21.15.49:0/3210862985 learned_addr learned my addr 172.21.15.49:0/3210862985 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:10.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fada979c700 1 --2- 172.21.15.49:0/3210862985 >> v2:172.21.15.107:3300/0 conn(0x7fada40b0e20 0x7fada40b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:10.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fada3fff700 1 -- 172.21.15.49:0/3210862985 >> v2:172.21.15.107:3300/0 conn(0x7fada40b0e20 msgr2=0x7fada40b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:10.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fada3fff700 1 --2- 172.21.15.49:0/3210862985 >> v2:172.21.15.107:3300/0 conn(0x7fada40b0e20 0x7fada40b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fada3fff700 1 -- 172.21.15.49:0/3210862985 >> v1:172.21.15.107:6789/0 conn(0x7fada40b8a60 legacy=0x7fada40b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:10.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.152+0000 7fada3fff700 1 -- 172.21.15.49:0/3210862985 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada40afd80 con 0x7fada40d0b10 2023-07-18T03:39:10.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.153+0000 7fada3fff700 1 --2- 172.21.15.49:0/3210862985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada40d0b10 0x7fada40d0f90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fada40674e0 tx=0x7fad90009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80854c455cc8ccc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:10.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.153+0000 7fada37fe700 1 -- 172.21.15.49:0/3210862985 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9000b870 con 0x7fada40d0b10 2023-07-18T03:39:10.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.153+0000 7fada37fe700 1 -- 172.21.15.49:0/3210862985 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9000b9d0 con 0x7fada40d0b10 2023-07-18T03:39:10.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.153+0000 7fada37fe700 1 -- 172.21.15.49:0/3210862985 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9000bd60 con 0x7fada40d0b10 2023-07-18T03:39:10.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.153+0000 7fadab1ff700 1 -- 172.21.15.49:0/3210862985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada40d0b10 msgr2=0x7fada40d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:10.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.153+0000 7fadab1ff700 1 --2- 172.21.15.49:0/3210862985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada40d0b10 0x7fada40d0f90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fada40674e0 tx=0x7fad90009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.154+0000 7fadab1ff700 1 -- 172.21.15.49:0/3210862985 shutdown_connections 2023-07-18T03:39:10.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.154+0000 7fadab1ff700 1 --2- 172.21.15.49:0/3210862985 >> v2:172.21.15.107:3300/0 conn(0x7fada40b0e20 0x7fada40b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.154+0000 7fadab1ff700 1 --2- 172.21.15.49:0/3210862985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada40d0b10 0x7fada40d0f90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.154+0000 7fadab1ff700 1 -- 172.21.15.49:0/3210862985 >> 172.21.15.49:0/3210862985 conn(0x7fada409c1b0 msgr2=0x7fada40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.154+0000 7fadab1ff700 1 -- 172.21.15.49:0/3210862985 shutdown_connections 2023-07-18T03:39:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.154+0000 7fadab1ff700 1 -- 172.21.15.49:0/3210862985 wait complete. 2023-07-18T03:39:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.155+0000 7fadab1ff700 1 Processor -- start 2023-07-18T03:39:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.156+0000 7fadab1ff700 1 -- start start 2023-07-18T03:39:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.156+0000 7fadab1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fada40b0e20 0x7fada415b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.156+0000 7fadab1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada40b8a60 0x7fada40cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fadab1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fada40d0b10 0x7fada40cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fadab1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada415eab0 con 0x7fada40b8a60 2023-07-18T03:39:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fadab1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada40c7200 con 0x7fada40d0b10 2023-07-18T03:39:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fadab1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada415b870 con 0x7fada40b0e20 2023-07-18T03:39:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada40b8a60 0x7fada40cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada40b8a60 0x7fada40cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54526/0 (socket says 172.21.15.49:54526) 2023-07-18T03:39:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada3fff700 1 -- 172.21.15.49:0/4280735064 learned_addr learned my addr 172.21.15.49:0/4280735064 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada979c700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fada40d0b10 0x7fada40cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada3fff700 1 -- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fada40b0e20 msgr2=0x7fada415b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada8f9b700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fada40b0e20 0x7fada415b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada3fff700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fada40b0e20 0x7fada415b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada3fff700 1 -- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fada40d0b10 msgr2=0x7fada40cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada3fff700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fada40d0b10 0x7fada40cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada3fff700 1 -- 172.21.15.49:0/4280735064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad90009550 con 0x7fada40b8a60 2023-07-18T03:39:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada979c700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fada40d0b10 0x7fada40cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada8f9b700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fada40b0e20 0x7fada415b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.157+0000 7fada3fff700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada40b8a60 0x7fada40cb850 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad90000c00 tx=0x7fad90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:10.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.158+0000 7fada17fa700 1 -- 172.21.15.49:0/4280735064 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90004520 con 0x7fada40b8a60 2023-07-18T03:39:10.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.158+0000 7fada17fa700 1 -- 172.21.15.49:0/4280735064 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9001e070 con 0x7fada40b8a60 2023-07-18T03:39:10.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.158+0000 7fada17fa700 1 -- 172.21.15.49:0/4280735064 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad900038a0 con 0x7fada40b8a60 2023-07-18T03:39:10.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.158+0000 7fadab1ff700 1 -- 172.21.15.49:0/4280735064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada40c68d0 con 0x7fada40b8a60 2023-07-18T03:39:10.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.158+0000 7fadab1ff700 1 -- 172.21.15.49:0/4280735064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada40c6e90 con 0x7fada40b8a60 2023-07-18T03:39:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.159+0000 7fadab1ff700 1 -- 172.21.15.49:0/4280735064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada40c7050 con 0x7fada40b8a60 2023-07-18T03:39:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.160+0000 7fada17fa700 1 -- 172.21.15.49:0/4280735064 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad90003a00 con 0x7fada40b8a60 2023-07-18T03:39:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.161+0000 7fada17fa700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad8806ace0 0x7fad8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.161+0000 7fada17fa700 1 -- 172.21.15.49:0/4280735064 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad90020070 con 0x7fada40b8a60 2023-07-18T03:39:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.161+0000 7fada8f9b700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad8806ace0 0x7fad8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:10.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.162+0000 7fada8f9b700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad8806ace0 0x7fad8806d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fad98000f60 tx=0x7fad98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:10.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.165+0000 7fada17fa700 1 -- 172.21.15.49:0/4280735064 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fad90055b30 con 0x7fada40b8a60 2023-07-18T03:39:10.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.372+0000 7fadab1ff700 1 -- 172.21.15.49:0/4280735064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fada40c7370 con 0x7fada40b8a60 2023-07-18T03:39:10.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.372+0000 7fada17fa700 1 -- 172.21.15.49:0/4280735064 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad900150a0 con 0x7fada40b8a60 2023-07-18T03:39:10.369 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:39:10.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.374+0000 7fadab1ff700 1 -- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad8806ace0 msgr2=0x7fad8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:10.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.374+0000 7fadab1ff700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad8806ace0 0x7fad8806d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fad98000f60 tx=0x7fad98009040 comp rx=0 tx=0).stop 2023-07-18T03:39:10.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.374+0000 7fadab1ff700 1 -- 172.21.15.49:0/4280735064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada40b8a60 msgr2=0x7fada40cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:10.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.374+0000 7fadab1ff700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada40b8a60 0x7fada40cb850 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad90000c00 tx=0x7fad90004750 comp rx=0 tx=0).stop 2023-07-18T03:39:10.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.374+0000 7fadab1ff700 1 -- 172.21.15.49:0/4280735064 shutdown_connections 2023-07-18T03:39:10.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.374+0000 7fadab1ff700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fada40d0b10 0x7fada40cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.374+0000 7fadab1ff700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fada40b0e20 0x7fada415b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.374+0000 7fadab1ff700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad8806ace0 0x7fad8806d190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.374+0000 7fadab1ff700 1 --2- 172.21.15.49:0/4280735064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada40b8a60 0x7fada40cb850 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.374+0000 7fadab1ff700 1 -- 172.21.15.49:0/4280735064 >> 172.21.15.49:0/4280735064 conn(0x7fada409c1b0 msgr2=0x7fada406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:10.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.375+0000 7fadab1ff700 1 -- 172.21.15.49:0/4280735064 shutdown_connections 2023-07-18T03:39:10.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.375+0000 7fadab1ff700 1 -- 172.21.15.49:0/4280735064 wait complete. 2023-07-18T03:39:10.387 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:39:10.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.600+0000 7f7d413ff700 1 Processor -- start 2023-07-18T03:39:10.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.600+0000 7f7d413ff700 1 -- start start 2023-07-18T03:39:10.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.601+0000 7f7d413ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7d3c0d0b10 0x7f7d3c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:10.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.601+0000 7f7d413ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d3c0b0e20 0x7f7d3c0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:10.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.601+0000 7f7d413ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3c0b8e30 con 0x7f7d3c0b8a60 2023-07-18T03:39:10.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.601+0000 7f7d413ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3c0d14d0 con 0x7f7d3c0d0b10 2023-07-18T03:39:10.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.601+0000 7f7d413ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0afaa0 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.602+0000 7f7d3affd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7d3c0b8a60 0x7f7d3c0b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42536/0 (socket says 172.21.15.49:42536) 2023-07-18T03:39:10.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.602+0000 7f7d3affd700 1 -- 172.21.15.49:0/1935636722 learned_addr learned my addr 172.21.15.49:0/1935636722 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:10.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.602+0000 7f7d3a7fc700 1 --2- 172.21.15.49:0/1935636722 >> v2:172.21.15.107:3300/0 conn(0x7f7d3c0d0b10 0x7f7d3c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:10.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.602+0000 7f7d3b7fe700 1 --2- 172.21.15.49:0/1935636722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d3c0b0e20 0x7f7d3c0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:10.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.602+0000 7f7d39ffb700 1 -- 172.21.15.49:0/1935636722 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177791723 0 0) 0x7f7d3c0b8e30 con 0x7f7d3c0b8a60 2023-07-18T03:39:10.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.602+0000 7f7d39ffb700 1 -- 172.21.15.49:0/1935636722 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d240035b0 con 0x7f7d3c0b8a60 2023-07-18T03:39:10.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.602+0000 7f7d3b7fe700 1 -- 172.21.15.49:0/1935636722 >> v2:172.21.15.107:3300/0 conn(0x7f7d3c0d0b10 msgr2=0x7f7d3c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:10.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.602+0000 7f7d3b7fe700 1 --2- 172.21.15.49:0/1935636722 >> v2:172.21.15.107:3300/0 conn(0x7f7d3c0d0b10 0x7f7d3c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.602+0000 7f7d3b7fe700 1 -- 172.21.15.49:0/1935636722 >> v1:172.21.15.49:6789/0 conn(0x7f7d3c0b8a60 legacy=0x7f7d3c0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.602+0000 7f7d3b7fe700 1 -- 172.21.15.49:0/1935636722 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c0afd80 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.603+0000 7f7d3b7fe700 1 --2- 172.21.15.49:0/1935636722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d3c0b0e20 0x7f7d3c0b12a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009d90 tx=0x7f7d3400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dae00a4d6a0c13e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.603+0000 7f7d39ffb700 1 -- 172.21.15.49:0/1935636722 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400e070 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.603+0000 7f7d39ffb700 1 -- 172.21.15.49:0/1935636722 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3400bb20 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.603+0000 7f7d39ffb700 1 -- 172.21.15.49:0/1935636722 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34012430 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.603+0000 7f7d413ff700 1 -- 172.21.15.49:0/1935636722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d3c0b0e20 msgr2=0x7f7d3c0b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:10.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.603+0000 7f7d413ff700 1 --2- 172.21.15.49:0/1935636722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d3c0b0e20 0x7f7d3c0b12a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009d90 tx=0x7f7d3400b330 comp rx=0 tx=0).stop 2023-07-18T03:39:10.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.604+0000 7f7d413ff700 1 -- 172.21.15.49:0/1935636722 shutdown_connections 2023-07-18T03:39:10.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.604+0000 7f7d413ff700 1 --2- 172.21.15.49:0/1935636722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d3c0b0e20 0x7f7d3c0b12a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.604+0000 7f7d413ff700 1 --2- 172.21.15.49:0/1935636722 >> v2:172.21.15.107:3300/0 conn(0x7f7d3c0d0b10 0x7f7d3c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.604+0000 7f7d413ff700 1 -- 172.21.15.49:0/1935636722 >> 172.21.15.49:0/1935636722 conn(0x7f7d3c09c1b0 msgr2=0x7f7d3c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:10.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.604+0000 7f7d413ff700 1 -- 172.21.15.49:0/1935636722 shutdown_connections 2023-07-18T03:39:10.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.604+0000 7f7d413ff700 1 -- 172.21.15.49:0/1935636722 wait complete. 2023-07-18T03:39:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.605+0000 7f7d413ff700 1 Processor -- start 2023-07-18T03:39:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.606+0000 7f7d413ff700 1 -- start start 2023-07-18T03:39:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.606+0000 7f7d413ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c0b0e20 0x7f7d3c15a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.606+0000 7f7d413ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d3c0b8a60 0x7f7d3c15d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.606+0000 7f7d413ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d3c0d0b10 0x7f7d3c0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:10.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.606+0000 7f7d413ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c15e700 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.606+0000 7f7d413ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c15da80 con 0x7f7d3c0b8a60 2023-07-18T03:39:10.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.606+0000 7f7d413ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c15dd60 con 0x7f7d3c0d0b10 2023-07-18T03:39:10.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.607+0000 7f7d3affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c0b0e20 0x7f7d3c15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:10.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.607+0000 7f7d3affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c0b0e20 0x7f7d3c15a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54534/0 (socket says 172.21.15.49:54534) 2023-07-18T03:39:10.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.607+0000 7f7d3affd700 1 -- 172.21.15.49:0/3581335757 learned_addr learned my addr 172.21.15.49:0/3581335757 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:10.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.607+0000 7f7d3b7fe700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d3c0d0b10 0x7f7d3c0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:10.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.607+0000 7f7d3a7fc700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d3c0b8a60 0x7f7d3c15d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:10.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.607+0000 7f7d3affd700 1 -- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d3c0d0b10 msgr2=0x7f7d3c0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:10.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.607+0000 7f7d3affd700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d3c0d0b10 0x7f7d3c0cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.607+0000 7f7d3affd700 1 -- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d3c0b8a60 msgr2=0x7f7d3c15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:10.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.607+0000 7f7d3affd700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d3c0b8a60 0x7f7d3c15d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.607+0000 7f7d3affd700 1 -- 172.21.15.49:0/3581335757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d34009a70 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.607+0000 7f7d3affd700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c0b0e20 0x7f7d3c15a670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c700 tx=0x7f7d2c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:10.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.608+0000 7f7d237fe700 1 -- 172.21.15.49:0/3581335757 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c005d70 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.608+0000 7f7d237fe700 1 -- 172.21.15.49:0/3581335757 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c0070d0 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.608+0000 7f7d413ff700 1 -- 172.21.15.49:0/3581335757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3c15e040 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.608+0000 7f7d237fe700 1 -- 172.21.15.49:0/3581335757 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c007460 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.608+0000 7f7d413ff700 1 -- 172.21.15.49:0/3581335757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c15f1d0 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.609+0000 7f7d413ff700 1 -- 172.21.15.49:0/3581335757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3c074350 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.610+0000 7f7d237fe700 1 -- 172.21.15.49:0/3581335757 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d2c0075c0 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.611+0000 7f7d237fe700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7d2806ace0 0x7f7d2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:10.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.611+0000 7f7d237fe700 1 -- 172.21.15.49:0/3581335757 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d2c0888e0 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.611+0000 7f7d3a7fc700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7d2806ace0 0x7f7d2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:10.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.612+0000 7f7d3a7fc700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7d2806ace0 0x7f7d2806d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c071a90 tx=0x7f7d30009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:10.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.615+0000 7f7d237fe700 1 -- 172.21.15.49:0/3581335757 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7d2c053980 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.822+0000 7f7d413ff700 1 -- 172.21.15.49:0/3581335757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d3c0b5550 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.822+0000 7f7d237fe700 1 -- 172.21.15.49:0/3581335757 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d2c017020 con 0x7f7d3c0b0e20 2023-07-18T03:39:10.817 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:39:10.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.824+0000 7f7d413ff700 1 -- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7d2806ace0 msgr2=0x7f7d2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:10.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.824+0000 7f7d413ff700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7d2806ace0 0x7f7d2806d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c071a90 tx=0x7f7d30009210 comp rx=0 tx=0).stop 2023-07-18T03:39:10.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.824+0000 7f7d413ff700 1 -- 172.21.15.49:0/3581335757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c0b0e20 msgr2=0x7f7d3c15a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:10.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.824+0000 7f7d413ff700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c0b0e20 0x7f7d3c15a670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c700 tx=0x7f7d2c00ca10 comp rx=0 tx=0).stop 2023-07-18T03:39:10.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.824+0000 7f7d413ff700 1 -- 172.21.15.49:0/3581335757 shutdown_connections 2023-07-18T03:39:10.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.824+0000 7f7d413ff700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d3c0d0b10 0x7f7d3c0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.824+0000 7f7d413ff700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7d2806ace0 0x7f7d2806d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.824+0000 7f7d413ff700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c0b0e20 0x7f7d3c15a670 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.824+0000 7f7d413ff700 1 --2- 172.21.15.49:0/3581335757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d3c0b8a60 0x7f7d3c15d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:10.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.825+0000 7f7d413ff700 1 -- 172.21.15.49:0/3581335757 >> 172.21.15.49:0/3581335757 conn(0x7f7d3c09c1b0 msgr2=0x7f7d3c1e4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:10.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.825+0000 7f7d413ff700 1 -- 172.21.15.49:0/3581335757 shutdown_connections 2023-07-18T03:39:10.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:10.825+0000 7f7d413ff700 1 -- 172.21.15.49:0/3581335757 wait complete. 2023-07-18T03:39:10.838 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:39:11.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.052+0000 7fcfd53ff700 1 Processor -- start 2023-07-18T03:39:11.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.052+0000 7fcfd53ff700 1 -- start start 2023-07-18T03:39:11.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.053+0000 7fcfd53ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcfd00d0010 0x7fcfd00b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.053+0000 7fcfd53ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd015cd60 con 0x7fcfd00d0480 2023-07-18T03:39:11.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.053+0000 7fcfd53ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd00cf340 con 0x7fcfd00b4600 2023-07-18T03:39:11.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.053+0000 7fcfd53ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd00d1550 con 0x7fcfd00d0010 2023-07-18T03:39:11.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.053+0000 7fcfce7fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcfd00d0010 0x7fcfd00b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.054+0000 7fcfcf7fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fcfd00d0480 0x7fcfd00b6430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42538/0 (socket says 172.21.15.49:42538) 2023-07-18T03:39:11.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.054+0000 7fcfcf7fe700 1 -- 172.21.15.49:0/291412013 learned_addr learned my addr 172.21.15.49:0/291412013 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:11.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.054+0000 7fcfce7fc700 1 -- 172.21.15.49:0/291412013 >> v1:172.21.15.107:6789/0 conn(0x7fcfd00b4600 legacy=0x7fcfd00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.054+0000 7fcfce7fc700 1 -- 172.21.15.49:0/291412013 >> v1:172.21.15.49:6789/0 conn(0x7fcfd00d0480 legacy=0x7fcfd00b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.054+0000 7fcfce7fc700 1 -- 172.21.15.49:0/291412013 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd00d0e90 con 0x7fcfd00d0010 2023-07-18T03:39:11.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.054+0000 7fcfcdffb700 1 -- 172.21.15.49:0/291412013 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895159219 0 0) 0x7fcfd015cd60 con 0x7fcfd00d0480 2023-07-18T03:39:11.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.054+0000 7fcfce7fc700 1 --2- 172.21.15.49:0/291412013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd00d0010 0x7fcfd00b5ef0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009840 tx=0x7fcfc4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ecca5658003bd0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:11.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.055+0000 7fcfcdffb700 1 -- 172.21.15.49:0/291412013 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4011830 con 0x7fcfd00d0010 2023-07-18T03:39:11.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.055+0000 7fcfcdffb700 1 -- 172.21.15.49:0/291412013 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc4011990 con 0x7fcfd00d0010 2023-07-18T03:39:11.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.055+0000 7fcfcdffb700 1 -- 172.21.15.49:0/291412013 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4011d20 con 0x7fcfd00d0010 2023-07-18T03:39:11.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.055+0000 7fcfd53ff700 1 -- 172.21.15.49:0/291412013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd00d0010 msgr2=0x7fcfd00b5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.055+0000 7fcfd53ff700 1 --2- 172.21.15.49:0/291412013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd00d0010 0x7fcfd00b5ef0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009840 tx=0x7fcfc4009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:11.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.055+0000 7fcfd53ff700 1 -- 172.21.15.49:0/291412013 shutdown_connections 2023-07-18T03:39:11.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.055+0000 7fcfd53ff700 1 --2- 172.21.15.49:0/291412013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd00d0010 0x7fcfd00b5ef0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.055+0000 7fcfd53ff700 1 -- 172.21.15.49:0/291412013 >> 172.21.15.49:0/291412013 conn(0x7fcfd009c200 msgr2=0x7fcfd00d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:11.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.055+0000 7fcfd53ff700 1 -- 172.21.15.49:0/291412013 shutdown_connections 2023-07-18T03:39:11.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.055+0000 7fcfd53ff700 1 -- 172.21.15.49:0/291412013 wait complete. 2023-07-18T03:39:11.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.057+0000 7fcfd53ff700 1 Processor -- start 2023-07-18T03:39:11.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.057+0000 7fcfd53ff700 1 -- start start 2023-07-18T03:39:11.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.058+0000 7fcfd53ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd00d0010 0x7fcfd01e5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.058+0000 7fcfd53ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfd00d0480 0x7fcfd00b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.058+0000 7fcfd53ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfd00c8440 0x7fcfd015b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.058+0000 7fcfd53ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd009cf10 con 0x7fcfd00d0010 2023-07-18T03:39:11.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.058+0000 7fcfd53ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00b2150 con 0x7fcfd00d0480 2023-07-18T03:39:11.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.058+0000 7fcfd53ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00b2430 con 0x7fcfd00c8440 2023-07-18T03:39:11.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.058+0000 7fcfceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd00d0010 0x7fcfd01e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.058+0000 7fcfceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd00d0010 0x7fcfd01e5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54550/0 (socket says 172.21.15.49:54550) 2023-07-18T03:39:11.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.058+0000 7fcfceffd700 1 -- 172.21.15.49:0/63583019 learned_addr learned my addr 172.21.15.49:0/63583019 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:11.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.059+0000 7fcfce7fc700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfd00d0480 0x7fcfd00b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.059+0000 7fcfcf7fe700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfd00c8440 0x7fcfd015b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.059+0000 7fcfceffd700 1 -- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfd00c8440 msgr2=0x7fcfd015b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.059+0000 7fcfceffd700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfd00c8440 0x7fcfd015b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.059+0000 7fcfceffd700 1 -- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfd00d0480 msgr2=0x7fcfd00b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.059+0000 7fcfceffd700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfd00d0480 0x7fcfd00b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.059+0000 7fcfceffd700 1 -- 172.21.15.49:0/63583019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc40094a0 con 0x7fcfd00d0010 2023-07-18T03:39:11.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.059+0000 7fcfceffd700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd00d0010 0x7fcfd01e5260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000c4e0 tx=0x7fcfc000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:11.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.059+0000 7fcfb77fe700 1 -- 172.21.15.49:0/63583019 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00052d0 con 0x7fcfd00d0010 2023-07-18T03:39:11.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.059+0000 7fcfb77fe700 1 -- 172.21.15.49:0/63583019 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc0005430 con 0x7fcfd00d0010 2023-07-18T03:39:11.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.059+0000 7fcfd53ff700 1 -- 172.21.15.49:0/63583019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd00b2710 con 0x7fcfd00d0010 2023-07-18T03:39:11.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.059+0000 7fcfb77fe700 1 -- 172.21.15.49:0/63583019 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00038c0 con 0x7fcfd00d0010 2023-07-18T03:39:11.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.060+0000 7fcfd53ff700 1 -- 172.21.15.49:0/63583019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd00c88b0 con 0x7fcfd00d0010 2023-07-18T03:39:11.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.061+0000 7fcfd53ff700 1 -- 172.21.15.49:0/63583019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd009cc70 con 0x7fcfd00d0010 2023-07-18T03:39:11.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.062+0000 7fcfb77fe700 1 -- 172.21.15.49:0/63583019 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcfc0003a20 con 0x7fcfd00d0010 2023-07-18T03:39:11.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.063+0000 7fcfb77fe700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcfbc06ace0 0x7fcfbc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.063+0000 7fcfb77fe700 1 -- 172.21.15.49:0/63583019 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcfc0088890 con 0x7fcfd00d0010 2023-07-18T03:39:11.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.063+0000 7fcfce7fc700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcfbc06ace0 0x7fcfbc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.064+0000 7fcfce7fc700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcfbc06ace0 0x7fcfbc06d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40090b0 tx=0x7fcfc401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:11.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.067+0000 7fcfb77fe700 1 -- 172.21.15.49:0/63583019 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcfd009cc70 con 0x7fcfd00d0010 2023-07-18T03:39:11.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.276+0000 7fcfd53ff700 1 -- 172.21.15.49:0/63583019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcfd009cc70 con 0x7fcfd00d0010 2023-07-18T03:39:11.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.277+0000 7fcfb77fe700 1 -- 172.21.15.49:0/63583019 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcfc0017020 con 0x7fcfd00d0010 2023-07-18T03:39:11.272 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:39:11.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.279+0000 7fcfd53ff700 1 -- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcfbc06ace0 msgr2=0x7fcfbc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.279+0000 7fcfd53ff700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcfbc06ace0 0x7fcfbc06d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40090b0 tx=0x7fcfc401c040 comp rx=0 tx=0).stop 2023-07-18T03:39:11.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.279+0000 7fcfd53ff700 1 -- 172.21.15.49:0/63583019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd00d0010 msgr2=0x7fcfd01e5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.279+0000 7fcfd53ff700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd00d0010 0x7fcfd01e5260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000c4e0 tx=0x7fcfc000c7f0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.279+0000 7fcfd53ff700 1 -- 172.21.15.49:0/63583019 shutdown_connections 2023-07-18T03:39:11.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.279+0000 7fcfd53ff700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfd00c8440 0x7fcfd015b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.279+0000 7fcfd53ff700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcfbc06ace0 0x7fcfbc06d190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.279+0000 7fcfd53ff700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcfd00d0010 0x7fcfd01e5260 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.279+0000 7fcfd53ff700 1 --2- 172.21.15.49:0/63583019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfd00d0480 0x7fcfd00b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.279+0000 7fcfd53ff700 1 -- 172.21.15.49:0/63583019 >> 172.21.15.49:0/63583019 conn(0x7fcfd009c200 msgr2=0x7fcfd00d3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:11.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.279+0000 7fcfd53ff700 1 -- 172.21.15.49:0/63583019 shutdown_connections 2023-07-18T03:39:11.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.279+0000 7fcfd53ff700 1 -- 172.21.15.49:0/63583019 wait complete. 2023-07-18T03:39:11.294 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:11.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.504+0000 7f79877ff700 1 Processor -- start 2023-07-18T03:39:11.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.504+0000 7f79877ff700 1 -- start start 2023-07-18T03:39:11.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.505+0000 7f79877ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f79800d1220 0x7f79800b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.505+0000 7f79877ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79800d2ce0 con 0x7f79800b1e00 2023-07-18T03:39:11.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.505+0000 7f79877ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79800b3350 con 0x7f798015c950 2023-07-18T03:39:11.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.505+0000 7f79877ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79800b3630 con 0x7f79800d1220 2023-07-18T03:39:11.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.505+0000 7f7984d9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f79800d1220 0x7f79800b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.506+0000 7f7985d9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f79800b1e00 0x7f79800afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42540/0 (socket says 172.21.15.49:42540) 2023-07-18T03:39:11.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.506+0000 7f7985d9c700 1 -- 172.21.15.49:0/2932085537 learned_addr learned my addr 172.21.15.49:0/2932085537 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:11.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.506+0000 7f7984d9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f79800d1220 0x7f79800b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54560/0 (socket says 172.21.15.49:54560) 2023-07-18T03:39:11.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.506+0000 7f7984d9a700 1 -- 172.21.15.49:0/2932085537 >> v1:172.21.15.107:6789/0 conn(0x7f798015c950 legacy=0x7f79800d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.506+0000 7f7984d9a700 1 -- 172.21.15.49:0/2932085537 >> v1:172.21.15.49:6789/0 conn(0x7f79800b1e00 legacy=0x7f79800afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.506+0000 7f7984d9a700 1 -- 172.21.15.49:0/2932085537 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79800b3910 con 0x7f79800d1220 2023-07-18T03:39:11.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.506+0000 7f7977fff700 1 -- 172.21.15.49:0/2932085537 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267361515 0 0) 0x7f79800d2ce0 con 0x7f79800b1e00 2023-07-18T03:39:11.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.506+0000 7f7984d9a700 1 --2- 172.21.15.49:0/2932085537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d1220 0x7f79800b43f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7970009840 tx=0x7f7970009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709c8a8ca827adde server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:11.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.507+0000 7f7977fff700 1 -- 172.21.15.49:0/2932085537 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970011830 con 0x7f79800d1220 2023-07-18T03:39:11.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.507+0000 7f7977fff700 1 -- 172.21.15.49:0/2932085537 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970011990 con 0x7f79800d1220 2023-07-18T03:39:11.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.507+0000 7f7977fff700 1 -- 172.21.15.49:0/2932085537 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970011d20 con 0x7f79800d1220 2023-07-18T03:39:11.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.507+0000 7f79877ff700 1 -- 172.21.15.49:0/2932085537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d1220 msgr2=0x7f79800b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.507+0000 7f79877ff700 1 --2- 172.21.15.49:0/2932085537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d1220 0x7f79800b43f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7970009840 tx=0x7f7970009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:11.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.507+0000 7f79877ff700 1 -- 172.21.15.49:0/2932085537 shutdown_connections 2023-07-18T03:39:11.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.507+0000 7f79877ff700 1 --2- 172.21.15.49:0/2932085537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d1220 0x7f79800b43f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.507+0000 7f79877ff700 1 -- 172.21.15.49:0/2932085537 >> 172.21.15.49:0/2932085537 conn(0x7f798009c1d0 msgr2=0x7f79800d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:11.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.507+0000 7f79877ff700 1 -- 172.21.15.49:0/2932085537 shutdown_connections 2023-07-18T03:39:11.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.507+0000 7f79877ff700 1 -- 172.21.15.49:0/2932085537 wait complete. 2023-07-18T03:39:11.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.509+0000 7f79877ff700 1 Processor -- start 2023-07-18T03:39:11.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.509+0000 7f79877ff700 1 -- start start 2023-07-18T03:39:11.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.510+0000 7f79877ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79800b1e00 0x7f7980151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.510+0000 7f79877ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d1220 0x7f79800b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.510+0000 7f79877ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f798015c950 0x7f79800b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.510+0000 7f79877ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79800d03a0 con 0x7f79800d1220 2023-07-18T03:39:11.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.510+0000 7f79877ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79800b8780 con 0x7f798015c950 2023-07-18T03:39:11.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.510+0000 7f79877ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79800b8a60 con 0x7f79800b1e00 2023-07-18T03:39:11.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.510+0000 7f7984d9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d1220 0x7f79800b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.510+0000 7f7984d9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d1220 0x7f79800b7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54564/0 (socket says 172.21.15.49:54564) 2023-07-18T03:39:11.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.510+0000 7f7984d9a700 1 -- 172.21.15.49:0/1286438840 learned_addr learned my addr 172.21.15.49:0/1286438840 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:11.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.511+0000 7f798559b700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79800b1e00 0x7f7980151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.511+0000 7f7985d9c700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f798015c950 0x7f79800b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.511+0000 7f7984d9a700 1 -- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79800b1e00 msgr2=0x7f7980151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.511+0000 7f7984d9a700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79800b1e00 0x7f7980151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.511+0000 7f7984d9a700 1 -- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f798015c950 msgr2=0x7f79800b80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.511+0000 7f7984d9a700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f798015c950 0x7f79800b80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.511+0000 7f7984d9a700 1 -- 172.21.15.49:0/1286438840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79700094a0 con 0x7f79800d1220 2023-07-18T03:39:11.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.511+0000 7f7984d9a700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d1220 0x7f79800b7ba0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f79700090b0 tx=0x7f7970004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:11.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.512+0000 7f7975ffb700 1 -- 172.21.15.49:0/1286438840 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797001a860 con 0x7f79800d1220 2023-07-18T03:39:11.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.512+0000 7f7975ffb700 1 -- 172.21.15.49:0/1286438840 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797001a9c0 con 0x7f79800d1220 2023-07-18T03:39:11.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.512+0000 7f7975ffb700 1 -- 172.21.15.49:0/1286438840 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797001ad50 con 0x7f79800d1220 2023-07-18T03:39:11.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.512+0000 7f79877ff700 1 -- 172.21.15.49:0/1286438840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7980158900 con 0x7f79800d1220 2023-07-18T03:39:11.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.512+0000 7f79877ff700 1 -- 172.21.15.49:0/1286438840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7980158ec0 con 0x7f79800d1220 2023-07-18T03:39:11.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.513+0000 7f79877ff700 1 -- 172.21.15.49:0/1286438840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f798004e280 con 0x7f79800d1220 2023-07-18T03:39:11.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.514+0000 7f7975ffb700 1 -- 172.21.15.49:0/1286438840 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7970022070 con 0x7f79800d1220 2023-07-18T03:39:11.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.515+0000 7f7975ffb700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f796406ace0 0x7f796406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.515+0000 7f7975ffb700 1 -- 172.21.15.49:0/1286438840 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f797008a3a0 con 0x7f79800d1220 2023-07-18T03:39:11.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.515+0000 7f798559b700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f796406ace0 0x7f796406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.516+0000 7f798559b700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f796406ace0 0x7f796406d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f798015ba90 tx=0x7f797c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:11.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.519+0000 7f7975ffb700 1 -- 172.21.15.49:0/1286438840 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f79700555a0 con 0x7f79800d1220 2023-07-18T03:39:11.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.730+0000 7f79877ff700 1 -- 172.21.15.49:0/1286438840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f798008c5a0 con 0x7f79800d1220 2023-07-18T03:39:11.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.731+0000 7f7975ffb700 1 -- 172.21.15.49:0/1286438840 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f7970058bc0 con 0x7f79800d1220 2023-07-18T03:39:11.726 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:11.727 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:37.510006+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:11.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.734+0000 7f79877ff700 1 -- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f796406ace0 msgr2=0x7f796406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.734+0000 7f79877ff700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f796406ace0 0x7f796406d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f798015ba90 tx=0x7f797c009040 comp rx=0 tx=0).stop 2023-07-18T03:39:11.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.734+0000 7f79877ff700 1 -- 172.21.15.49:0/1286438840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d1220 msgr2=0x7f79800b7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.734+0000 7f79877ff700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d1220 0x7f79800b7ba0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f79700090b0 tx=0x7f7970004750 comp rx=0 tx=0).stop 2023-07-18T03:39:11.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.734+0000 7f79877ff700 1 -- 172.21.15.49:0/1286438840 shutdown_connections 2023-07-18T03:39:11.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.734+0000 7f79877ff700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f798015c950 0x7f79800b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.734+0000 7f79877ff700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79800b1e00 0x7f7980151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.734+0000 7f79877ff700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f796406ace0 0x7f796406d190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.734+0000 7f79877ff700 1 --2- 172.21.15.49:0/1286438840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d1220 0x7f79800b7ba0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.734+0000 7f79877ff700 1 -- 172.21.15.49:0/1286438840 >> 172.21.15.49:0/1286438840 conn(0x7f798009c1d0 msgr2=0x7f79800b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:11.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.734+0000 7f79877ff700 1 -- 172.21.15.49:0/1286438840 shutdown_connections 2023-07-18T03:39:11.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.734+0000 7f79877ff700 1 -- 172.21.15.49:0/1286438840 wait complete. 2023-07-18T03:39:11.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:38:33.773236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:38:35.023692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:39:11.749 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:39:11.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.964+0000 7f12c5dff700 1 Processor -- start 2023-07-18T03:39:11.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.964+0000 7f12c5dff700 1 -- start start 2023-07-18T03:39:11.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.965+0000 7f12c5dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12c00d0480 0x7f12c00b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.965+0000 7f12c5dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c015cd60 con 0x7f12c00b4600 2023-07-18T03:39:11.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.965+0000 7f12c5dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c00cf340 con 0x7f12c00d0010 2023-07-18T03:39:11.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.965+0000 7f12c5dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c00d1550 con 0x7f12c00d0480 2023-07-18T03:39:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.966+0000 7f12bf7fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f12c00b4600 0x7f12c00cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42548/0 (socket says 172.21.15.49:42548) 2023-07-18T03:39:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.966+0000 7f12bf7fe700 1 -- 172.21.15.49:0/506996111 learned_addr learned my addr 172.21.15.49:0/506996111 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:11.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.966+0000 7f12bffff700 1 --2- 172.21.15.49:0/506996111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12c00d0480 0x7f12c00b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.966+0000 7f12be7fc700 1 -- 172.21.15.49:0/506996111 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623464544 0 0) 0x7f12c015cd60 con 0x7f12c00b4600 2023-07-18T03:39:11.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.966+0000 7f12be7fc700 1 -- 172.21.15.49:0/506996111 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12a80035b0 con 0x7f12c00b4600 2023-07-18T03:39:11.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.966+0000 7f12be7fc700 1 -- 172.21.15.49:0/506996111 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355690966 0 0) 0x7f12c00cf340 con 0x7f12c00d0010 2023-07-18T03:39:11.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.966+0000 7f12be7fc700 1 -- 172.21.15.49:0/506996111 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12c015cd60 con 0x7f12c00d0010 2023-07-18T03:39:11.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.966+0000 7f12bffff700 1 -- 172.21.15.49:0/506996111 >> v1:172.21.15.107:6789/0 conn(0x7f12c00d0010 legacy=0x7f12c00b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.966+0000 7f12bffff700 1 -- 172.21.15.49:0/506996111 >> v1:172.21.15.49:6789/0 conn(0x7f12c00b4600 legacy=0x7f12c00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.966+0000 7f12bffff700 1 -- 172.21.15.49:0/506996111 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c00d0e90 con 0x7f12c00d0480 2023-07-18T03:39:11.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.967+0000 7f12bffff700 1 --2- 172.21.15.49:0/506996111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12c00d0480 0x7f12c00b6600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009dc0 tx=0x7f12b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0b220fd5a625647 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:11.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.967+0000 7f12be7fc700 1 -- 172.21.15.49:0/506996111 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b800e070 con 0x7f12c00d0480 2023-07-18T03:39:11.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.967+0000 7f12be7fc700 1 -- 172.21.15.49:0/506996111 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b800bb20 con 0x7f12c00d0480 2023-07-18T03:39:11.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.967+0000 7f12be7fc700 1 -- 172.21.15.49:0/506996111 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b8012430 con 0x7f12c00d0480 2023-07-18T03:39:11.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.967+0000 7f12c5dff700 1 -- 172.21.15.49:0/506996111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12c00d0480 msgr2=0x7f12c00b6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.967+0000 7f12c5dff700 1 --2- 172.21.15.49:0/506996111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12c00d0480 0x7f12c00b6600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009dc0 tx=0x7f12b800b330 comp rx=0 tx=0).stop 2023-07-18T03:39:11.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.967+0000 7f12c5dff700 1 -- 172.21.15.49:0/506996111 shutdown_connections 2023-07-18T03:39:11.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.967+0000 7f12c5dff700 1 --2- 172.21.15.49:0/506996111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12c00d0480 0x7f12c00b6600 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.967+0000 7f12c5dff700 1 -- 172.21.15.49:0/506996111 >> 172.21.15.49:0/506996111 conn(0x7f12c009c200 msgr2=0x7f12c00d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:11.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.967+0000 7f12c5dff700 1 -- 172.21.15.49:0/506996111 shutdown_connections 2023-07-18T03:39:11.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.967+0000 7f12c5dff700 1 -- 172.21.15.49:0/506996111 wait complete. 2023-07-18T03:39:11.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.969+0000 7f12c5dff700 1 Processor -- start 2023-07-18T03:39:11.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.969+0000 7f12c5dff700 1 -- start start 2023-07-18T03:39:11.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.970+0000 7f12c5dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12c00d0010 0x7f12c00afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.970+0000 7f12c5dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12c00d0480 0x7f12c015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.970+0000 7f12c5dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12c00b0280 0x7f12c015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.970+0000 7f12c5dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c015b410 con 0x7f12c00d0480 2023-07-18T03:39:11.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.970+0000 7f12c5dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c00b2820 con 0x7f12c00b0280 2023-07-18T03:39:11.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.970+0000 7f12c5dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c00b2b00 con 0x7f12c00d0010 2023-07-18T03:39:11.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.971+0000 7f12beffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12c00d0480 0x7f12c015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.971+0000 7f12beffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12c00d0480 0x7f12c015a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54566/0 (socket says 172.21.15.49:54566) 2023-07-18T03:39:11.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.971+0000 7f12beffd700 1 -- 172.21.15.49:0/55313215 learned_addr learned my addr 172.21.15.49:0/55313215 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:11.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.971+0000 7f12bf7fe700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12c00d0010 0x7f12c00afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.971+0000 7f12bffff700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12c00b0280 0x7f12c015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.971+0000 7f12beffd700 1 -- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12c00d0010 msgr2=0x7f12c00afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.971+0000 7f12beffd700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12c00d0010 0x7f12c00afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.971+0000 7f12beffd700 1 -- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12c00b0280 msgr2=0x7f12c015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:11.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.971+0000 7f12beffd700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12c00b0280 0x7f12c015aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:11.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.971+0000 7f12beffd700 1 -- 172.21.15.49:0/55313215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b8009a70 con 0x7f12c00d0480 2023-07-18T03:39:11.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.971+0000 7f12bffff700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12c00b0280 0x7f12c015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:11.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.971+0000 7f12beffd700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12c00d0480 0x7f12c015a990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f12b400c520 tx=0x7f12b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:11.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.972+0000 7f12a7fff700 1 -- 172.21.15.49:0/55313215 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b40052d0 con 0x7f12c00d0480 2023-07-18T03:39:11.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.972+0000 7f12a7fff700 1 -- 172.21.15.49:0/55313215 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b4005430 con 0x7f12c00d0480 2023-07-18T03:39:11.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.972+0000 7f12a7fff700 1 -- 172.21.15.49:0/55313215 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b40038c0 con 0x7f12c00d0480 2023-07-18T03:39:11.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.972+0000 7f12c5dff700 1 -- 172.21.15.49:0/55313215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c00b2de0 con 0x7f12c00d0480 2023-07-18T03:39:11.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.972+0000 7f12c5dff700 1 -- 172.21.15.49:0/55313215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c00b33a0 con 0x7f12c00d0480 2023-07-18T03:39:11.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.973+0000 7f12c5dff700 1 -- 172.21.15.49:0/55313215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c0074350 con 0x7f12c00d0480 2023-07-18T03:39:11.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.974+0000 7f12a7fff700 1 -- 172.21.15.49:0/55313215 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f12b4003a20 con 0x7f12c00d0480 2023-07-18T03:39:11.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.975+0000 7f12a7fff700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f12ac06ace0 0x7f12ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:11.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.975+0000 7f12a7fff700 1 -- 172.21.15.49:0/55313215 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f12b4088b80 con 0x7f12c00d0480 2023-07-18T03:39:11.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.976+0000 7f12bf7fe700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f12ac06ace0 0x7f12ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:11.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.976+0000 7f12bf7fe700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f12ac06ace0 0x7f12ac06d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f12b00011e0 tx=0x7f12b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:11.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:11.979+0000 7f12a7fff700 1 -- 172.21.15.49:0/55313215 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f12b4053b00 con 0x7f12c00d0480 2023-07-18T03:39:12.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.189+0000 7f12c5dff700 1 -- 172.21.15.49:0/55313215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f12c00d1830 con 0x7f12c00d0480 2023-07-18T03:39:12.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.189+0000 7f12a7fff700 1 -- 172.21.15.49:0/55313215 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f12b4088e60 con 0x7f12c00d0480 2023-07-18T03:39:12.184 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:39:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.191+0000 7f12c5dff700 1 -- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f12ac06ace0 msgr2=0x7f12ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.191+0000 7f12c5dff700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f12ac06ace0 0x7f12ac06d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f12b00011e0 tx=0x7f12b0009040 comp rx=0 tx=0).stop 2023-07-18T03:39:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.191+0000 7f12c5dff700 1 -- 172.21.15.49:0/55313215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12c00d0480 msgr2=0x7f12c015a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.191+0000 7f12c5dff700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12c00d0480 0x7f12c015a990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f12b400c520 tx=0x7f12b400c8e0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.191+0000 7f12c5dff700 1 -- 172.21.15.49:0/55313215 shutdown_connections 2023-07-18T03:39:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.191+0000 7f12c5dff700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12c00b0280 0x7f12c015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.191+0000 7f12c5dff700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12c00d0010 0x7f12c00afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.191+0000 7f12c5dff700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f12ac06ace0 0x7f12ac06d190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.191+0000 7f12c5dff700 1 --2- 172.21.15.49:0/55313215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12c00d0480 0x7f12c015a990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.191+0000 7f12c5dff700 1 -- 172.21.15.49:0/55313215 >> 172.21.15.49:0/55313215 conn(0x7f12c009c200 msgr2=0x7f12c00b7110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:12.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.192+0000 7f12c5dff700 1 -- 172.21.15.49:0/55313215 shutdown_connections 2023-07-18T03:39:12.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.192+0000 7f12c5dff700 1 -- 172.21.15.49:0/55313215 wait complete. 2023-07-18T03:39:12.204 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:39:12.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.417+0000 7f745ffff700 1 Processor -- start 2023-07-18T03:39:12.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.417+0000 7f745ffff700 1 -- start start 2023-07-18T03:39:12.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.419+0000 7f745ffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74580b8a60 0x7f74580b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.419+0000 7f745ffff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f74580b4530 0x7f74580d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.419+0000 7f745ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74580d1050 0x7f74580b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.419+0000 7f745ffff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74580d14d0 con 0x7f74580b8a60 2023-07-18T03:39:12.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.419+0000 7f745ffff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74580b8e30 con 0x7f74580b4530 2023-07-18T03:39:12.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.419+0000 7f745ffff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74580afaa0 con 0x7f74580d1050 2023-07-18T03:39:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.419+0000 7f745dd9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74580b8a60 0x7f74580b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.419+0000 7f745dd9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74580b8a60 0x7f74580b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54572/0 (socket says 172.21.15.49:54572) 2023-07-18T03:39:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.419+0000 7f745dd9b700 1 -- 172.21.15.49:0/3308437531 learned_addr learned my addr 172.21.15.49:0/3308437531 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.419+0000 7f745e59c700 1 --2- 172.21.15.49:0/3308437531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74580d1050 0x7f74580b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.419+0000 7f745d59a700 1 --2- 172.21.15.49:0/3308437531 >> v2:172.21.15.107:3300/0 conn(0x7f74580b4530 0x7f74580d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.420+0000 7f745dd9b700 1 -- 172.21.15.49:0/3308437531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74580d1050 msgr2=0x7f74580b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.420+0000 7f745dd9b700 1 --2- 172.21.15.49:0/3308437531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74580d1050 0x7f74580b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.420+0000 7f745dd9b700 1 -- 172.21.15.49:0/3308437531 >> v2:172.21.15.107:3300/0 conn(0x7f74580b4530 msgr2=0x7f74580d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.420+0000 7f745dd9b700 1 --2- 172.21.15.49:0/3308437531 >> v2:172.21.15.107:3300/0 conn(0x7f74580b4530 0x7f74580d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.420+0000 7f745dd9b700 1 -- 172.21.15.49:0/3308437531 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580afd80 con 0x7f74580b8a60 2023-07-18T03:39:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.420+0000 7f745e59c700 1 --2- 172.21.15.49:0/3308437531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74580d1050 0x7f74580b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.420+0000 7f745dd9b700 1 --2- 172.21.15.49:0/3308437531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74580b8a60 0x7f74580b3ff0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f745809c8e0 tx=0x7f7454009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc831636d268e3d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.420+0000 7f745cd99700 1 -- 172.21.15.49:0/3308437531 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745400b870 con 0x7f74580b8a60 2023-07-18T03:39:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.420+0000 7f745cd99700 1 -- 172.21.15.49:0/3308437531 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745400b9d0 con 0x7f74580b8a60 2023-07-18T03:39:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.420+0000 7f745cd99700 1 -- 172.21.15.49:0/3308437531 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745400bd60 con 0x7f74580b8a60 2023-07-18T03:39:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.421+0000 7f745ffff700 1 -- 172.21.15.49:0/3308437531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74580b8a60 msgr2=0x7f74580b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.421+0000 7f745ffff700 1 --2- 172.21.15.49:0/3308437531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74580b8a60 0x7f74580b3ff0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f745809c8e0 tx=0x7f7454009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.421+0000 7f745ffff700 1 -- 172.21.15.49:0/3308437531 shutdown_connections 2023-07-18T03:39:12.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.421+0000 7f745ffff700 1 --2- 172.21.15.49:0/3308437531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74580d1050 0x7f74580b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.421+0000 7f745ffff700 1 --2- 172.21.15.49:0/3308437531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74580b8a60 0x7f74580b3ff0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.421+0000 7f745ffff700 1 --2- 172.21.15.49:0/3308437531 >> v2:172.21.15.107:3300/0 conn(0x7f74580b4530 0x7f74580d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.421+0000 7f745ffff700 1 -- 172.21.15.49:0/3308437531 >> 172.21.15.49:0/3308437531 conn(0x7f745809c1b0 msgr2=0x7f74580d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:12.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.421+0000 7f745ffff700 1 -- 172.21.15.49:0/3308437531 shutdown_connections 2023-07-18T03:39:12.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.421+0000 7f745ffff700 1 -- 172.21.15.49:0/3308437531 wait complete. 2023-07-18T03:39:12.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.423+0000 7f745ffff700 1 Processor -- start 2023-07-18T03:39:12.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.424+0000 7f745ffff700 1 -- start start 2023-07-18T03:39:12.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.425+0000 7f745ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74580b4530 0x7f74580c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.425+0000 7f745ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74580b8a60 0x7f7458152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.425+0000 7f745ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74580d1050 0x7f74581f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.425+0000 7f745ffff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74581f2cc0 con 0x7f74580b8a60 2023-07-18T03:39:12.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.425+0000 7f745ffff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580ccd80 con 0x7f74580d1050 2023-07-18T03:39:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.425+0000 7f745ffff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74580c9e90 con 0x7f74580b4530 2023-07-18T03:39:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.425+0000 7f745d59a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74580b8a60 0x7f7458152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.425+0000 7f745d59a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74580b8a60 0x7f7458152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54584/0 (socket says 172.21.15.49:54584) 2023-07-18T03:39:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.425+0000 7f745d59a700 1 -- 172.21.15.49:0/881625832 learned_addr learned my addr 172.21.15.49:0/881625832 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.425+0000 7f745e59c700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74580d1050 0x7f74581f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.425+0000 7f745dd9b700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74580b4530 0x7f74580c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.426+0000 7f745d59a700 1 -- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74580b4530 msgr2=0x7f74580c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:12.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.426+0000 7f745d59a700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74580b4530 0x7f74580c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.426+0000 7f745d59a700 1 -- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74580d1050 msgr2=0x7f74581f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:12.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.426+0000 7f745d59a700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74580d1050 0x7f74581f1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.426+0000 7f745d59a700 1 -- 172.21.15.49:0/881625832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7454009550 con 0x7f74580b8a60 2023-07-18T03:39:12.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.426+0000 7f745d59a700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74580b8a60 0x7f7458152440 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f744800e810 tx=0x7f744800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:12.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.426+0000 7f744e7fc700 1 -- 172.21.15.49:0/881625832 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744800c070 con 0x7f74580b8a60 2023-07-18T03:39:12.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.426+0000 7f744e7fc700 1 -- 172.21.15.49:0/881625832 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7448013610 con 0x7f74580b8a60 2023-07-18T03:39:12.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.426+0000 7f745ffff700 1 -- 172.21.15.49:0/881625832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7458152980 con 0x7f74580b8a60 2023-07-18T03:39:12.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.426+0000 7f744e7fc700 1 -- 172.21.15.49:0/881625832 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74480139a0 con 0x7f74580b8a60 2023-07-18T03:39:12.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.426+0000 7f745ffff700 1 -- 172.21.15.49:0/881625832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74580cc7c0 con 0x7f74580b8a60 2023-07-18T03:39:12.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.427+0000 7f745ffff700 1 -- 172.21.15.49:0/881625832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7458074350 con 0x7f74580b8a60 2023-07-18T03:39:12.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.428+0000 7f744e7fc700 1 -- 172.21.15.49:0/881625832 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7448013b00 con 0x7f74580b8a60 2023-07-18T03:39:12.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.429+0000 7f744e7fc700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f743c06ace0 0x7f743c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.429+0000 7f744e7fc700 1 -- 172.21.15.49:0/881625832 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7448089160 con 0x7f74580b8a60 2023-07-18T03:39:12.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.430+0000 7f745dd9b700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f743c06ace0 0x7f743c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:12.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.430+0000 7f745dd9b700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f743c06ace0 0x7f743c06d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7454000c00 tx=0x7f7454008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:12.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.434+0000 7f744e7fc700 1 -- 172.21.15.49:0/881625832 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7448054200 con 0x7f74580b8a60 2023-07-18T03:39:12.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.641+0000 7f745ffff700 1 -- 172.21.15.49:0/881625832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74580d23e0 con 0x7f74580b8a60 2023-07-18T03:39:12.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.642+0000 7f744e7fc700 1 -- 172.21.15.49:0/881625832 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7448009440 con 0x7f74580b8a60 2023-07-18T03:39:12.637 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:39:12.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.644+0000 7f745ffff700 1 -- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f743c06ace0 msgr2=0x7f743c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:12.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.644+0000 7f745ffff700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f743c06ace0 0x7f743c06d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7454000c00 tx=0x7f7454008030 comp rx=0 tx=0).stop 2023-07-18T03:39:12.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.644+0000 7f745ffff700 1 -- 172.21.15.49:0/881625832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74580b8a60 msgr2=0x7f7458152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:12.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.644+0000 7f745ffff700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74580b8a60 0x7f7458152440 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f744800e810 tx=0x7f744800eb20 comp rx=0 tx=0).stop 2023-07-18T03:39:12.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.644+0000 7f745ffff700 1 -- 172.21.15.49:0/881625832 shutdown_connections 2023-07-18T03:39:12.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.644+0000 7f745ffff700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74580d1050 0x7f74581f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.644+0000 7f745ffff700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74580b4530 0x7f74580c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.644+0000 7f745ffff700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f743c06ace0 0x7f743c06d190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.644+0000 7f745ffff700 1 --2- 172.21.15.49:0/881625832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74580b8a60 0x7f7458152440 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.644+0000 7f745ffff700 1 -- 172.21.15.49:0/881625832 >> 172.21.15.49:0/881625832 conn(0x7f745809c1b0 msgr2=0x7f74580b7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:12.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.645+0000 7f745ffff700 1 -- 172.21.15.49:0/881625832 shutdown_connections 2023-07-18T03:39:12.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.645+0000 7f745ffff700 1 -- 172.21.15.49:0/881625832 wait complete. 2023-07-18T03:39:12.658 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:39:12.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.870+0000 7fde075ff700 1 Processor -- start 2023-07-18T03:39:12.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.871+0000 7fde075ff700 1 -- start start 2023-07-18T03:39:12.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.872+0000 7fde075ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fde000b8a60 0x7fde000b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.872+0000 7fde075ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fde000b4530 0x7fde000d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.872+0000 7fde075ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde000d1490 con 0x7fde000d1050 2023-07-18T03:39:12.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.872+0000 7fde075ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde000b8e30 con 0x7fde000b8a60 2023-07-18T03:39:12.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.872+0000 7fde075ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde000b1e30 con 0x7fde000b4530 2023-07-18T03:39:12.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.872+0000 7fde0539b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fde000b8a60 0x7fde000b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:12.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.872+0000 7fde0539b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fde000b8a60 0x7fde000b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54588/0 (socket says 172.21.15.49:54588) 2023-07-18T03:39:12.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.872+0000 7fde0539b700 1 -- 172.21.15.49:0/2224534688 learned_addr learned my addr 172.21.15.49:0/2224534688 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:12.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.873+0000 7fde0539b700 1 -- 172.21.15.49:0/2224534688 >> v2:172.21.15.107:3300/0 conn(0x7fde000b4530 msgr2=0x7fde000d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:12.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.873+0000 7fde0539b700 1 --2- 172.21.15.49:0/2224534688 >> v2:172.21.15.107:3300/0 conn(0x7fde000b4530 0x7fde000d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.873+0000 7fde0539b700 1 -- 172.21.15.49:0/2224534688 >> v1:172.21.15.107:6789/0 conn(0x7fde000d1050 legacy=0x7fde000b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:12.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.873+0000 7fde0539b700 1 -- 172.21.15.49:0/2224534688 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000afaa0 con 0x7fde000b8a60 2023-07-18T03:39:12.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.873+0000 7fde0539b700 1 --2- 172.21.15.49:0/2224534688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde000b8a60 0x7fde000b3ff0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fde0009c8e0 tx=0x7fddfc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8832ff9f778ed9db server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:12.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.873+0000 7fddf7fff700 1 -- 172.21.15.49:0/2224534688 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc004230 con 0x7fde000b8a60 2023-07-18T03:39:12.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.873+0000 7fddf7fff700 1 -- 172.21.15.49:0/2224534688 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc004390 con 0x7fde000b8a60 2023-07-18T03:39:12.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.873+0000 7fddf7fff700 1 -- 172.21.15.49:0/2224534688 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0036a0 con 0x7fde000b8a60 2023-07-18T03:39:12.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.874+0000 7fde075ff700 1 -- 172.21.15.49:0/2224534688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde000b8a60 msgr2=0x7fde000b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:12.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.874+0000 7fde075ff700 1 --2- 172.21.15.49:0/2224534688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde000b8a60 0x7fde000b3ff0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fde0009c8e0 tx=0x7fddfc009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.874+0000 7fde075ff700 1 -- 172.21.15.49:0/2224534688 shutdown_connections 2023-07-18T03:39:12.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.874+0000 7fde075ff700 1 --2- 172.21.15.49:0/2224534688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde000b8a60 0x7fde000b3ff0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.874+0000 7fde075ff700 1 --2- 172.21.15.49:0/2224534688 >> v2:172.21.15.107:3300/0 conn(0x7fde000b4530 0x7fde000d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.874+0000 7fde075ff700 1 -- 172.21.15.49:0/2224534688 >> 172.21.15.49:0/2224534688 conn(0x7fde0009c1b0 msgr2=0x7fde000d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:12.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.874+0000 7fde075ff700 1 -- 172.21.15.49:0/2224534688 shutdown_connections 2023-07-18T03:39:12.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.874+0000 7fde075ff700 1 -- 172.21.15.49:0/2224534688 wait complete. 2023-07-18T03:39:12.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.876+0000 7fde075ff700 1 Processor -- start 2023-07-18T03:39:12.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.876+0000 7fde075ff700 1 -- start start 2023-07-18T03:39:12.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde075ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde000b4530 0x7fde001524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde075ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde000b8a60 0x7fde0015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde075ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde000d1050 0x7fde000cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde075ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000cae30 con 0x7fde000d1050 2023-07-18T03:39:12.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde075ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000ca1b0 con 0x7fde000b8a60 2023-07-18T03:39:12.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde075ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde000ca490 con 0x7fde000b4530 2023-07-18T03:39:12.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde05b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde000d1050 0x7fde000cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:12.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde05b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde000d1050 0x7fde000cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54600/0 (socket says 172.21.15.49:54600) 2023-07-18T03:39:12.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde05b9c700 1 -- 172.21.15.49:0/2896116156 learned_addr learned my addr 172.21.15.49:0/2896116156 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:12.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde0539b700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde000b4530 0x7fde001524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:12.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde04b9a700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde000b8a60 0x7fde0015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:12.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde05b9c700 1 -- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde000b4530 msgr2=0x7fde001524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:12.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde05b9c700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde000b4530 0x7fde001524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde05b9c700 1 -- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde000b8a60 msgr2=0x7fde0015dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:12.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde05b9c700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde000b8a60 0x7fde0015dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:12.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde05b9c700 1 -- 172.21.15.49:0/2896116156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc009550 con 0x7fde000d1050 2023-07-18T03:39:12.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.877+0000 7fde0539b700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde000b4530 0x7fde001524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:12.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.878+0000 7fde05b9c700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde000d1050 0x7fde000cf520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fddf800b990 tx=0x7fddf800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:12.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.878+0000 7fddf5ffb700 1 -- 172.21.15.49:0/2896116156 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf800c9d0 con 0x7fde000d1050 2023-07-18T03:39:12.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.878+0000 7fddf5ffb700 1 -- 172.21.15.49:0/2896116156 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf800cb30 con 0x7fde000d1050 2023-07-18T03:39:12.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.878+0000 7fddf5ffb700 1 -- 172.21.15.49:0/2896116156 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8014440 con 0x7fde000d1050 2023-07-18T03:39:12.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.878+0000 7fde075ff700 1 -- 172.21.15.49:0/2896116156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde000ca770 con 0x7fde000d1050 2023-07-18T03:39:12.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.878+0000 7fde075ff700 1 -- 172.21.15.49:0/2896116156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde00152a30 con 0x7fde000d1050 2023-07-18T03:39:12.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.879+0000 7fde075ff700 1 -- 172.21.15.49:0/2896116156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde000caf70 con 0x7fde000d1050 2023-07-18T03:39:12.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.880+0000 7fddf5ffb700 1 -- 172.21.15.49:0/2896116156 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fddf8006d50 con 0x7fde000d1050 2023-07-18T03:39:12.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.881+0000 7fddf5ffb700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdde406ace0 0x7fdde406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:12.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.881+0000 7fddf5ffb700 1 -- 172.21.15.49:0/2896116156 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fddf8088f70 con 0x7fde000d1050 2023-07-18T03:39:12.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.882+0000 7fde0539b700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdde406ace0 0x7fdde406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:12.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.882+0000 7fde0539b700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdde406ace0 0x7fdde406d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009130 tx=0x7fddfc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:12.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:12.886+0000 7fddf5ffb700 1 -- 172.21.15.49:0/2896116156 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fddf80bb330 con 0x7fde000d1050 2023-07-18T03:39:13.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.093+0000 7fde075ff700 1 -- 172.21.15.49:0/2896116156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde00074350 con 0x7fde000d1050 2023-07-18T03:39:13.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.094+0000 7fddf5ffb700 1 -- 172.21.15.49:0/2896116156 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fddf8019020 con 0x7fde000d1050 2023-07-18T03:39:13.089 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:39:13.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.096+0000 7fde075ff700 1 -- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdde406ace0 msgr2=0x7fdde406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.096+0000 7fde075ff700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdde406ace0 0x7fdde406d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009130 tx=0x7fddfc008030 comp rx=0 tx=0).stop 2023-07-18T03:39:13.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.096+0000 7fde075ff700 1 -- 172.21.15.49:0/2896116156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde000d1050 msgr2=0x7fde000cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.096+0000 7fde075ff700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde000d1050 0x7fde000cf520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fddf800b990 tx=0x7fddf800bd50 comp rx=0 tx=0).stop 2023-07-18T03:39:13.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.096+0000 7fde075ff700 1 -- 172.21.15.49:0/2896116156 shutdown_connections 2023-07-18T03:39:13.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.096+0000 7fde075ff700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdde406ace0 0x7fdde406d190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.096+0000 7fde075ff700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde000d1050 0x7fde000cf520 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.096+0000 7fde075ff700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde000b4530 0x7fde001524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.096+0000 7fde075ff700 1 --2- 172.21.15.49:0/2896116156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde000b8a60 0x7fde0015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.096+0000 7fde075ff700 1 -- 172.21.15.49:0/2896116156 >> 172.21.15.49:0/2896116156 conn(0x7fde0009c1b0 msgr2=0x7fde000d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:13.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.096+0000 7fde075ff700 1 -- 172.21.15.49:0/2896116156 shutdown_connections 2023-07-18T03:39:13.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.096+0000 7fde075ff700 1 -- 172.21.15.49:0/2896116156 wait complete. 2023-07-18T03:39:13.110 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:39:13.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.323+0000 7f2602dff700 1 Processor -- start 2023-07-18T03:39:13.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.323+0000 7f2602dff700 1 -- start start 2023-07-18T03:39:13.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.324+0000 7f2602dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25fc0d0b10 0x7f25fc0d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.324+0000 7f2602dff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25fc0b0e20 0x7f25fc0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.324+0000 7f2602dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25fc0d14a0 con 0x7f25fc0b8a60 2023-07-18T03:39:13.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.324+0000 7f2602dff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25fc0b8e30 con 0x7f25fc0b0e20 2023-07-18T03:39:13.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.324+0000 7f2602dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0afaa0 con 0x7f25fc0d0b10 2023-07-18T03:39:13.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.325+0000 7f2600b9b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f25fc0b8a60 0x7f25fc0b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42562/0 (socket says 172.21.15.49:42562) 2023-07-18T03:39:13.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.325+0000 7f2600b9b700 1 -- 172.21.15.49:0/1734571349 learned_addr learned my addr 172.21.15.49:0/1734571349 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:13.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.325+0000 7f260139c700 1 --2- 172.21.15.49:0/1734571349 >> v2:172.21.15.49:3300/0 conn(0x7f25fc0b0e20 0x7f25fc0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.325+0000 7f25fbfff700 1 --2- 172.21.15.49:0/1734571349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25fc0d0b10 0x7f25fc0d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.325+0000 7f25fb7fe700 1 -- 172.21.15.49:0/1734571349 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903662367 0 0) 0x7f25f0001110 con 0x7f25fc0b8a60 2023-07-18T03:39:13.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.325+0000 7f25fb7fe700 1 -- 172.21.15.49:0/1734571349 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ec0035b0 con 0x7f25fc0b8a60 2023-07-18T03:39:13.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.325+0000 7f260139c700 1 -- 172.21.15.49:0/1734571349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25fc0d0b10 msgr2=0x7f25fc0d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:13.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.325+0000 7f260139c700 1 --2- 172.21.15.49:0/1734571349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25fc0d0b10 0x7f25fc0d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.325+0000 7f260139c700 1 -- 172.21.15.49:0/1734571349 >> v1:172.21.15.49:6789/0 conn(0x7f25fc0b8a60 legacy=0x7f25fc0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.325+0000 7f260139c700 1 -- 172.21.15.49:0/1734571349 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc0afd80 con 0x7f25fc0b0e20 2023-07-18T03:39:13.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.325+0000 7f260139c700 1 --2- 172.21.15.49:0/1734571349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25fc0b0e20 0x7f25fc0b12a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009d90 tx=0x7f25f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebca7db6c6e3921 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:13.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.326+0000 7f25fb7fe700 1 -- 172.21.15.49:0/1734571349 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f400e070 con 0x7f25fc0b0e20 2023-07-18T03:39:13.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.326+0000 7f25fb7fe700 1 -- 172.21.15.49:0/1734571349 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f400bb20 con 0x7f25fc0b0e20 2023-07-18T03:39:13.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.326+0000 7f25fb7fe700 1 -- 172.21.15.49:0/1734571349 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4012430 con 0x7f25fc0b0e20 2023-07-18T03:39:13.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.326+0000 7f2602dff700 1 -- 172.21.15.49:0/1734571349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25fc0b0e20 msgr2=0x7f25fc0b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.326+0000 7f2602dff700 1 --2- 172.21.15.49:0/1734571349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25fc0b0e20 0x7f25fc0b12a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009d90 tx=0x7f25f400b330 comp rx=0 tx=0).stop 2023-07-18T03:39:13.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.326+0000 7f2602dff700 1 -- 172.21.15.49:0/1734571349 shutdown_connections 2023-07-18T03:39:13.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.326+0000 7f2602dff700 1 --2- 172.21.15.49:0/1734571349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25fc0b0e20 0x7f25fc0b12a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.326+0000 7f2602dff700 1 --2- 172.21.15.49:0/1734571349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25fc0d0b10 0x7f25fc0d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.326+0000 7f2602dff700 1 -- 172.21.15.49:0/1734571349 >> 172.21.15.49:0/1734571349 conn(0x7f25fc09c1b0 msgr2=0x7f25fc0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:13.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.326+0000 7f2602dff700 1 -- 172.21.15.49:0/1734571349 shutdown_connections 2023-07-18T03:39:13.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.326+0000 7f2602dff700 1 -- 172.21.15.49:0/1734571349 wait complete. 2023-07-18T03:39:13.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.328+0000 7f2602dff700 1 Processor -- start 2023-07-18T03:39:13.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.328+0000 7f2602dff700 1 -- start start 2023-07-18T03:39:13.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.329+0000 7f2602dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25fc0b0e20 0x7f25fc1523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.329+0000 7f2602dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25fc0b8a60 0x7f25fc0cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.329+0000 7f2602dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25fc0d0b10 0x7f25fc0cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.329+0000 7f2602dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0ceb40 con 0x7f25fc0b8a60 2023-07-18T03:39:13.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.329+0000 7f2602dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0cc8e0 con 0x7f25fc0d0b10 2023-07-18T03:39:13.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.329+0000 7f2602dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0ccbc0 con 0x7f25fc0b0e20 2023-07-18T03:39:13.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.330+0000 7f25fbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25fc0b8a60 0x7f25fc0cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.330+0000 7f25fbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25fc0b8a60 0x7f25fc0cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54610/0 (socket says 172.21.15.49:54610) 2023-07-18T03:39:13.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.330+0000 7f25fbfff700 1 -- 172.21.15.49:0/1020839004 learned_addr learned my addr 172.21.15.49:0/1020839004 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:13.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.330+0000 7f2600b9b700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25fc0b0e20 0x7f25fc1523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.330+0000 7f260139c700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25fc0d0b10 0x7f25fc0cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.330+0000 7f25fbfff700 1 -- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25fc0b0e20 msgr2=0x7f25fc1523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.330+0000 7f25fbfff700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25fc0b0e20 0x7f25fc1523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.330+0000 7f25fbfff700 1 -- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25fc0d0b10 msgr2=0x7f25fc0cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.330+0000 7f25fbfff700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25fc0d0b10 0x7f25fc0cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.330+0000 7f25fbfff700 1 -- 172.21.15.49:0/1020839004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f4009a70 con 0x7f25fc0b8a60 2023-07-18T03:39:13.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.330+0000 7f260139c700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25fc0d0b10 0x7f25fc0cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:13.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.330+0000 7f25fbfff700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25fc0b8a60 0x7f25fc0cabe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f25e800d800 tx=0x7f25e800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:13.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.331+0000 7f25f97fa700 1 -- 172.21.15.49:0/1020839004 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e80125f0 con 0x7f25fc0b8a60 2023-07-18T03:39:13.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.331+0000 7f25f97fa700 1 -- 172.21.15.49:0/1020839004 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e8012750 con 0x7f25fc0b8a60 2023-07-18T03:39:13.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.331+0000 7f25f97fa700 1 -- 172.21.15.49:0/1020839004 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e8012b00 con 0x7f25fc0b8a60 2023-07-18T03:39:13.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.331+0000 7f2602dff700 1 -- 172.21.15.49:0/1020839004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25fc0ccea0 con 0x7f25fc0b8a60 2023-07-18T03:39:13.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.331+0000 7f2602dff700 1 -- 172.21.15.49:0/1020839004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc0ce760 con 0x7f25fc0b8a60 2023-07-18T03:39:13.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.332+0000 7f2602dff700 1 -- 172.21.15.49:0/1020839004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25fc0cd0e0 con 0x7f25fc0b8a60 2023-07-18T03:39:13.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.333+0000 7f25f97fa700 1 -- 172.21.15.49:0/1020839004 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f25e8012c60 con 0x7f25fc0b8a60 2023-07-18T03:39:13.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.334+0000 7f25f97fa700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f25e006ace0 0x7f25e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.334+0000 7f25f97fa700 1 -- 172.21.15.49:0/1020839004 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f25e8089860 con 0x7f25fc0b8a60 2023-07-18T03:39:13.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.335+0000 7f2600b9b700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f25e006ace0 0x7f25e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.335+0000 7f2600b9b700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f25e006ace0 0x7f25e006d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f25f0007fd0 tx=0x7f25f0007c40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:13.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.338+0000 7f25f97fa700 1 -- 172.21.15.49:0/1020839004 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f25e8054870 con 0x7f25fc0b8a60 2023-07-18T03:39:13.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.547+0000 7f2602dff700 1 -- 172.21.15.49:0/1020839004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f25fc074350 con 0x7f25fc0b8a60 2023-07-18T03:39:13.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.548+0000 7f25f97fa700 1 -- 172.21.15.49:0/1020839004 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f25e8057e90 con 0x7f25fc0b8a60 2023-07-18T03:39:13.543 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:39:13.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.550+0000 7f2602dff700 1 -- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f25e006ace0 msgr2=0x7f25e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.550+0000 7f2602dff700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f25e006ace0 0x7f25e006d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f25f0007fd0 tx=0x7f25f0007c40 comp rx=0 tx=0).stop 2023-07-18T03:39:13.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.550+0000 7f2602dff700 1 -- 172.21.15.49:0/1020839004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25fc0b8a60 msgr2=0x7f25fc0cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.550+0000 7f2602dff700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25fc0b8a60 0x7f25fc0cabe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f25e800d800 tx=0x7f25e800db10 comp rx=0 tx=0).stop 2023-07-18T03:39:13.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.550+0000 7f2602dff700 1 -- 172.21.15.49:0/1020839004 shutdown_connections 2023-07-18T03:39:13.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.550+0000 7f2602dff700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25fc0d0b10 0x7f25fc0cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.550+0000 7f2602dff700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25fc0b0e20 0x7f25fc1523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.550+0000 7f2602dff700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f25e006ace0 0x7f25e006d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.550+0000 7f2602dff700 1 --2- 172.21.15.49:0/1020839004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25fc0b8a60 0x7f25fc0cabe0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.550+0000 7f2602dff700 1 -- 172.21.15.49:0/1020839004 >> 172.21.15.49:0/1020839004 conn(0x7f25fc09c1b0 msgr2=0x7f25fc06a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.551+0000 7f2602dff700 1 -- 172.21.15.49:0/1020839004 shutdown_connections 2023-07-18T03:39:13.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.551+0000 7f2602dff700 1 -- 172.21.15.49:0/1020839004 wait complete. 2023-07-18T03:39:13.563 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:13.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.774+0000 7fa2c17ff700 1 Processor -- start 2023-07-18T03:39:13.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.774+0000 7fa2c17ff700 1 -- start start 2023-07-18T03:39:13.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.776+0000 7fa2c17ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc159c60 0x7fa2bc0b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.776+0000 7fa2c17ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2bc15a090 con 0x7fa2bc1500d0 2023-07-18T03:39:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.776+0000 7fa2c17ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2bc0cdf90 con 0x7fa2bc1597c0 2023-07-18T03:39:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.776+0000 7fa2c17ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc15a2f0 con 0x7fa2bc159c60 2023-07-18T03:39:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.776+0000 7fa2baffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa2bc1500d0 0x7fa2bc070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42576/0 (socket says 172.21.15.49:42576) 2023-07-18T03:39:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.776+0000 7fa2baffd700 1 -- 172.21.15.49:0/1742802162 learned_addr learned my addr 172.21.15.49:0/1742802162 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:13.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.776+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163131656 0 0) 0x7fa2bc15a090 con 0x7fa2bc1500d0 2023-07-18T03:39:13.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.776+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a40035b0 con 0x7fa2bc1500d0 2023-07-18T03:39:13.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.776+0000 7fa2bb7fe700 1 --2- 172.21.15.49:0/1742802162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc159c60 0x7fa2bc0b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3668025833 0 0) 0x7fa2a40035b0 con 0x7fa2bc1500d0 2023-07-18T03:39:13.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc159c60 msgr2=0x7fa2bc0b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:13.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 --2- 172.21.15.49:0/1742802162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc159c60 0x7fa2bc0b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 >> v1:172.21.15.107:6789/0 conn(0x7fa2bc1597c0 legacy=0x7fa2bc0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc15a5d0 con 0x7fa2bc1500d0 2023-07-18T03:39:13.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7fa2ac001670 con 0x7fa2bc1500d0 2023-07-18T03:39:13.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 >> v1:172.21.15.49:6789/0 conn(0x7fa2bc1500d0 legacy=0x7fa2bc070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 --2- 172.21.15.49:0/1742802162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2a4003a30 0x7fa2a4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 --2- 172.21.15.49:0/1742802162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4004850 0x7fa2a4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 --2- 172.21.15.49:0/1742802162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc159c60 0x7fa2a4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc15a2f0 con 0x7fa2a4003a30 2023-07-18T03:39:13.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a400baf0 con 0x7fa2a4004850 2023-07-18T03:39:13.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a400dec0 con 0x7fa2bc159c60 2023-07-18T03:39:13.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2baffd700 1 --2- 172.21.15.49:0/1742802162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2a4003a30 0x7fa2a4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2ba7fc700 1 --2- 172.21.15.49:0/1742802162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4004850 0x7fa2a4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.777+0000 7fa2bb7fe700 1 --2- 172.21.15.49:0/1742802162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc159c60 0x7fa2a4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.778+0000 7fa2baffd700 1 -- 172.21.15.49:0/1742802162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc159c60 msgr2=0x7fa2a4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:13.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.778+0000 7fa2baffd700 1 --2- 172.21.15.49:0/1742802162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc159c60 0x7fa2a4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.778+0000 7fa2baffd700 1 -- 172.21.15.49:0/1742802162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4004850 msgr2=0x7fa2a4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:13.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.778+0000 7fa2baffd700 1 --2- 172.21.15.49:0/1742802162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4004850 0x7fa2a4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.778+0000 7fa2baffd700 1 -- 172.21.15.49:0/1742802162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a40035b0 con 0x7fa2a4003a30 2023-07-18T03:39:13.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.778+0000 7fa2baffd700 1 --2- 172.21.15.49:0/1742802162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2a4003a30 0x7fa2a4003e40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00c160 tx=0x7fa2a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165b5875b4ac4cb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:13.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.778+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac007b70 con 0x7fa2a4003a30 2023-07-18T03:39:13.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.778+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ac00cd20 con 0x7fa2a4003a30 2023-07-18T03:39:13.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.778+0000 7fa2b9ffb700 1 -- 172.21.15.49:0/1742802162 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac005ba0 con 0x7fa2a4003a30 2023-07-18T03:39:13.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.779+0000 7fa2c17ff700 1 -- 172.21.15.49:0/1742802162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2a4003a30 msgr2=0x7fa2a4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.779+0000 7fa2c17ff700 1 --2- 172.21.15.49:0/1742802162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2a4003a30 0x7fa2a4003e40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00c160 tx=0x7fa2a4005880 comp rx=0 tx=0).stop 2023-07-18T03:39:13.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.779+0000 7fa2bb7fe700 1 -- 172.21.15.49:0/1742802162 reap_dead start 2023-07-18T03:39:13.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.779+0000 7fa2c17ff700 1 -- 172.21.15.49:0/1742802162 shutdown_connections 2023-07-18T03:39:13.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.779+0000 7fa2c17ff700 1 -- 172.21.15.49:0/1742802162 >> 172.21.15.49:0/1742802162 conn(0x7fa2bc069f30 msgr2=0x7fa2bc06a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:13.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.779+0000 7fa2c17ff700 1 -- 172.21.15.49:0/1742802162 shutdown_connections 2023-07-18T03:39:13.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.779+0000 7fa2c17ff700 1 -- 172.21.15.49:0/1742802162 wait complete. 2023-07-18T03:39:13.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.781+0000 7fa2c17ff700 1 Processor -- start 2023-07-18T03:39:13.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.781+0000 7fa2c17ff700 1 -- start start 2023-07-18T03:39:13.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.782+0000 7fa2c17ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4003a30 0x7fa2bc1f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.782+0000 7fa2c17ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc14a300 0x7fa2bc1475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.782+0000 7fa2c17ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc147b10 0x7fa2bc136c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.782+0000 7fa2c17ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc149ee0 con 0x7fa2bc14a300 2023-07-18T03:39:13.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.782+0000 7fa2c17ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc158cd0 con 0x7fa2a4003a30 2023-07-18T03:39:13.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.782+0000 7fa2c17ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc147f80 con 0x7fa2bc147b10 2023-07-18T03:39:13.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.782+0000 7fa2ba7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc14a300 0x7fa2bc1475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.782+0000 7fa2ba7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc14a300 0x7fa2bc1475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54624/0 (socket says 172.21.15.49:54624) 2023-07-18T03:39:13.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.782+0000 7fa2ba7fc700 1 -- 172.21.15.49:0/3401951851 learned_addr learned my addr 172.21.15.49:0/3401951851 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:13.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.782+0000 7fa2baffd700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4003a30 0x7fa2bc1f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.783+0000 7fa2bb7fe700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc147b10 0x7fa2bc136c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.783+0000 7fa2ba7fc700 1 -- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc147b10 msgr2=0x7fa2bc136c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.783+0000 7fa2ba7fc700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc147b10 0x7fa2bc136c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.783+0000 7fa2ba7fc700 1 -- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4003a30 msgr2=0x7fa2bc1f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:13.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.783+0000 7fa2ba7fc700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4003a30 0x7fa2bc1f2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:13.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.783+0000 7fa2ba7fc700 1 -- 172.21.15.49:0/3401951851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc1f27c0 con 0x7fa2bc14a300 2023-07-18T03:39:13.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.783+0000 7fa2ba7fc700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc14a300 0x7fa2bc1475d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000f850 tx=0x7fa2b000fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:13.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.783+0000 7fa2a37fe700 1 -- 172.21.15.49:0/3401951851 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b00050d0 con 0x7fa2bc14a300 2023-07-18T03:39:13.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.783+0000 7fa2a37fe700 1 -- 172.21.15.49:0/3401951851 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b0005230 con 0x7fa2bc14a300 2023-07-18T03:39:13.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.783+0000 7fa2a37fe700 1 -- 172.21.15.49:0/3401951851 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0003680 con 0x7fa2bc14a300 2023-07-18T03:39:13.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.783+0000 7fa2c17ff700 1 -- 172.21.15.49:0/3401951851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2bc1379b0 con 0x7fa2bc14a300 2023-07-18T03:39:13.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.783+0000 7fa2c17ff700 1 -- 172.21.15.49:0/3401951851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2bc149b00 con 0x7fa2bc14a300 2023-07-18T03:39:13.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.784+0000 7fa2c17ff700 1 -- 172.21.15.49:0/3401951851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2bc04e280 con 0x7fa2bc14a300 2023-07-18T03:39:13.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.785+0000 7fa2a37fe700 1 -- 172.21.15.49:0/3401951851 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa2b00037e0 con 0x7fa2bc14a300 2023-07-18T03:39:13.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.786+0000 7fa2a37fe700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa2a806ace0 0x7fa2a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:13.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.786+0000 7fa2a37fe700 1 -- 172.21.15.49:0/3401951851 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa2b00891d0 con 0x7fa2bc14a300 2023-07-18T03:39:13.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.787+0000 7fa2baffd700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa2a806ace0 0x7fa2a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:13.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.788+0000 7fa2baffd700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa2a806ace0 0x7fa2a806d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac015040 tx=0x7fa2ac007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:13.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:13.791+0000 7fa2a37fe700 1 -- 172.21.15.49:0/3401951851 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa2b0054310 con 0x7fa2bc14a300 2023-07-18T03:39:13.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.004+0000 7fa2c17ff700 1 -- 172.21.15.49:0/3401951851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2bc0b1560 con 0x7fa2bc14a300 2023-07-18T03:39:14.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.005+0000 7fa2a37fe700 1 -- 172.21.15.49:0/3401951851 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18185 (secure 0 0 0) 0x7fa2b0057930 con 0x7fa2bc14a300 2023-07-18T03:39:14.000 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:14.001 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:37.510006+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.008+0000 7fa2c17ff700 1 -- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa2a806ace0 msgr2=0x7fa2a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.008+0000 7fa2c17ff700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa2a806ace0 0x7fa2a806d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac015040 tx=0x7fa2ac007000 comp rx=0 tx=0).stop 2023-07-18T03:39:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.008+0000 7fa2c17ff700 1 -- 172.21.15.49:0/3401951851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc14a300 msgr2=0x7fa2bc1475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.008+0000 7fa2c17ff700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc14a300 0x7fa2bc1475d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000f850 tx=0x7fa2b000fc10 comp rx=0 tx=0).stop 2023-07-18T03:39:14.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.008+0000 7fa2c17ff700 1 -- 172.21.15.49:0/3401951851 shutdown_connections 2023-07-18T03:39:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.008+0000 7fa2c17ff700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2bc147b10 0x7fa2bc136c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.008+0000 7fa2c17ff700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4003a30 0x7fa2bc1f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.008+0000 7fa2c17ff700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa2a806ace0 0x7fa2a806d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.008+0000 7fa2c17ff700 1 --2- 172.21.15.49:0/3401951851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc14a300 0x7fa2bc1475d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.008+0000 7fa2c17ff700 1 -- 172.21.15.49:0/3401951851 >> 172.21.15.49:0/3401951851 conn(0x7fa2bc069f30 msgr2=0x7fa2bc06b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.008+0000 7fa2c17ff700 1 -- 172.21.15.49:0/3401951851 shutdown_connections 2023-07-18T03:39:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.008+0000 7fa2c17ff700 1 -- 172.21.15.49:0/3401951851 wait complete. 2023-07-18T03:39:14.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:38:33.773236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:38:35.023692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:39:14.022 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:39:14.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.235+0000 7f77d6bff700 1 Processor -- start 2023-07-18T03:39:14.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.235+0000 7f77d6bff700 1 -- start start 2023-07-18T03:39:14.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.236+0000 7f77d6bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f77d00b8a60 0x7f77d00b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:14.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.236+0000 7f77d6bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d00b8e30 con 0x7f77d00b4530 2023-07-18T03:39:14.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.236+0000 7f77d6bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d00afaa0 con 0x7f77d00d1220 2023-07-18T03:39:14.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.236+0000 7f77d6bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d00afd80 con 0x7f77d00b8a60 2023-07-18T03:39:14.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.236+0000 7f77d499b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f77d00b8a60 0x7f77d00b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:14.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.236+0000 7f77cffff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f77d00b4530 0x7f77d00d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42586/0 (socket says 172.21.15.49:42586) 2023-07-18T03:39:14.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.236+0000 7f77cffff700 1 -- 172.21.15.49:0/753338851 learned_addr learned my addr 172.21.15.49:0/753338851 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:14.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.237+0000 7f77d499b700 1 -- 172.21.15.49:0/753338851 >> v1:172.21.15.107:6789/0 conn(0x7f77d00d1220 legacy=0x7f77d00b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:14.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.237+0000 7f77d499b700 1 -- 172.21.15.49:0/753338851 >> v1:172.21.15.49:6789/0 conn(0x7f77d00b4530 legacy=0x7f77d00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.237+0000 7f77d499b700 1 -- 172.21.15.49:0/753338851 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00b0060 con 0x7f77d00b8a60 2023-07-18T03:39:14.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.237+0000 7f77d499b700 1 --2- 172.21.15.49:0/753338851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00b8a60 0x7f77d00b3ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009840 tx=0x7f77c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd2ad42c38fcdff server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:14.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.237+0000 7f77cf7fe700 1 -- 172.21.15.49:0/753338851 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4004230 con 0x7f77d00b8a60 2023-07-18T03:39:14.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.237+0000 7f77cf7fe700 1 -- 172.21.15.49:0/753338851 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c4004390 con 0x7f77d00b8a60 2023-07-18T03:39:14.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.237+0000 7f77cf7fe700 1 -- 172.21.15.49:0/753338851 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c40036a0 con 0x7f77d00b8a60 2023-07-18T03:39:14.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.238+0000 7f77d6bff700 1 -- 172.21.15.49:0/753338851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00b8a60 msgr2=0x7f77d00b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.238+0000 7f77d6bff700 1 --2- 172.21.15.49:0/753338851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00b8a60 0x7f77d00b3ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009840 tx=0x7f77c4009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:14.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.238+0000 7f77d6bff700 1 -- 172.21.15.49:0/753338851 shutdown_connections 2023-07-18T03:39:14.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.238+0000 7f77d6bff700 1 --2- 172.21.15.49:0/753338851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00b8a60 0x7f77d00b3ff0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.238+0000 7f77d6bff700 1 -- 172.21.15.49:0/753338851 >> 172.21.15.49:0/753338851 conn(0x7f77d009c1b0 msgr2=0x7f77d00d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:14.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.238+0000 7f77d6bff700 1 -- 172.21.15.49:0/753338851 shutdown_connections 2023-07-18T03:39:14.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.238+0000 7f77d6bff700 1 -- 172.21.15.49:0/753338851 wait complete. 2023-07-18T03:39:14.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.240+0000 7f77d6bff700 1 Processor -- start 2023-07-18T03:39:14.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.240+0000 7f77d6bff700 1 -- start start 2023-07-18T03:39:14.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.241+0000 7f77d6bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d00b4530 0x7f77d00c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:14.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.241+0000 7f77d6bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00b8a60 0x7f77d00cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:14.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.241+0000 7f77d6bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00d1220 0x7f77d015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:14.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.241+0000 7f77d6bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00cd950 con 0x7f77d00b8a60 2023-07-18T03:39:14.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.241+0000 7f77d6bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00cdc30 con 0x7f77d00b4530 2023-07-18T03:39:14.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.241+0000 7f77d6bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00cdf10 con 0x7f77d00d1220 2023-07-18T03:39:14.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.241+0000 7f77cffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00b8a60 0x7f77d00cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:14.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.241+0000 7f77cffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00b8a60 0x7f77d00cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54642/0 (socket says 172.21.15.49:54642) 2023-07-18T03:39:14.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.241+0000 7f77cffff700 1 -- 172.21.15.49:0/1403201513 learned_addr learned my addr 172.21.15.49:0/1403201513 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:14.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77d499b700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d00b4530 0x7f77d00c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:14.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77d519c700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00d1220 0x7f77d015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:14.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77cffff700 1 -- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00d1220 msgr2=0x7f77d015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77cffff700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00d1220 0x7f77d015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77cffff700 1 -- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d00b4530 msgr2=0x7f77d00c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77cffff700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d00b4530 0x7f77d00c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77cffff700 1 -- 172.21.15.49:0/1403201513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c40094a0 con 0x7f77d00b8a60 2023-07-18T03:39:14.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77d519c700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00d1220 0x7f77d015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:14.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77cffff700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00b8a60 0x7f77d00cf320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00c490 tx=0x7f77bc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:14.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77cd7fa700 1 -- 172.21.15.49:0/1403201513 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc005d70 con 0x7f77d00b8a60 2023-07-18T03:39:14.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77cd7fa700 1 -- 172.21.15.49:0/1403201513 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc0070d0 con 0x7f77d00b8a60 2023-07-18T03:39:14.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77d6bff700 1 -- 172.21.15.49:0/1403201513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d00ce1f0 con 0x7f77d00b8a60 2023-07-18T03:39:14.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77cd7fa700 1 -- 172.21.15.49:0/1403201513 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc007460 con 0x7f77d00b8a60 2023-07-18T03:39:14.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.242+0000 7f77d6bff700 1 -- 172.21.15.49:0/1403201513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d00ce9f0 con 0x7f77d00b8a60 2023-07-18T03:39:14.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.243+0000 7f77d6bff700 1 -- 172.21.15.49:0/1403201513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d00ceb80 con 0x7f77d00b8a60 2023-07-18T03:39:14.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.244+0000 7f77cd7fa700 1 -- 172.21.15.49:0/1403201513 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f77bc0075c0 con 0x7f77d00b8a60 2023-07-18T03:39:14.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.246+0000 7f77cd7fa700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f77b406ace0 0x7f77b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:14.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.246+0000 7f77cd7fa700 1 -- 172.21.15.49:0/1403201513 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77bc0889c0 con 0x7f77d00b8a60 2023-07-18T03:39:14.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.246+0000 7f77d499b700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f77b406ace0 0x7f77b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:14.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.247+0000 7f77d499b700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f77b406ace0 0x7f77b406d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f77c4000c00 tx=0x7f77c401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:14.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.250+0000 7f77cd7fa700 1 -- 172.21.15.49:0/1403201513 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f77d00ceb80 con 0x7f77d00b8a60 2023-07-18T03:39:14.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.454+0000 7f77d6bff700 1 -- 172.21.15.49:0/1403201513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f77d00ce710 con 0x7f77d00b8a60 2023-07-18T03:39:14.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.455+0000 7f77cd7fa700 1 -- 172.21.15.49:0/1403201513 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f77d00ce710 con 0x7f77d00b8a60 2023-07-18T03:39:14.450 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:39:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.457+0000 7f77d6bff700 1 -- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f77b406ace0 msgr2=0x7f77b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.457+0000 7f77d6bff700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f77b406ace0 0x7f77b406d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f77c4000c00 tx=0x7f77c401c040 comp rx=0 tx=0).stop 2023-07-18T03:39:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.457+0000 7f77d6bff700 1 -- 172.21.15.49:0/1403201513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00b8a60 msgr2=0x7f77d00cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.457+0000 7f77d6bff700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00b8a60 0x7f77d00cf320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00c490 tx=0x7f77bc00c850 comp rx=0 tx=0).stop 2023-07-18T03:39:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.457+0000 7f77d6bff700 1 -- 172.21.15.49:0/1403201513 shutdown_connections 2023-07-18T03:39:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.457+0000 7f77d6bff700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00d1220 0x7f77d015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.457+0000 7f77d6bff700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d00b4530 0x7f77d00c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.457+0000 7f77d6bff700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f77b406ace0 0x7f77b406d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.457+0000 7f77d6bff700 1 --2- 172.21.15.49:0/1403201513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00b8a60 0x7f77d00cf320 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.457+0000 7f77d6bff700 1 -- 172.21.15.49:0/1403201513 >> 172.21.15.49:0/1403201513 conn(0x7f77d009c1b0 msgr2=0x7f77d00b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.457+0000 7f77d6bff700 1 -- 172.21.15.49:0/1403201513 shutdown_connections 2023-07-18T03:39:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.457+0000 7f77d6bff700 1 -- 172.21.15.49:0/1403201513 wait complete. 2023-07-18T03:39:14.470 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:39:14.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.682+0000 7f1193bff700 1 Processor -- start 2023-07-18T03:39:14.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.683+0000 7f1193bff700 1 -- start start 2023-07-18T03:39:14.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.684+0000 7f1193bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f118c0d0b10 0x7f118c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:14.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.684+0000 7f1193bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c0b8e30 con 0x7f118c0b0e20 2023-07-18T03:39:14.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.684+0000 7f1193bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c0afaa0 con 0x7f118c0b8a60 2023-07-18T03:39:14.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.684+0000 7f1193bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118c0d14d0 con 0x7f118c0d0b10 2023-07-18T03:39:14.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.684+0000 7f119219c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f118c0b0e20 0x7f118c0b1260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42600/0 (socket says 172.21.15.49:42600) 2023-07-18T03:39:14.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.684+0000 7f119219c700 1 -- 172.21.15.49:0/983969045 learned_addr learned my addr 172.21.15.49:0/983969045 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:14.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.684+0000 7f119119a700 1 --2- 172.21.15.49:0/983969045 >> v2:172.21.15.107:3300/0 conn(0x7f118c0d0b10 0x7f118c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:14.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.684+0000 7f1190999700 1 -- 172.21.15.49:0/983969045 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226136921 0 0) 0x7f118c0b8e30 con 0x7f118c0b0e20 2023-07-18T03:39:14.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.685+0000 7f1190999700 1 -- 172.21.15.49:0/983969045 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11780035b0 con 0x7f118c0b0e20 2023-07-18T03:39:14.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.685+0000 7f1190999700 1 -- 172.21.15.49:0/983969045 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079154995 0 0) 0x7f118c0afaa0 con 0x7f118c0b8a60 2023-07-18T03:39:14.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.685+0000 7f1190999700 1 -- 172.21.15.49:0/983969045 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f118c0b8e30 con 0x7f118c0b8a60 2023-07-18T03:39:14.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.685+0000 7f119119a700 1 -- 172.21.15.49:0/983969045 >> v1:172.21.15.107:6789/0 conn(0x7f118c0b8a60 legacy=0x7f118c0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.685+0000 7f119119a700 1 -- 172.21.15.49:0/983969045 >> v1:172.21.15.49:6789/0 conn(0x7f118c0b0e20 legacy=0x7f118c0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.685+0000 7f119119a700 1 -- 172.21.15.49:0/983969045 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c0afd80 con 0x7f118c0d0b10 2023-07-18T03:39:14.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.685+0000 7f119119a700 1 --2- 172.21.15.49:0/983969045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f118c0d0b10 0x7f118c0d0f90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f117c009840 tx=0x7f117c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b70f321db142fe server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:14.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.685+0000 7f1190999700 1 -- 172.21.15.49:0/983969045 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c011830 con 0x7f118c0d0b10 2023-07-18T03:39:14.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.685+0000 7f1190999700 1 -- 172.21.15.49:0/983969045 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c011990 con 0x7f118c0d0b10 2023-07-18T03:39:14.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.685+0000 7f1190999700 1 -- 172.21.15.49:0/983969045 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c011d20 con 0x7f118c0d0b10 2023-07-18T03:39:14.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.686+0000 7f1193bff700 1 -- 172.21.15.49:0/983969045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f118c0d0b10 msgr2=0x7f118c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.686+0000 7f1193bff700 1 --2- 172.21.15.49:0/983969045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f118c0d0b10 0x7f118c0d0f90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f117c009840 tx=0x7f117c009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:14.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.686+0000 7f1193bff700 1 -- 172.21.15.49:0/983969045 shutdown_connections 2023-07-18T03:39:14.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.686+0000 7f1193bff700 1 --2- 172.21.15.49:0/983969045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f118c0d0b10 0x7f118c0d0f90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.686+0000 7f1193bff700 1 -- 172.21.15.49:0/983969045 >> 172.21.15.49:0/983969045 conn(0x7f118c09c1b0 msgr2=0x7f118c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:14.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.686+0000 7f1193bff700 1 -- 172.21.15.49:0/983969045 shutdown_connections 2023-07-18T03:39:14.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.686+0000 7f1193bff700 1 -- 172.21.15.49:0/983969045 wait complete. 2023-07-18T03:39:14.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.689+0000 7f1193bff700 1 Processor -- start 2023-07-18T03:39:14.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.689+0000 7f1193bff700 1 -- start start 2023-07-18T03:39:14.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.690+0000 7f1193bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f118c0b0e20 0x7f118c06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:14.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.690+0000 7f1193bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f118c0b8a60 0x7f118c0ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:14.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.690+0000 7f1193bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f118c0d0b10 0x7f118c14f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:14.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.690+0000 7f1193bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0cdc00 con 0x7f118c0b0e20 2023-07-18T03:39:14.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.690+0000 7f1193bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0cdea0 con 0x7f118c0b8a60 2023-07-18T03:39:14.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.690+0000 7f1193bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0ce180 con 0x7f118c0d0b10 2023-07-18T03:39:14.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.690+0000 7f119199b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f118c0b0e20 0x7f118c06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:14.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.690+0000 7f119199b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f118c0b0e20 0x7f118c06b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54652/0 (socket says 172.21.15.49:54652) 2023-07-18T03:39:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.690+0000 7f119199b700 1 -- 172.21.15.49:0/3647908221 learned_addr learned my addr 172.21.15.49:0/3647908221 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f119219c700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f118c0d0b10 0x7f118c14f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f119119a700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f118c0b8a60 0x7f118c0ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f119199b700 1 -- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f118c0d0b10 msgr2=0x7f118c14f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f119199b700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f118c0d0b10 0x7f118c14f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f119199b700 1 -- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f118c0b8a60 msgr2=0x7f118c0ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f119199b700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f118c0b8a60 0x7f118c0ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f119199b700 1 -- 172.21.15.49:0/3647908221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c0094a0 con 0x7f118c0b0e20 2023-07-18T03:39:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f119119a700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f118c0b8a60 0x7f118c0ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f119199b700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f118c0b0e20 0x7f118c06b1f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f118800c520 tx=0x7f118800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:14.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f11827fc700 1 -- 172.21.15.49:0/3647908221 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11880052d0 con 0x7f118c0b0e20 2023-07-18T03:39:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f11827fc700 1 -- 172.21.15.49:0/3647908221 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1188005430 con 0x7f118c0b0e20 2023-07-18T03:39:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f1193bff700 1 -- 172.21.15.49:0/3647908221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c0ce460 con 0x7f118c0b0e20 2023-07-18T03:39:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f11827fc700 1 -- 172.21.15.49:0/3647908221 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11880038c0 con 0x7f118c0b0e20 2023-07-18T03:39:14.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.691+0000 7f1193bff700 1 -- 172.21.15.49:0/3647908221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c0cd570 con 0x7f118c0b0e20 2023-07-18T03:39:14.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.692+0000 7f1193bff700 1 -- 172.21.15.49:0/3647908221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c0ce6a0 con 0x7f118c0b0e20 2023-07-18T03:39:14.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.693+0000 7f11827fc700 1 -- 172.21.15.49:0/3647908221 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1188003a20 con 0x7f118c0b0e20 2023-07-18T03:39:14.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.694+0000 7f11827fc700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f117006ace0 0x7f117006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:14.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.695+0000 7f11827fc700 1 -- 172.21.15.49:0/3647908221 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1188088b40 con 0x7f118c0b0e20 2023-07-18T03:39:14.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.695+0000 7f119119a700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f117006ace0 0x7f117006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:14.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.696+0000 7f119119a700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f117006ace0 0x7f117006d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f118c0b5460 tx=0x7f117c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:14.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.699+0000 7f11827fc700 1 -- 172.21.15.49:0/3647908221 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1188053b50 con 0x7f118c0b0e20 2023-07-18T03:39:14.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.907+0000 7f1193bff700 1 -- 172.21.15.49:0/3647908221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f118c074350 con 0x7f118c0b0e20 2023-07-18T03:39:14.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.909+0000 7f11827fc700 1 -- 172.21.15.49:0/3647908221 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1188017020 con 0x7f118c0b0e20 2023-07-18T03:39:14.903 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:39:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.911+0000 7f1193bff700 1 -- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f117006ace0 msgr2=0x7f117006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.911+0000 7f1193bff700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f117006ace0 0x7f117006d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f118c0b5460 tx=0x7f117c01c040 comp rx=0 tx=0).stop 2023-07-18T03:39:14.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.911+0000 7f1193bff700 1 -- 172.21.15.49:0/3647908221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f118c0b0e20 msgr2=0x7f118c06b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:14.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.911+0000 7f1193bff700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f118c0b0e20 0x7f118c06b1f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f118800c520 tx=0x7f118800c8e0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.911+0000 7f1193bff700 1 -- 172.21.15.49:0/3647908221 shutdown_connections 2023-07-18T03:39:14.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.911+0000 7f1193bff700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f118c0d0b10 0x7f118c14f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.911+0000 7f1193bff700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f117006ace0 0x7f117006d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.911+0000 7f1193bff700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f118c0b0e20 0x7f118c06b1f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.911+0000 7f1193bff700 1 --2- 172.21.15.49:0/3647908221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f118c0b8a60 0x7f118c0ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:14.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.911+0000 7f1193bff700 1 -- 172.21.15.49:0/3647908221 >> 172.21.15.49:0/3647908221 conn(0x7f118c09c1b0 msgr2=0x7f118c0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:14.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.912+0000 7f1193bff700 1 -- 172.21.15.49:0/3647908221 shutdown_connections 2023-07-18T03:39:14.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:14.912+0000 7f1193bff700 1 -- 172.21.15.49:0/3647908221 wait complete. 2023-07-18T03:39:14.925 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:39:15.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.137+0000 7f96619ff700 1 Processor -- start 2023-07-18T03:39:15.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.137+0000 7f96619ff700 1 -- start start 2023-07-18T03:39:15.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.138+0000 7f96619ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f965c0b8a60 0x7f965c0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:15.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.138+0000 7f96619ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f965c0d1220 0x7f965c0b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:15.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.138+0000 7f96619ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965c0b8e30 con 0x7f965c0b4530 2023-07-18T03:39:15.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.138+0000 7f96619ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965c0afaa0 con 0x7f965c0b8a60 2023-07-18T03:39:15.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.138+0000 7f96619ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c0afd80 con 0x7f965c0d1220 2023-07-18T03:39:15.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.139+0000 7f965affd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f965c0b8a60 0x7f965c0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:15.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.139+0000 7f965affd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f965c0b8a60 0x7f965c0b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54656/0 (socket says 172.21.15.49:54656) 2023-07-18T03:39:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.139+0000 7f965affd700 1 -- 172.21.15.49:0/2792363120 learned_addr learned my addr 172.21.15.49:0/2792363120 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.139+0000 7f965a7fc700 1 --1- 172.21.15.49:0/2792363120 >> v1:172.21.15.49:6789/0 conn(0x7f965c0b4530 0x7f965c0d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42608/0 (socket says 172.21.15.49:42608) 2023-07-18T03:39:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.139+0000 7f965b7fe700 1 --2- 172.21.15.49:0/2792363120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f965c0d1220 0x7f965c0b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.139+0000 7f965affd700 1 -- 172.21.15.49:0/2792363120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f965c0d1220 msgr2=0x7f965c0b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.139+0000 7f965affd700 1 --2- 172.21.15.49:0/2792363120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f965c0d1220 0x7f965c0b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.139+0000 7f965affd700 1 -- 172.21.15.49:0/2792363120 >> v1:172.21.15.49:6789/0 conn(0x7f965c0b4530 legacy=0x7f965c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.139+0000 7f965affd700 1 -- 172.21.15.49:0/2792363120 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c0b0060 con 0x7f965c0b8a60 2023-07-18T03:39:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.139+0000 7f965b7fe700 1 --2- 172.21.15.49:0/2792363120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f965c0d1220 0x7f965c0b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.139+0000 7f965affd700 1 --2- 172.21.15.49:0/2792363120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f965c0b8a60 0x7f965c0b3ff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f964c0098f0 tx=0x7f964c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce88041d4d73862 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.140+0000 7f9659ffb700 1 -- 172.21.15.49:0/2792363120 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c00b8a0 con 0x7f965c0b8a60 2023-07-18T03:39:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.140+0000 7f9659ffb700 1 -- 172.21.15.49:0/2792363120 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c00ba00 con 0x7f965c0b8a60 2023-07-18T03:39:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.140+0000 7f9659ffb700 1 -- 172.21.15.49:0/2792363120 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c00bd90 con 0x7f965c0b8a60 2023-07-18T03:39:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.140+0000 7f96619ff700 1 -- 172.21.15.49:0/2792363120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f965c0b8a60 msgr2=0x7f965c0b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.140+0000 7f96619ff700 1 --2- 172.21.15.49:0/2792363120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f965c0b8a60 0x7f965c0b3ff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f964c0098f0 tx=0x7f964c009c00 comp rx=0 tx=0).stop 2023-07-18T03:39:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.140+0000 7f96619ff700 1 -- 172.21.15.49:0/2792363120 shutdown_connections 2023-07-18T03:39:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.140+0000 7f96619ff700 1 --2- 172.21.15.49:0/2792363120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f965c0d1220 0x7f965c0b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.140+0000 7f96619ff700 1 --2- 172.21.15.49:0/2792363120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f965c0b8a60 0x7f965c0b3ff0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.140+0000 7f96619ff700 1 -- 172.21.15.49:0/2792363120 >> 172.21.15.49:0/2792363120 conn(0x7f965c09c1b0 msgr2=0x7f965c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.140+0000 7f96619ff700 1 -- 172.21.15.49:0/2792363120 shutdown_connections 2023-07-18T03:39:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.140+0000 7f96619ff700 1 -- 172.21.15.49:0/2792363120 wait complete. 2023-07-18T03:39:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.142+0000 7f96619ff700 1 Processor -- start 2023-07-18T03:39:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.142+0000 7f96619ff700 1 -- start start 2023-07-18T03:39:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.143+0000 7f96619ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f965c0b4530 0x7f965c15e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:15.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.143+0000 7f96619ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f965c0b8a60 0x7f965c0ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:15.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.143+0000 7f96619ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f965c0d1220 0x7f965c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:15.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.143+0000 7f96619ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c0cff20 con 0x7f965c0b4530 2023-07-18T03:39:15.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.143+0000 7f96619ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c15dbc0 con 0x7f965c0b8a60 2023-07-18T03:39:15.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.143+0000 7f96619ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c15dea0 con 0x7f965c0d1220 2023-07-18T03:39:15.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.144+0000 7f965affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f965c0b4530 0x7f965c15e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:15.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.144+0000 7f965affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f965c0b4530 0x7f965c15e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54664/0 (socket says 172.21.15.49:54664) 2023-07-18T03:39:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.144+0000 7f965affd700 1 -- 172.21.15.49:0/3519829647 learned_addr learned my addr 172.21.15.49:0/3519829647 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.144+0000 7f965affd700 1 -- 172.21.15.49:0/3519829647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f965c0d1220 msgr2=0x7f965c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.144+0000 7f965affd700 1 --2- 172.21.15.49:0/3519829647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f965c0d1220 0x7f965c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.144+0000 7f965affd700 1 -- 172.21.15.49:0/3519829647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f965c0b8a60 msgr2=0x7f965c0ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.144+0000 7f965affd700 1 --2- 172.21.15.49:0/3519829647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f965c0b8a60 0x7f965c0ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.144+0000 7f965affd700 1 -- 172.21.15.49:0/3519829647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c009550 con 0x7f965c0b4530 2023-07-18T03:39:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.144+0000 7f965affd700 1 --2- 172.21.15.49:0/3519829647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f965c0b4530 0x7f965c15e6a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f964c000c00 tx=0x7f964c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.144+0000 7f9643fff700 1 -- 172.21.15.49:0/3519829647 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c004570 con 0x7f965c0b4530 2023-07-18T03:39:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.144+0000 7f9643fff700 1 -- 172.21.15.49:0/3519829647 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c01f070 con 0x7f965c0b4530 2023-07-18T03:39:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.145+0000 7f96619ff700 1 -- 172.21.15.49:0/3519829647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965c15e180 con 0x7f965c0b4530 2023-07-18T03:39:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.145+0000 7f9643fff700 1 -- 172.21.15.49:0/3519829647 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0038e0 con 0x7f965c0b4530 2023-07-18T03:39:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.145+0000 7f96619ff700 1 -- 172.21.15.49:0/3519829647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965c0cc770 con 0x7f965c0b4530 2023-07-18T03:39:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.146+0000 7f96619ff700 1 -- 172.21.15.49:0/3519829647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965c074350 con 0x7f965c0b4530 2023-07-18T03:39:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.147+0000 7f9643fff700 1 -- 172.21.15.49:0/3519829647 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f964c021030 con 0x7f965c0b4530 2023-07-18T03:39:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.148+0000 7f9643fff700 1 --2- 172.21.15.49:0/3519829647 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f964806ace0 0x7f964806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.148+0000 7f9643fff700 1 -- 172.21.15.49:0/3519829647 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f964c08aaa0 con 0x7f965c0b4530 2023-07-18T03:39:15.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.148+0000 7f965a7fc700 1 --2- 172.21.15.49:0/3519829647 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f964806ace0 0x7f964806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:15.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.149+0000 7f965a7fc700 1 --2- 172.21.15.49:0/3519829647 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f964806ace0 0x7f964806d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f965c06be00 tx=0x7f9650007a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:15.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.152+0000 7f9643fff700 1 -- 172.21.15.49:0/3519829647 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f964c055ab0 con 0x7f965c0b4530 2023-07-18T03:39:15.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.358+0000 7f96619ff700 1 -- 172.21.15.49:0/3519829647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f965c0d19c0 con 0x7f965c0b4530 2023-07-18T03:39:15.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.359+0000 7f9643fff700 1 -- 172.21.15.49:0/3519829647 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f964c0590d0 con 0x7f965c0b4530 2023-07-18T03:39:15.354 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:39:15.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.361+0000 7f96619ff700 1 -- 172.21.15.49:0/3519829647 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f964806ace0 msgr2=0x7f964806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:15.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.361+0000 7f96619ff700 1 --2- 172.21.15.49:0/3519829647 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f964806ace0 0x7f964806d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f965c06be00 tx=0x7f9650007a50 comp rx=0 tx=0).stop 2023-07-18T03:39:15.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.361+0000 7f96619ff700 1 -- 172.21.15.49:0/3519829647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f965c0b4530 msgr2=0x7f965c15e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:15.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.361+0000 7f96619ff700 1 --2- 172.21.15.49:0/3519829647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f965c0b4530 0x7f965c15e6a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f964c000c00 tx=0x7f964c004750 comp rx=0 tx=0).stop 2023-07-18T03:39:15.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.361+0000 7f96619ff700 1 -- 172.21.15.49:0/3519829647 shutdown_connections 2023-07-18T03:39:15.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.361+0000 7f96619ff700 1 --2- 172.21.15.49:0/3519829647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f965c0d1220 0x7f965c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.361+0000 7f96619ff700 1 --2- 172.21.15.49:0/3519829647 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f964806ace0 0x7f964806d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.361+0000 7f96619ff700 1 --2- 172.21.15.49:0/3519829647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f965c0b4530 0x7f965c15e6a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.361+0000 7f96619ff700 1 --2- 172.21.15.49:0/3519829647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f965c0b8a60 0x7f965c0ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.361+0000 7f96619ff700 1 -- 172.21.15.49:0/3519829647 >> 172.21.15.49:0/3519829647 conn(0x7f965c09c1b0 msgr2=0x7f965c0b8070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:15.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.361+0000 7f96619ff700 1 -- 172.21.15.49:0/3519829647 shutdown_connections 2023-07-18T03:39:15.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.361+0000 7f96619ff700 1 -- 172.21.15.49:0/3519829647 wait complete. 2023-07-18T03:39:15.374 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:39:15.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.590+0000 7fe28a5ff700 1 Processor -- start 2023-07-18T03:39:15.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.590+0000 7fe28a5ff700 1 -- start start 2023-07-18T03:39:15.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.591+0000 7fe28a5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe2840b8a60 0x7fe2840b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.591+0000 7fe28a5ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2840d1220 0x7fe2840b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.591+0000 7fe28a5ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2840b8e30 con 0x7fe2840b4530 2023-07-18T03:39:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.591+0000 7fe28a5ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2840afaa0 con 0x7fe2840d1220 2023-07-18T03:39:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.591+0000 7fe28a5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2840afd80 con 0x7fe2840b8a60 2023-07-18T03:39:15.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.591+0000 7fe2837fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe2840b4530 0x7fe2840d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42624/0 (socket says 172.21.15.49:42624) 2023-07-18T03:39:15.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.591+0000 7fe2837fe700 1 -- 172.21.15.49:0/2368360652 learned_addr learned my addr 172.21.15.49:0/2368360652 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:15.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.592+0000 7fe288b9c700 1 --2- 172.21.15.49:0/2368360652 >> v2:172.21.15.49:3300/0 conn(0x7fe2840d1220 0x7fe2840b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:15.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.592+0000 7fe283fff700 1 --2- 172.21.15.49:0/2368360652 >> v2:172.21.15.107:3300/0 conn(0x7fe2840b8a60 0x7fe2840b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:15.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.592+0000 7fe282ffd700 1 -- 172.21.15.49:0/2368360652 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874418261 0 0) 0x7fe2840b8e30 con 0x7fe2840b4530 2023-07-18T03:39:15.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.592+0000 7fe282ffd700 1 -- 172.21.15.49:0/2368360652 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe26c0035b0 con 0x7fe2840b4530 2023-07-18T03:39:15.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.592+0000 7fe288b9c700 1 -- 172.21.15.49:0/2368360652 >> v2:172.21.15.107:3300/0 conn(0x7fe2840b8a60 msgr2=0x7fe2840b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:15.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.592+0000 7fe288b9c700 1 --2- 172.21.15.49:0/2368360652 >> v2:172.21.15.107:3300/0 conn(0x7fe2840b8a60 0x7fe2840b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.592+0000 7fe288b9c700 1 -- 172.21.15.49:0/2368360652 >> v1:172.21.15.49:6789/0 conn(0x7fe2840b4530 legacy=0x7fe2840d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:15.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.592+0000 7fe288b9c700 1 -- 172.21.15.49:0/2368360652 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2840b0060 con 0x7fe2840d1220 2023-07-18T03:39:15.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.592+0000 7fe288b9c700 1 --2- 172.21.15.49:0/2368360652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2840d1220 0x7fe2840b0e20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe2840b5b00 tx=0x7fe27c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74af9aef970d17aa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:15.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.593+0000 7fe282ffd700 1 -- 172.21.15.49:0/2368360652 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c00e070 con 0x7fe2840d1220 2023-07-18T03:39:15.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.593+0000 7fe282ffd700 1 -- 172.21.15.49:0/2368360652 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c00bb20 con 0x7fe2840d1220 2023-07-18T03:39:15.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.593+0000 7fe282ffd700 1 -- 172.21.15.49:0/2368360652 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c00be90 con 0x7fe2840d1220 2023-07-18T03:39:15.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.593+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2368360652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2840d1220 msgr2=0x7fe2840b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:15.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.593+0000 7fe28a5ff700 1 --2- 172.21.15.49:0/2368360652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2840d1220 0x7fe2840b0e20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe2840b5b00 tx=0x7fe27c00b330 comp rx=0 tx=0).stop 2023-07-18T03:39:15.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.593+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2368360652 shutdown_connections 2023-07-18T03:39:15.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.593+0000 7fe28a5ff700 1 --2- 172.21.15.49:0/2368360652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2840d1220 0x7fe2840b0e20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.593+0000 7fe28a5ff700 1 --2- 172.21.15.49:0/2368360652 >> v2:172.21.15.107:3300/0 conn(0x7fe2840b8a60 0x7fe2840b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.593+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2368360652 >> 172.21.15.49:0/2368360652 conn(0x7fe28409c1b0 msgr2=0x7fe2840d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:15.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.593+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2368360652 shutdown_connections 2023-07-18T03:39:15.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.593+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2368360652 wait complete. 2023-07-18T03:39:15.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.596+0000 7fe28a5ff700 1 Processor -- start 2023-07-18T03:39:15.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.596+0000 7fe28a5ff700 1 -- start start 2023-07-18T03:39:15.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.597+0000 7fe28a5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2840b4530 0x7fe284157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:15.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.597+0000 7fe28a5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2840b8a60 0x7fe2841575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:15.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.597+0000 7fe28a5ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2840d1220 0x7fe28415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:15.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.597+0000 7fe28a5ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28415f690 con 0x7fe2840d1220 2023-07-18T03:39:15.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.597+0000 7fe28a5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28415f970 con 0x7fe2840b4530 2023-07-18T03:39:15.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.597+0000 7fe28a5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28415fc50 con 0x7fe2840b8a60 2023-07-18T03:39:15.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.598+0000 7fe288b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2840d1220 0x7fe28415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:15.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.598+0000 7fe288b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2840d1220 0x7fe28415ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54686/0 (socket says 172.21.15.49:54686) 2023-07-18T03:39:15.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.598+0000 7fe288b9c700 1 -- 172.21.15.49:0/2040161986 learned_addr learned my addr 172.21.15.49:0/2040161986 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:15.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.598+0000 7fe283fff700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2840b4530 0x7fe284157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:15.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.598+0000 7fe2837fe700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2840b8a60 0x7fe2841575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:15.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.598+0000 7fe288b9c700 1 -- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2840b8a60 msgr2=0x7fe2841575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:15.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.598+0000 7fe288b9c700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2840b8a60 0x7fe2841575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.598+0000 7fe288b9c700 1 -- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2840b4530 msgr2=0x7fe284157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:15.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.598+0000 7fe288b9c700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2840b4530 0x7fe284157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.598+0000 7fe288b9c700 1 -- 172.21.15.49:0/2040161986 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c009a70 con 0x7fe2840d1220 2023-07-18T03:39:15.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.598+0000 7fe288b9c700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2840d1220 0x7fe28415ec40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00b370 tx=0x7fe27c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:15.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.599+0000 7fe280ff9700 1 -- 172.21.15.49:0/2040161986 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c007000 con 0x7fe2840d1220 2023-07-18T03:39:15.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.599+0000 7fe280ff9700 1 -- 172.21.15.49:0/2040161986 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c012860 con 0x7fe2840d1220 2023-07-18T03:39:15.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.599+0000 7fe280ff9700 1 -- 172.21.15.49:0/2040161986 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c012c10 con 0x7fe2840d1220 2023-07-18T03:39:15.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.599+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2040161986 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28415d1b0 con 0x7fe2840d1220 2023-07-18T03:39:15.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.599+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2040161986 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28415f180 con 0x7fe2840d1220 2023-07-18T03:39:15.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.600+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2040161986 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2840723e0 con 0x7fe2840d1220 2023-07-18T03:39:15.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.601+0000 7fe280ff9700 1 -- 172.21.15.49:0/2040161986 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe27c01c400 con 0x7fe2840d1220 2023-07-18T03:39:15.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.602+0000 7fe280ff9700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe27006ace0 0x7fe27006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:15.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.602+0000 7fe280ff9700 1 -- 172.21.15.49:0/2040161986 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe27c08b2e0 con 0x7fe2840d1220 2023-07-18T03:39:15.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.602+0000 7fe283fff700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe27006ace0 0x7fe27006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:15.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.603+0000 7fe283fff700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe27006ace0 0x7fe27006d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe274009b70 tx=0x7fe274008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:15.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.606+0000 7fe280ff9700 1 -- 172.21.15.49:0/2040161986 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe27c0563a0 con 0x7fe2840d1220 2023-07-18T03:39:15.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.812+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2040161986 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe284074350 con 0x7fe2840d1220 2023-07-18T03:39:15.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.812+0000 7fe280ff9700 1 -- 172.21.15.49:0/2040161986 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe27c0599c0 con 0x7fe2840d1220 2023-07-18T03:39:15.808 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:39:15.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.815+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe27006ace0 msgr2=0x7fe27006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:15.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.815+0000 7fe28a5ff700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe27006ace0 0x7fe27006d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe274009b70 tx=0x7fe274008040 comp rx=0 tx=0).stop 2023-07-18T03:39:15.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.815+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2040161986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2840d1220 msgr2=0x7fe28415ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:15.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.815+0000 7fe28a5ff700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2840d1220 0x7fe28415ec40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00b370 tx=0x7fe27c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.815+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2040161986 shutdown_connections 2023-07-18T03:39:15.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.815+0000 7fe28a5ff700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe27006ace0 0x7fe27006d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.815+0000 7fe28a5ff700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2840d1220 0x7fe28415ec40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.815+0000 7fe28a5ff700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2840b4530 0x7fe284157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.815+0000 7fe28a5ff700 1 --2- 172.21.15.49:0/2040161986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2840b8a60 0x7fe2841575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:15.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.815+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2040161986 >> 172.21.15.49:0/2040161986 conn(0x7fe28409c1b0 msgr2=0x7fe2840b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:15.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.815+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2040161986 shutdown_connections 2023-07-18T03:39:15.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:15.815+0000 7fe28a5ff700 1 -- 172.21.15.49:0/2040161986 wait complete. 2023-07-18T03:39:15.828 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:16.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.040+0000 7f4532fff700 1 Processor -- start 2023-07-18T03:39:16.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.040+0000 7f4532fff700 1 -- start start 2023-07-18T03:39:16.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.041+0000 7f4532fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f452c15c950 0x7f452c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.041+0000 7f4532fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f452c0d1220 0x7f452c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.041+0000 7f4532fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c0d2ce0 con 0x7f452c0b1e00 2023-07-18T03:39:16.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.041+0000 7f4532fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c0b3350 con 0x7f452c0d1220 2023-07-18T03:39:16.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.041+0000 7f4532fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c0b3630 con 0x7f452c15c950 2023-07-18T03:39:16.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.042+0000 7f453159c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f452c0d1220 0x7f452c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.042+0000 7f453159c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f452c0d1220 0x7f452c0afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54698/0 (socket says 172.21.15.49:54698) 2023-07-18T03:39:16.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.042+0000 7f453159c700 1 -- 172.21.15.49:0/3119629554 learned_addr learned my addr 172.21.15.49:0/3119629554 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:16.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.042+0000 7f4530d9b700 1 --2- 172.21.15.49:0/3119629554 >> v2:172.21.15.107:3300/0 conn(0x7f452c15c950 0x7f452c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.042+0000 7f453159c700 1 -- 172.21.15.49:0/3119629554 >> v2:172.21.15.107:3300/0 conn(0x7f452c15c950 msgr2=0x7f452c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:16.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.042+0000 7f453159c700 1 --2- 172.21.15.49:0/3119629554 >> v2:172.21.15.107:3300/0 conn(0x7f452c15c950 0x7f452c0b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.042+0000 7f453159c700 1 -- 172.21.15.49:0/3119629554 >> v1:172.21.15.107:6789/0 conn(0x7f452c0b1e00 legacy=0x7f452c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.042+0000 7f453159c700 1 -- 172.21.15.49:0/3119629554 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c0b3910 con 0x7f452c0d1220 2023-07-18T03:39:16.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.042+0000 7f452b7fe700 1 -- 172.21.15.49:0/3119629554 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703148529 0 0) 0x7f452c0d2ce0 con 0x7f452c0b1e00 2023-07-18T03:39:16.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.043+0000 7f453159c700 1 --2- 172.21.15.49:0/3119629554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f452c0d1220 0x7f452c0afac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f452c08d170 tx=0x7f452400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25415092311ca9b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:16.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.043+0000 7f452b7fe700 1 -- 172.21.15.49:0/3119629554 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452400e070 con 0x7f452c0d1220 2023-07-18T03:39:16.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.043+0000 7f452b7fe700 1 -- 172.21.15.49:0/3119629554 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452400b910 con 0x7f452c0d1220 2023-07-18T03:39:16.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.043+0000 7f452b7fe700 1 -- 172.21.15.49:0/3119629554 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452400bcc0 con 0x7f452c0d1220 2023-07-18T03:39:16.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.043+0000 7f4532fff700 1 -- 172.21.15.49:0/3119629554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f452c0d1220 msgr2=0x7f452c0afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.043+0000 7f4532fff700 1 --2- 172.21.15.49:0/3119629554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f452c0d1220 0x7f452c0afac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f452c08d170 tx=0x7f452400b330 comp rx=0 tx=0).stop 2023-07-18T03:39:16.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.043+0000 7f4532fff700 1 -- 172.21.15.49:0/3119629554 shutdown_connections 2023-07-18T03:39:16.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.043+0000 7f4532fff700 1 --2- 172.21.15.49:0/3119629554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f452c0d1220 0x7f452c0afac0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.043+0000 7f4532fff700 1 --2- 172.21.15.49:0/3119629554 >> v2:172.21.15.107:3300/0 conn(0x7f452c15c950 0x7f452c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.043+0000 7f4532fff700 1 -- 172.21.15.49:0/3119629554 >> 172.21.15.49:0/3119629554 conn(0x7f452c09c1d0 msgr2=0x7f452c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:16.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.044+0000 7f4532fff700 1 -- 172.21.15.49:0/3119629554 shutdown_connections 2023-07-18T03:39:16.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.044+0000 7f4532fff700 1 -- 172.21.15.49:0/3119629554 wait complete. 2023-07-18T03:39:16.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.045+0000 7f4532fff700 1 Processor -- start 2023-07-18T03:39:16.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.046+0000 7f4532fff700 1 -- start start 2023-07-18T03:39:16.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.046+0000 7f4532fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f452c0b1e00 0x7f452c0c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.046+0000 7f4532fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f452c0d1220 0x7f452c0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.046+0000 7f4532fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f452c15c950 0x7f452c0c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.047+0000 7f4532fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c0c98f0 con 0x7f452c0b1e00 2023-07-18T03:39:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.047+0000 7f4532fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c0cf740 con 0x7f452c15c950 2023-07-18T03:39:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.047+0000 7f4532fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c151290 con 0x7f452c0d1220 2023-07-18T03:39:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.047+0000 7f4530d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f452c0b1e00 0x7f452c0c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.047+0000 7f4530d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f452c0b1e00 0x7f452c0c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54708/0 (socket says 172.21.15.49:54708) 2023-07-18T03:39:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.047+0000 7f4530d9b700 1 -- 172.21.15.49:0/3748831879 learned_addr learned my addr 172.21.15.49:0/3748831879 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:16.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.047+0000 7f4530d9b700 1 -- 172.21.15.49:0/3748831879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f452c0d1220 msgr2=0x7f452c0c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.047+0000 7f4530d9b700 1 --2- 172.21.15.49:0/3748831879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f452c0d1220 0x7f452c0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.047+0000 7f4530d9b700 1 -- 172.21.15.49:0/3748831879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f452c15c950 msgr2=0x7f452c0c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.047+0000 7f4530d9b700 1 --2- 172.21.15.49:0/3748831879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f452c15c950 0x7f452c0c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.047+0000 7f4530d9b700 1 -- 172.21.15.49:0/3748831879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4524009b30 con 0x7f452c0b1e00 2023-07-18T03:39:16.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.047+0000 7f4530d9b700 1 --2- 172.21.15.49:0/3748831879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f452c0b1e00 0x7f452c0c6a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f452000e900 tx=0x7f452000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:16.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.048+0000 7f45297fa700 1 -- 172.21.15.49:0/3748831879 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520009650 con 0x7f452c0b1e00 2023-07-18T03:39:16.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.048+0000 7f45297fa700 1 -- 172.21.15.49:0/3748831879 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452000c850 con 0x7f452c0b1e00 2023-07-18T03:39:16.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.048+0000 7f45297fa700 1 -- 172.21.15.49:0/3748831879 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452000cbc0 con 0x7f452c0b1e00 2023-07-18T03:39:16.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.048+0000 7f4532fff700 1 -- 172.21.15.49:0/3748831879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c0cc700 con 0x7f452c0b1e00 2023-07-18T03:39:16.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.048+0000 7f4532fff700 1 -- 172.21.15.49:0/3748831879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c0cf950 con 0x7f452c0b1e00 2023-07-18T03:39:16.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.049+0000 7f4532fff700 1 -- 172.21.15.49:0/3748831879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c04e280 con 0x7f452c0b1e00 2023-07-18T03:39:16.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.050+0000 7f45297fa700 1 -- 172.21.15.49:0/3748831879 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f452000cd20 con 0x7f452c0b1e00 2023-07-18T03:39:16.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.051+0000 7f45297fa700 1 --2- 172.21.15.49:0/3748831879 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f451006ace0 0x7f451006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.051+0000 7f45297fa700 1 -- 172.21.15.49:0/3748831879 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4520089940 con 0x7f452c0b1e00 2023-07-18T03:39:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.051+0000 7f452bfff700 1 --2- 172.21.15.49:0/3748831879 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f451006ace0 0x7f451006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.052+0000 7f452bfff700 1 --2- 172.21.15.49:0/3748831879 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f451006ace0 0x7f451006d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4518007fd0 tx=0x7f45180079a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.055+0000 7f45297fa700 1 -- 172.21.15.49:0/3748831879 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4520054950 con 0x7f452c0b1e00 2023-07-18T03:39:16.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.265+0000 7f4532fff700 1 -- 172.21.15.49:0/3748831879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f452c1e7120 con 0x7f452c0b1e00 2023-07-18T03:39:16.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.267+0000 7f45297fa700 1 -- 172.21.15.49:0/3748831879 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f4520017090 con 0x7f452c0b1e00 2023-07-18T03:39:16.265 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:16.265 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:37.510006+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:16.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.269+0000 7f4532fff700 1 -- 172.21.15.49:0/3748831879 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f451006ace0 msgr2=0x7f451006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.269+0000 7f4532fff700 1 --2- 172.21.15.49:0/3748831879 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f451006ace0 0x7f451006d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4518007fd0 tx=0x7f45180079a0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.269+0000 7f4532fff700 1 -- 172.21.15.49:0/3748831879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f452c0b1e00 msgr2=0x7f452c0c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.269+0000 7f4532fff700 1 --2- 172.21.15.49:0/3748831879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f452c0b1e00 0x7f452c0c6a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f452000e900 tx=0x7f452000ecc0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.269+0000 7f4532fff700 1 -- 172.21.15.49:0/3748831879 shutdown_connections 2023-07-18T03:39:16.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.269+0000 7f4532fff700 1 --2- 172.21.15.49:0/3748831879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f452c15c950 0x7f452c0c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.269+0000 7f4532fff700 1 --2- 172.21.15.49:0/3748831879 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f451006ace0 0x7f451006d190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.269+0000 7f4532fff700 1 --2- 172.21.15.49:0/3748831879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f452c0b1e00 0x7f452c0c6a70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.269+0000 7f4532fff700 1 --2- 172.21.15.49:0/3748831879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f452c0d1220 0x7f452c0c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.269+0000 7f4532fff700 1 -- 172.21.15.49:0/3748831879 >> 172.21.15.49:0/3748831879 conn(0x7f452c09c1d0 msgr2=0x7f452c0d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:16.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.269+0000 7f4532fff700 1 -- 172.21.15.49:0/3748831879 shutdown_connections 2023-07-18T03:39:16.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.269+0000 7f4532fff700 1 -- 172.21.15.49:0/3748831879 wait complete. 2023-07-18T03:39:16.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:38:33.773236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:38:35.023692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:39:16.283 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:39:16.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.495+0000 7fddc77ff700 1 Processor -- start 2023-07-18T03:39:16.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.496+0000 7fddc77ff700 1 -- start start 2023-07-18T03:39:16.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.497+0000 7fddc77ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fddc00b8a60 0x7fddc00b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.497+0000 7fddc77ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc00b8e30 con 0x7fddc00b4530 2023-07-18T03:39:16.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.497+0000 7fddc77ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc00afaa0 con 0x7fddc00d1220 2023-07-18T03:39:16.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.497+0000 7fddc77ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc00afd80 con 0x7fddc00b8a60 2023-07-18T03:39:16.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.497+0000 7fddc4d9a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fddc00b4530 0x7fddc00d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42628/0 (socket says 172.21.15.49:42628) 2023-07-18T03:39:16.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.497+0000 7fddc4d9a700 1 -- 172.21.15.49:0/4111153127 learned_addr learned my addr 172.21.15.49:0/4111153127 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:16.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957915969 0 0) 0x7fddc00b8e30 con 0x7fddc00b4530 2023-07-18T03:39:16.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddac0035b0 con 0x7fddc00b4530 2023-07-18T03:39:16.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddc559b700 1 --2- 172.21.15.49:0/4111153127 >> v2:172.21.15.107:3300/0 conn(0x7fddc00b8a60 0x7fddc00b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2158971352 0 0) 0x7fddac0035b0 con 0x7fddc00b4530 2023-07-18T03:39:16.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 >> v2:172.21.15.107:3300/0 conn(0x7fddc00b8a60 msgr2=0x7fddc00b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:16.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 --2- 172.21.15.49:0/4111153127 >> v2:172.21.15.107:3300/0 conn(0x7fddc00b8a60 0x7fddc00b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 >> v1:172.21.15.107:6789/0 conn(0x7fddc00d1220 legacy=0x7fddc00b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc00b0060 con 0x7fddc00b4530 2023-07-18T03:39:16.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7fddb0001670 con 0x7fddc00b4530 2023-07-18T03:39:16.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 >> v1:172.21.15.49:6789/0 conn(0x7fddc00b4530 legacy=0x7fddc00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 --2- 172.21.15.49:0/4111153127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddac003a50 0x7fddac003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 --2- 172.21.15.49:0/4111153127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddac004850 0x7fddac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 --2- 172.21.15.49:0/4111153127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac005200 0x7fddac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00afd80 con 0x7fddac004850 2023-07-18T03:39:16.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddac00c0c0 con 0x7fddac003a50 2023-07-18T03:39:16.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddac00c280 con 0x7fddac005200 2023-07-18T03:39:16.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.498+0000 7fddc4d9a700 1 --2- 172.21.15.49:0/4111153127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddac004850 0x7fddac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddc559b700 1 --2- 172.21.15.49:0/4111153127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddac003a50 0x7fddac003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddc5d9c700 1 --2- 172.21.15.49:0/4111153127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac005200 0x7fddac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddc4d9a700 1 -- 172.21.15.49:0/4111153127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac005200 msgr2=0x7fddac00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:16.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddc4d9a700 1 --2- 172.21.15.49:0/4111153127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac005200 0x7fddac00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddc4d9a700 1 -- 172.21.15.49:0/4111153127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddac003a50 msgr2=0x7fddac003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:16.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddc4d9a700 1 --2- 172.21.15.49:0/4111153127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddac003a50 0x7fddac003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddc4d9a700 1 -- 172.21.15.49:0/4111153127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddac0035b0 con 0x7fddac004850 2023-07-18T03:39:16.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddc559b700 1 --2- 172.21.15.49:0/4111153127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddac003a50 0x7fddac003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:16.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddc5d9c700 1 -- 172.21.15.49:0/4111153127 reap_dead start 2023-07-18T03:39:16.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddc4d9a700 1 --2- 172.21.15.49:0/4111153127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddac004850 0x7fddac004cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c160 tx=0x7fddac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d4bc7c75f497bf server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:16.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0007b70 con 0x7fddac004850 2023-07-18T03:39:16.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb000cd20 con 0x7fddac004850 2023-07-18T03:39:16.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.499+0000 7fddb7fff700 1 -- 172.21.15.49:0/4111153127 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0005ba0 con 0x7fddac004850 2023-07-18T03:39:16.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.500+0000 7fddc77ff700 1 -- 172.21.15.49:0/4111153127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddac004850 msgr2=0x7fddac004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.500+0000 7fddc77ff700 1 --2- 172.21.15.49:0/4111153127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddac004850 0x7fddac004cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c160 tx=0x7fddac005880 comp rx=0 tx=0).stop 2023-07-18T03:39:16.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.500+0000 7fddc77ff700 1 -- 172.21.15.49:0/4111153127 shutdown_connections 2023-07-18T03:39:16.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.500+0000 7fddc77ff700 1 --2- 172.21.15.49:0/4111153127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddac004850 0x7fddac004cc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.500+0000 7fddc77ff700 1 -- 172.21.15.49:0/4111153127 >> 172.21.15.49:0/4111153127 conn(0x7fddc009c1b0 msgr2=0x7fddc00d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:16.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.500+0000 7fddc77ff700 1 -- 172.21.15.49:0/4111153127 shutdown_connections 2023-07-18T03:39:16.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.500+0000 7fddc77ff700 1 -- 172.21.15.49:0/4111153127 wait complete. 2023-07-18T03:39:16.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.502+0000 7fddc77ff700 1 Processor -- start 2023-07-18T03:39:16.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.502+0000 7fddc77ff700 1 -- start start 2023-07-18T03:39:16.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.503+0000 7fddc77ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac004850 0x7fddc015ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.503+0000 7fddc77ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddc01f6680 0x7fddc00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.503+0000 7fddc77ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc015b0d0 0x7fddc015b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.503+0000 7fddc77ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc01f3150 con 0x7fddc015b0d0 2023-07-18T03:39:16.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.503+0000 7fddc77ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc015ba90 con 0x7fddc01f6680 2023-07-18T03:39:16.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.503+0000 7fddc77ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc01f27b0 con 0x7fddac004850 2023-07-18T03:39:16.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.503+0000 7fddc5d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc015b0d0 0x7fddc015b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.503+0000 7fddc5d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc015b0d0 0x7fddc015b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54722/0 (socket says 172.21.15.49:54722) 2023-07-18T03:39:16.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.503+0000 7fddc5d9c700 1 -- 172.21.15.49:0/65000479 learned_addr learned my addr 172.21.15.49:0/65000479 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:16.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.503+0000 7fddc559b700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac004850 0x7fddc015ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.503+0000 7fddc4d9a700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddc01f6680 0x7fddc00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.504+0000 7fddc5d9c700 1 -- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac004850 msgr2=0x7fddc015ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.504+0000 7fddc5d9c700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac004850 0x7fddc015ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.504+0000 7fddc5d9c700 1 -- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddc01f6680 msgr2=0x7fddc00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.504+0000 7fddc5d9c700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddc01f6680 0x7fddc00cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.504+0000 7fddc5d9c700 1 -- 172.21.15.49:0/65000479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc01f2a90 con 0x7fddc015b0d0 2023-07-18T03:39:16.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.504+0000 7fddc559b700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac004850 0x7fddc015ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:16.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.504+0000 7fddc5d9c700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc015b0d0 0x7fddc015b550 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fddc00a0df0 tx=0x7fddb800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:16.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.504+0000 7fddb5ffb700 1 -- 172.21.15.49:0/65000479 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8003c10 con 0x7fddc015b0d0 2023-07-18T03:39:16.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.504+0000 7fddb5ffb700 1 -- 172.21.15.49:0/65000479 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb8003d70 con 0x7fddc015b0d0 2023-07-18T03:39:16.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.504+0000 7fddc77ff700 1 -- 172.21.15.49:0/65000479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc01f2d70 con 0x7fddc015b0d0 2023-07-18T03:39:16.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.504+0000 7fddb5ffb700 1 -- 172.21.15.49:0/65000479 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8005310 con 0x7fddc015b0d0 2023-07-18T03:39:16.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.504+0000 7fddc77ff700 1 -- 172.21.15.49:0/65000479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc015f0f0 con 0x7fddc015b0d0 2023-07-18T03:39:16.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.505+0000 7fddc77ff700 1 -- 172.21.15.49:0/65000479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc01f3290 con 0x7fddc015b0d0 2023-07-18T03:39:16.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.506+0000 7fddb5ffb700 1 -- 172.21.15.49:0/65000479 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fddb8005470 con 0x7fddc015b0d0 2023-07-18T03:39:16.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.507+0000 7fddb5ffb700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdda406ace0 0x7fdda406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.507+0000 7fddb5ffb700 1 -- 172.21.15.49:0/65000479 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fddb8089440 con 0x7fddc015b0d0 2023-07-18T03:39:16.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.508+0000 7fddc559b700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdda406ace0 0x7fdda406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.508+0000 7fddc559b700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdda406ace0 0x7fdda406d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fddc0068ff0 tx=0x7fddbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:16.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.512+0000 7fddb5ffb700 1 -- 172.21.15.49:0/65000479 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fddc01f3290 con 0x7fddc015b0d0 2023-07-18T03:39:16.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.722+0000 7fddc77ff700 1 -- 172.21.15.49:0/65000479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fddc01f3290 con 0x7fddc015b0d0 2023-07-18T03:39:16.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.723+0000 7fddb5ffb700 1 -- 172.21.15.49:0/65000479 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fddc01f3290 con 0x7fddc015b0d0 2023-07-18T03:39:16.718 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:39:16.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.725+0000 7fddc77ff700 1 -- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdda406ace0 msgr2=0x7fdda406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.725+0000 7fddc77ff700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdda406ace0 0x7fdda406d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fddc0068ff0 tx=0x7fddbc008040 comp rx=0 tx=0).stop 2023-07-18T03:39:16.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.725+0000 7fddc77ff700 1 -- 172.21.15.49:0/65000479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc015b0d0 msgr2=0x7fddc015b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.725+0000 7fddc77ff700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc015b0d0 0x7fddc015b550 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fddc00a0df0 tx=0x7fddb800ee80 comp rx=0 tx=0).stop 2023-07-18T03:39:16.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.725+0000 7fddc77ff700 1 -- 172.21.15.49:0/65000479 shutdown_connections 2023-07-18T03:39:16.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.725+0000 7fddc77ff700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdda406ace0 0x7fdda406d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.725+0000 7fddc77ff700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc015b0d0 0x7fddc015b550 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.725+0000 7fddc77ff700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac004850 0x7fddc015ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.725+0000 7fddc77ff700 1 --2- 172.21.15.49:0/65000479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddc01f6680 0x7fddc00cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.725+0000 7fddc77ff700 1 -- 172.21.15.49:0/65000479 >> 172.21.15.49:0/65000479 conn(0x7fddc009c1b0 msgr2=0x7fddc00b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:16.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.725+0000 7fddc77ff700 1 -- 172.21.15.49:0/65000479 shutdown_connections 2023-07-18T03:39:16.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.725+0000 7fddc77ff700 1 -- 172.21.15.49:0/65000479 wait complete. 2023-07-18T03:39:16.738 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:39:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.950+0000 7fc8c57ff700 1 Processor -- start 2023-07-18T03:39:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.951+0000 7fc8c57ff700 1 -- start start 2023-07-18T03:39:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.952+0000 7fc8c57ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc8c00d0b10 0x7fc8c00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.952+0000 7fc8c57ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8c00b0e20 0x7fc8c00b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.952+0000 7fc8c57ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c00b8e30 con 0x7fc8c00b8a60 2023-07-18T03:39:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.952+0000 7fc8c57ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c00d14d0 con 0x7fc8c00d0b10 2023-07-18T03:39:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.952+0000 7fc8c57ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00afaa0 con 0x7fc8c00b0e20 2023-07-18T03:39:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.952+0000 7fc8be7fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc8c00d0b10 0x7fc8c00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.952+0000 7fc8be7fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc8c00d0b10 0x7fc8c00d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54730/0 (socket says 172.21.15.49:54730) 2023-07-18T03:39:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.952+0000 7fc8beffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc8c00b8a60 0x7fc8c00b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42634/0 (socket says 172.21.15.49:42634) 2023-07-18T03:39:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.952+0000 7fc8be7fc700 1 -- 172.21.15.49:0/2148917165 learned_addr learned my addr 172.21.15.49:0/2148917165 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.952+0000 7fc8bf7fe700 1 --2- 172.21.15.49:0/2148917165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8c00b0e20 0x7fc8c00b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.953+0000 7fc8be7fc700 1 -- 172.21.15.49:0/2148917165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8c00b0e20 msgr2=0x7fc8c00b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.953+0000 7fc8be7fc700 1 --2- 172.21.15.49:0/2148917165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8c00b0e20 0x7fc8c00b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.953+0000 7fc8be7fc700 1 -- 172.21.15.49:0/2148917165 >> v1:172.21.15.49:6789/0 conn(0x7fc8c00b8a60 legacy=0x7fc8c00b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.953+0000 7fc8be7fc700 1 -- 172.21.15.49:0/2148917165 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c00afd80 con 0x7fc8c00d0b10 2023-07-18T03:39:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.953+0000 7fc8bdffb700 1 -- 172.21.15.49:0/2148917165 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828244838 0 0) 0x7fc8c00b8e30 con 0x7fc8c00b8a60 2023-07-18T03:39:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.953+0000 7fc8be7fc700 1 --2- 172.21.15.49:0/2148917165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c00d0b10 0x7fc8c00d0f90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40098f0 tx=0x7fc8b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f701140cf547e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.953+0000 7fc8bdffb700 1 -- 172.21.15.49:0/2148917165 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4011830 con 0x7fc8c00d0b10 2023-07-18T03:39:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.953+0000 7fc8bdffb700 1 -- 172.21.15.49:0/2148917165 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b4011990 con 0x7fc8c00d0b10 2023-07-18T03:39:16.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.953+0000 7fc8bdffb700 1 -- 172.21.15.49:0/2148917165 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4011d20 con 0x7fc8c00d0b10 2023-07-18T03:39:16.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.954+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2148917165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c00d0b10 msgr2=0x7fc8c00d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.954+0000 7fc8c57ff700 1 --2- 172.21.15.49:0/2148917165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c00d0b10 0x7fc8c00d0f90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40098f0 tx=0x7fc8b4009c00 comp rx=0 tx=0).stop 2023-07-18T03:39:16.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.954+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2148917165 shutdown_connections 2023-07-18T03:39:16.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.954+0000 7fc8c57ff700 1 --2- 172.21.15.49:0/2148917165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8c00b0e20 0x7fc8c00b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.954+0000 7fc8c57ff700 1 --2- 172.21.15.49:0/2148917165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c00d0b10 0x7fc8c00d0f90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.954+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2148917165 >> 172.21.15.49:0/2148917165 conn(0x7fc8c009c1b0 msgr2=0x7fc8c00d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:16.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.954+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2148917165 shutdown_connections 2023-07-18T03:39:16.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.954+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2148917165 wait complete. 2023-07-18T03:39:16.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.956+0000 7fc8c57ff700 1 Processor -- start 2023-07-18T03:39:16.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.956+0000 7fc8c57ff700 1 -- start start 2023-07-18T03:39:16.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.957+0000 7fc8c57ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8c00b0e20 0x7fc8c00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.957+0000 7fc8c57ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c00b8a60 0x7fc8c00cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.957+0000 7fc8c57ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c00d0b10 0x7fc8c015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.957+0000 7fc8c57ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00cacc0 con 0x7fc8c00d0b10 2023-07-18T03:39:16.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.957+0000 7fc8c57ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00ca0b0 con 0x7fc8c00b8a60 2023-07-18T03:39:16.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.957+0000 7fc8c57ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00ca390 con 0x7fc8c00b0e20 2023-07-18T03:39:16.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.957+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c00d0b10 0x7fc8c015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.957+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c00d0b10 0x7fc8c015a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54746/0 (socket says 172.21.15.49:54746) 2023-07-18T03:39:16.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.957+0000 7fc8bf7fe700 1 -- 172.21.15.49:0/2688193993 learned_addr learned my addr 172.21.15.49:0/2688193993 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:16.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.957+0000 7fc8be7fc700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c00b8a60 0x7fc8c00cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.957+0000 7fc8beffd700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8c00b0e20 0x7fc8c00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8bf7fe700 1 -- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8c00b0e20 msgr2=0x7fc8c00c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8bf7fe700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8c00b0e20 0x7fc8c00c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8bf7fe700 1 -- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c00b8a60 msgr2=0x7fc8c00cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:16.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8bf7fe700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c00b8a60 0x7fc8c00cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:16.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8bf7fe700 1 -- 172.21.15.49:0/2688193993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b4009550 con 0x7fc8c00d0b10 2023-07-18T03:39:16.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8be7fc700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c00b8a60 0x7fc8c00cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:16.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8beffd700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8c00b0e20 0x7fc8c00c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:16.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8bf7fe700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c00d0b10 0x7fc8c015a5e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800efc0 tx=0x7fc8b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:16.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8a77fe700 1 -- 172.21.15.49:0/2688193993 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800cd40 con 0x7fc8c00d0b10 2023-07-18T03:39:16.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8a77fe700 1 -- 172.21.15.49:0/2688193993 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b800f040 con 0x7fc8c00d0b10 2023-07-18T03:39:16.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2688193993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c00ca670 con 0x7fc8c00d0b10 2023-07-18T03:39:16.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8a77fe700 1 -- 172.21.15.49:0/2688193993 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b80146c0 con 0x7fc8c00d0b10 2023-07-18T03:39:16.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.958+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2688193993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c015ec00 con 0x7fc8c00d0b10 2023-07-18T03:39:16.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.959+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2688193993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c008b870 con 0x7fc8c00d0b10 2023-07-18T03:39:16.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.960+0000 7fc8a77fe700 1 -- 172.21.15.49:0/2688193993 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc8b8014820 con 0x7fc8c00d0b10 2023-07-18T03:39:16.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.961+0000 7fc8a77fe700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc8ac06ace0 0x7fc8ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:16.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.961+0000 7fc8a77fe700 1 -- 172.21.15.49:0/2688193993 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc8b8089f40 con 0x7fc8c00d0b10 2023-07-18T03:39:16.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.962+0000 7fc8beffd700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc8ac06ace0 0x7fc8ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:16.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.963+0000 7fc8beffd700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc8ac06ace0 0x7fc8ac06d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0000f60 tx=0x7fc8b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:16.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:16.966+0000 7fc8a77fe700 1 -- 172.21.15.49:0/2688193993 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc8b80589e0 con 0x7fc8c00d0b10 2023-07-18T03:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.172+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2688193993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8c006a3e0 con 0x7fc8c00d0b10 2023-07-18T03:39:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.173+0000 7fc8a77fe700 1 -- 172.21.15.49:0/2688193993 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8b8018020 con 0x7fc8c00d0b10 2023-07-18T03:39:17.168 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:39:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.175+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc8ac06ace0 msgr2=0x7fc8ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.175+0000 7fc8c57ff700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc8ac06ace0 0x7fc8ac06d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0000f60 tx=0x7fc8b0009040 comp rx=0 tx=0).stop 2023-07-18T03:39:17.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.175+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2688193993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c00d0b10 msgr2=0x7fc8c015a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:17.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.175+0000 7fc8c57ff700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c00d0b10 0x7fc8c015a5e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800efc0 tx=0x7fc8b800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.176+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2688193993 shutdown_connections 2023-07-18T03:39:17.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.176+0000 7fc8c57ff700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc8ac06ace0 0x7fc8ac06d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.176+0000 7fc8c57ff700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8c00d0b10 0x7fc8c015a5e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.176+0000 7fc8c57ff700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8c00b0e20 0x7fc8c00c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.176+0000 7fc8c57ff700 1 --2- 172.21.15.49:0/2688193993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c00b8a60 0x7fc8c00cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.176+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2688193993 >> 172.21.15.49:0/2688193993 conn(0x7fc8c009c1b0 msgr2=0x7fc8c00b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:17.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.176+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2688193993 shutdown_connections 2023-07-18T03:39:17.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.176+0000 7fc8c57ff700 1 -- 172.21.15.49:0/2688193993 wait complete. 2023-07-18T03:39:17.188 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:39:17.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.400+0000 7fa1135ff700 1 Processor -- start 2023-07-18T03:39:17.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.400+0000 7fa1135ff700 1 -- start start 2023-07-18T03:39:17.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.401+0000 7fa1135ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa10c0d0b10 0x7fa10c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:17.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.401+0000 7fa1135ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa10c0b0e20 0x7fa10c0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:17.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.401+0000 7fa1135ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa10c0b8e30 con 0x7fa10c0b8a60 2023-07-18T03:39:17.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.401+0000 7fa1135ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10c0d14d0 con 0x7fa10c0b0e20 2023-07-18T03:39:17.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.401+0000 7fa1135ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10c0afaa0 con 0x7fa10c0d0b10 2023-07-18T03:39:17.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.402+0000 7fa111b9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa10c0b0e20 0x7fa10c0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:17.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.402+0000 7fa111b9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa10c0b0e20 0x7fa10c0b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54748/0 (socket says 172.21.15.49:54748) 2023-07-18T03:39:17.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.402+0000 7fa111b9c700 1 -- 172.21.15.49:0/3121686346 learned_addr learned my addr 172.21.15.49:0/3121686346 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:17.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.402+0000 7fa110b9a700 1 --2- 172.21.15.49:0/3121686346 >> v2:172.21.15.107:3300/0 conn(0x7fa10c0d0b10 0x7fa10c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:17.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.402+0000 7fa111b9c700 1 -- 172.21.15.49:0/3121686346 >> v2:172.21.15.107:3300/0 conn(0x7fa10c0d0b10 msgr2=0x7fa10c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:17.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.402+0000 7fa111b9c700 1 --2- 172.21.15.49:0/3121686346 >> v2:172.21.15.107:3300/0 conn(0x7fa10c0d0b10 0x7fa10c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.402+0000 7fa111b9c700 1 -- 172.21.15.49:0/3121686346 >> v1:172.21.15.107:6789/0 conn(0x7fa10c0b8a60 legacy=0x7fa10c0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:17.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.402+0000 7fa111b9c700 1 -- 172.21.15.49:0/3121686346 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c0afd80 con 0x7fa10c0b0e20 2023-07-18T03:39:17.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.402+0000 7fa110b9a700 1 --2- 172.21.15.49:0/3121686346 >> v2:172.21.15.107:3300/0 conn(0x7fa10c0d0b10 0x7fa10c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:17.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.402+0000 7fa111b9c700 1 --2- 172.21.15.49:0/3121686346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0b0e20 0x7fa10c0b1260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa10c09c8e0 tx=0x7fa10400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5553012c23d09cf server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:17.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.403+0000 7fa103fff700 1 -- 172.21.15.49:0/3121686346 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10400e070 con 0x7fa10c0b0e20 2023-07-18T03:39:17.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.403+0000 7fa103fff700 1 -- 172.21.15.49:0/3121686346 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa104003a20 con 0x7fa10c0b0e20 2023-07-18T03:39:17.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.403+0000 7fa103fff700 1 -- 172.21.15.49:0/3121686346 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104003dd0 con 0x7fa10c0b0e20 2023-07-18T03:39:17.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.403+0000 7fa1135ff700 1 -- 172.21.15.49:0/3121686346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0b0e20 msgr2=0x7fa10c0b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:17.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.403+0000 7fa1135ff700 1 --2- 172.21.15.49:0/3121686346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0b0e20 0x7fa10c0b1260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa10c09c8e0 tx=0x7fa10400b330 comp rx=0 tx=0).stop 2023-07-18T03:39:17.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.403+0000 7fa1135ff700 1 -- 172.21.15.49:0/3121686346 shutdown_connections 2023-07-18T03:39:17.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.403+0000 7fa1135ff700 1 --2- 172.21.15.49:0/3121686346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0b0e20 0x7fa10c0b1260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.403+0000 7fa1135ff700 1 --2- 172.21.15.49:0/3121686346 >> v2:172.21.15.107:3300/0 conn(0x7fa10c0d0b10 0x7fa10c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.403+0000 7fa1135ff700 1 -- 172.21.15.49:0/3121686346 >> 172.21.15.49:0/3121686346 conn(0x7fa10c09c1b0 msgr2=0x7fa10c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:17.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.404+0000 7fa1135ff700 1 -- 172.21.15.49:0/3121686346 shutdown_connections 2023-07-18T03:39:17.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.404+0000 7fa1135ff700 1 -- 172.21.15.49:0/3121686346 wait complete. 2023-07-18T03:39:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.405+0000 7fa1135ff700 1 Processor -- start 2023-07-18T03:39:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.405+0000 7fa1135ff700 1 -- start start 2023-07-18T03:39:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.406+0000 7fa1135ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa10c0b0e20 0x7fa10c0cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.406+0000 7fa1135ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0b8a60 0x7fa10c1f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:17.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.406+0000 7fa1135ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa10c0d0b10 0x7fa10c1f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.406+0000 7fa1135ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0d03c0 con 0x7fa10c0b8a60 2023-07-18T03:39:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.406+0000 7fa1135ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c151ba0 con 0x7fa10c0b0e20 2023-07-18T03:39:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.406+0000 7fa1135ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c1f5e50 con 0x7fa10c0d0b10 2023-07-18T03:39:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.407+0000 7fa110b9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0b8a60 0x7fa10c1f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.407+0000 7fa110b9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0b8a60 0x7fa10c1f52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54760/0 (socket says 172.21.15.49:54760) 2023-07-18T03:39:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.407+0000 7fa110b9a700 1 -- 172.21.15.49:0/2563885255 learned_addr learned my addr 172.21.15.49:0/2563885255 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.407+0000 7fa11139b700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa10c0b0e20 0x7fa10c0cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:17.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.407+0000 7fa111b9c700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa10c0d0b10 0x7fa10c1f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:17.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.407+0000 7fa110b9a700 1 -- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa10c0d0b10 msgr2=0x7fa10c1f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:17.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.407+0000 7fa110b9a700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa10c0d0b10 0x7fa10c1f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.407+0000 7fa110b9a700 1 -- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa10c0b0e20 msgr2=0x7fa10c0cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:17.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.407+0000 7fa110b9a700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa10c0b0e20 0x7fa10c0cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.407+0000 7fa110b9a700 1 -- 172.21.15.49:0/2563885255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa104009b30 con 0x7fa10c0b8a60 2023-07-18T03:39:17.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.407+0000 7fa11139b700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa10c0b0e20 0x7fa10c0cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:17.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.407+0000 7fa110b9a700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0b8a60 0x7fa10c1f52e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00c7f0 tx=0x7fa0fc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:17.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.408+0000 7fa101ffb700 1 -- 172.21.15.49:0/2563885255 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc011800 con 0x7fa10c0b8a60 2023-07-18T03:39:17.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.408+0000 7fa101ffb700 1 -- 172.21.15.49:0/2563885255 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc011960 con 0x7fa10c0b8a60 2023-07-18T03:39:17.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.408+0000 7fa101ffb700 1 -- 172.21.15.49:0/2563885255 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc011d10 con 0x7fa10c0b8a60 2023-07-18T03:39:17.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.408+0000 7fa1135ff700 1 -- 172.21.15.49:0/2563885255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10c1f6130 con 0x7fa10c0b8a60 2023-07-18T03:39:17.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.408+0000 7fa1135ff700 1 -- 172.21.15.49:0/2563885255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c0ca390 con 0x7fa10c0b8a60 2023-07-18T03:39:17.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.409+0000 7fa1135ff700 1 -- 172.21.15.49:0/2563885255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10c0d0960 con 0x7fa10c0b8a60 2023-07-18T03:39:17.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.410+0000 7fa101ffb700 1 -- 172.21.15.49:0/2563885255 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa0fc018360 con 0x7fa10c0b8a60 2023-07-18T03:39:17.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.411+0000 7fa101ffb700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa0f006ace0 0x7fa0f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:17.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.411+0000 7fa101ffb700 1 -- 172.21.15.49:0/2563885255 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa0fc0895e0 con 0x7fa10c0b8a60 2023-07-18T03:39:17.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.411+0000 7fa11139b700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa0f006ace0 0x7fa0f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:17.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.412+0000 7fa11139b700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa0f006ace0 0x7fa0f006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa108003fc0 tx=0x7fa108009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:17.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.415+0000 7fa101ffb700 1 -- 172.21.15.49:0/2563885255 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa0fc057ff0 con 0x7fa10c0b8a60 2023-07-18T03:39:17.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.622+0000 7fa1135ff700 1 -- 172.21.15.49:0/2563885255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa10c074350 con 0x7fa10c0b8a60 2023-07-18T03:39:17.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.623+0000 7fa101ffb700 1 -- 172.21.15.49:0/2563885255 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0fc057b80 con 0x7fa10c0b8a60 2023-07-18T03:39:17.618 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:39:17.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.626+0000 7fa1135ff700 1 -- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa0f006ace0 msgr2=0x7fa0f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:17.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.626+0000 7fa1135ff700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa0f006ace0 0x7fa0f006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa108003fc0 tx=0x7fa108009040 comp rx=0 tx=0).stop 2023-07-18T03:39:17.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.626+0000 7fa1135ff700 1 -- 172.21.15.49:0/2563885255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0b8a60 msgr2=0x7fa10c1f52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:17.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.626+0000 7fa1135ff700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0b8a60 0x7fa10c1f52e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00c7f0 tx=0x7fa0fc00cbb0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.626+0000 7fa1135ff700 1 -- 172.21.15.49:0/2563885255 shutdown_connections 2023-07-18T03:39:17.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.626+0000 7fa1135ff700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa10c0d0b10 0x7fa10c1f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.626+0000 7fa1135ff700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa10c0b0e20 0x7fa10c0cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.626+0000 7fa1135ff700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa0f006ace0 0x7fa0f006d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.626+0000 7fa1135ff700 1 --2- 172.21.15.49:0/2563885255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0b8a60 0x7fa10c1f52e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.626+0000 7fa1135ff700 1 -- 172.21.15.49:0/2563885255 >> 172.21.15.49:0/2563885255 conn(0x7fa10c09c1b0 msgr2=0x7fa10c15c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:17.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.626+0000 7fa1135ff700 1 -- 172.21.15.49:0/2563885255 shutdown_connections 2023-07-18T03:39:17.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.626+0000 7fa1135ff700 1 -- 172.21.15.49:0/2563885255 wait complete. 2023-07-18T03:39:17.638 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:39:17.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.852+0000 7f5b765ff700 1 Processor -- start 2023-07-18T03:39:17.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.852+0000 7f5b765ff700 1 -- start start 2023-07-18T03:39:17.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.853+0000 7f5b765ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b700d0b10 0x7f5b700d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:17.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.854+0000 7f5b765ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b700d14a0 con 0x7f5b700b0e20 2023-07-18T03:39:17.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.854+0000 7f5b765ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b700b8e30 con 0x7f5b700b8a60 2023-07-18T03:39:17.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.854+0000 7f5b765ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b700afaa0 con 0x7f5b700d0b10 2023-07-18T03:39:17.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.854+0000 7f5b74b9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5b700b0e20 0x7f5b700b12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42640/0 (socket says 172.21.15.49:42640) 2023-07-18T03:39:17.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.854+0000 7f5b74b9c700 1 -- 172.21.15.49:0/520791414 learned_addr learned my addr 172.21.15.49:0/520791414 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:17.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.854+0000 7f5b6f7fe700 1 --2- 172.21.15.49:0/520791414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b700d0b10 0x7f5b700d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:17.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.854+0000 7f5b6effd700 1 -- 172.21.15.49:0/520791414 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307111585 0 0) 0x7f5b68001490 con 0x7f5b700b0e20 2023-07-18T03:39:17.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.854+0000 7f5b6effd700 1 -- 172.21.15.49:0/520791414 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b580035b0 con 0x7f5b700b0e20 2023-07-18T03:39:17.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.854+0000 7f5b6effd700 1 -- 172.21.15.49:0/520791414 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352403923 0 0) 0x7f5b700b8e30 con 0x7f5b700b8a60 2023-07-18T03:39:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.854+0000 7f5b6effd700 1 -- 172.21.15.49:0/520791414 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b68001490 con 0x7f5b700b8a60 2023-07-18T03:39:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.855+0000 7f5b6effd700 1 -- 172.21.15.49:0/520791414 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4021594887 0 0) 0x7f5b580035b0 con 0x7f5b700b0e20 2023-07-18T03:39:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.855+0000 7f5b6f7fe700 1 -- 172.21.15.49:0/520791414 >> v1:172.21.15.107:6789/0 conn(0x7f5b700b8a60 legacy=0x7f5b700b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.855+0000 7f5b6f7fe700 1 -- 172.21.15.49:0/520791414 >> v1:172.21.15.49:6789/0 conn(0x7f5b700b0e20 legacy=0x7f5b700b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.855+0000 7f5b6f7fe700 1 -- 172.21.15.49:0/520791414 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b700afd80 con 0x7f5b700d0b10 2023-07-18T03:39:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.855+0000 7f5b6f7fe700 1 --2- 172.21.15.49:0/520791414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b700d0b10 0x7f5b700d0f60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009870 tx=0x7f5b64009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5b20d634254e2f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.855+0000 7f5b6effd700 1 -- 172.21.15.49:0/520791414 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68001b80 con 0x7f5b700d0b10 2023-07-18T03:39:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.855+0000 7f5b6effd700 1 -- 172.21.15.49:0/520791414 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b64004030 con 0x7f5b700d0b10 2023-07-18T03:39:17.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.855+0000 7f5b6effd700 1 -- 172.21.15.49:0/520791414 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b640043c0 con 0x7f5b700d0b10 2023-07-18T03:39:17.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.856+0000 7f5b765ff700 1 -- 172.21.15.49:0/520791414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b700d0b10 msgr2=0x7f5b700d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.856+0000 7f5b765ff700 1 --2- 172.21.15.49:0/520791414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b700d0b10 0x7f5b700d0f60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009870 tx=0x7f5b64009b80 comp rx=0 tx=0).stop 2023-07-18T03:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.856+0000 7f5b765ff700 1 -- 172.21.15.49:0/520791414 shutdown_connections 2023-07-18T03:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.856+0000 7f5b765ff700 1 --2- 172.21.15.49:0/520791414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b700d0b10 0x7f5b700d0f60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.856+0000 7f5b765ff700 1 -- 172.21.15.49:0/520791414 >> 172.21.15.49:0/520791414 conn(0x7f5b7009c1b0 msgr2=0x7f5b700d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.856+0000 7f5b765ff700 1 -- 172.21.15.49:0/520791414 shutdown_connections 2023-07-18T03:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.856+0000 7f5b765ff700 1 -- 172.21.15.49:0/520791414 wait complete. 2023-07-18T03:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.858+0000 7f5b765ff700 1 Processor -- start 2023-07-18T03:39:17.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.858+0000 7f5b765ff700 1 -- start start 2023-07-18T03:39:17.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.859+0000 7f5b765ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b700b0e20 0x7f5b70159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:17.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.859+0000 7f5b765ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b700b8a60 0x7f5b7015b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:17.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.859+0000 7f5b765ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b700d0b10 0x7f5b700cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:17.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.859+0000 7f5b765ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7015d010 con 0x7f5b700b0e20 2023-07-18T03:39:17.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.859+0000 7f5b765ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7015db90 con 0x7f5b700b8a60 2023-07-18T03:39:17.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.859+0000 7f5b765ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7015d260 con 0x7f5b700d0b10 2023-07-18T03:39:17.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.859+0000 7f5b6ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b700b0e20 0x7f5b70159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:17.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.859+0000 7f5b6ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b700b0e20 0x7f5b70159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54770/0 (socket says 172.21.15.49:54770) 2023-07-18T03:39:17.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.859+0000 7f5b6ffff700 1 -- 172.21.15.49:0/3994788589 learned_addr learned my addr 172.21.15.49:0/3994788589 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:17.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.859+0000 7f5b6f7fe700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b700b8a60 0x7f5b7015b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:17.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.859+0000 7f5b74b9c700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b700d0b10 0x7f5b700cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:17.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.860+0000 7f5b6ffff700 1 -- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b700d0b10 msgr2=0x7f5b700cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:17.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.860+0000 7f5b6ffff700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b700d0b10 0x7f5b700cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.860+0000 7f5b6ffff700 1 -- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b700b8a60 msgr2=0x7f5b7015b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:17.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.860+0000 7f5b6ffff700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b700b8a60 0x7f5b7015b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:17.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.860+0000 7f5b6ffff700 1 -- 172.21.15.49:0/3994788589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b640094a0 con 0x7f5b700b0e20 2023-07-18T03:39:17.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.860+0000 7f5b74b9c700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b700d0b10 0x7f5b700cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:17.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.860+0000 7f5b6ffff700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b700b0e20 0x7f5b70159800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000c520 tx=0x7f5b6000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:17.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.860+0000 7f5b6cff9700 1 -- 172.21.15.49:0/3994788589 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b600052d0 con 0x7f5b700b0e20 2023-07-18T03:39:17.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.860+0000 7f5b6cff9700 1 -- 172.21.15.49:0/3994788589 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b60005430 con 0x7f5b700b0e20 2023-07-18T03:39:17.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.860+0000 7f5b6cff9700 1 -- 172.21.15.49:0/3994788589 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b600038a0 con 0x7f5b700b0e20 2023-07-18T03:39:17.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.860+0000 7f5b765ff700 1 -- 172.21.15.49:0/3994788589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7015d540 con 0x7f5b700b0e20 2023-07-18T03:39:17.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.860+0000 7f5b765ff700 1 -- 172.21.15.49:0/3994788589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b70159d40 con 0x7f5b700b0e20 2023-07-18T03:39:17.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.861+0000 7f5b765ff700 1 -- 172.21.15.49:0/3994788589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b700d23e0 con 0x7f5b700b0e20 2023-07-18T03:39:17.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.862+0000 7f5b6cff9700 1 -- 172.21.15.49:0/3994788589 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5b60003a00 con 0x7f5b700b0e20 2023-07-18T03:39:17.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.863+0000 7f5b6cff9700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5b5c06ace0 0x7f5b5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:17.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.863+0000 7f5b6cff9700 1 -- 172.21.15.49:0/3994788589 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5b600889c0 con 0x7f5b700b0e20 2023-07-18T03:39:17.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.864+0000 7f5b6f7fe700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5b5c06ace0 0x7f5b5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:17.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.865+0000 7f5b6f7fe700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5b5c06ace0 0x7f5b5c06d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5b640090b0 tx=0x7f5b64008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:17.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:17.868+0000 7f5b6cff9700 1 -- 172.21.15.49:0/3994788589 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5b60053a60 con 0x7f5b700b0e20 2023-07-18T03:39:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.078+0000 7f5b765ff700 1 -- 172.21.15.49:0/3994788589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b70074350 con 0x7f5b700b0e20 2023-07-18T03:39:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.078+0000 7f5b6cff9700 1 -- 172.21.15.49:0/3994788589 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b60017020 con 0x7f5b700b0e20 2023-07-18T03:39:18.073 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:39:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.081+0000 7f5b765ff700 1 -- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5b5c06ace0 msgr2=0x7f5b5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.081+0000 7f5b765ff700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5b5c06ace0 0x7f5b5c06d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5b640090b0 tx=0x7f5b64008030 comp rx=0 tx=0).stop 2023-07-18T03:39:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.081+0000 7f5b765ff700 1 -- 172.21.15.49:0/3994788589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b700b0e20 msgr2=0x7f5b70159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.081+0000 7f5b765ff700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b700b0e20 0x7f5b70159800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000c520 tx=0x7f5b6000c830 comp rx=0 tx=0).stop 2023-07-18T03:39:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.081+0000 7f5b765ff700 1 -- 172.21.15.49:0/3994788589 shutdown_connections 2023-07-18T03:39:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.081+0000 7f5b765ff700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b700d0b10 0x7f5b700cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.081+0000 7f5b765ff700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5b5c06ace0 0x7f5b5c06d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.081+0000 7f5b765ff700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b700b0e20 0x7f5b70159800 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.081+0000 7f5b765ff700 1 --2- 172.21.15.49:0/3994788589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b700b8a60 0x7f5b7015b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.081+0000 7f5b765ff700 1 -- 172.21.15.49:0/3994788589 >> 172.21.15.49:0/3994788589 conn(0x7f5b7009c1b0 msgr2=0x7f5b7006a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.081+0000 7f5b765ff700 1 -- 172.21.15.49:0/3994788589 shutdown_connections 2023-07-18T03:39:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.081+0000 7f5b765ff700 1 -- 172.21.15.49:0/3994788589 wait complete. 2023-07-18T03:39:18.171 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-07-18T03:39:18.172 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-07-18T03:39:18.172 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-07-18T03:39:18.172 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-07-18T03:39:18.172 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-07-18T03:39:18.173 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-07-18T03:39:18.173 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-07-18T03:39:18.173 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-07-18T03:39:18.173 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-07-18T03:39:18.174 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-07-18T03:39:18.174 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-07-18T03:39:18.174 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2023-07-18T03:39:18.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.387+0000 7f58611ff700 1 Processor -- start 2023-07-18T03:39:18.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.387+0000 7f58611ff700 1 -- start start 2023-07-18T03:39:18.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.388+0000 7f58611ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f585c15c950 0x7f585c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:18.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.388+0000 7f58611ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f585c0d1220 0x7f585c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:18.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.388+0000 7f58611ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585c0d2ce0 con 0x7f585c0b1e00 2023-07-18T03:39:18.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.388+0000 7f58611ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585c0b3350 con 0x7f585c0d1220 2023-07-18T03:39:18.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.388+0000 7f58611ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585c0b3630 con 0x7f585c15c950 2023-07-18T03:39:18.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.389+0000 7f585ad9d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f585c15c950 0x7f585c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:18.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.389+0000 7f585ad9d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f585c15c950 0x7f585c0b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:50960/0 (socket says 172.21.15.49:50960) 2023-07-18T03:39:18.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.389+0000 7f585ad9d700 1 -- 172.21.15.49:0/1013127733 learned_addr learned my addr 172.21.15.49:0/1013127733 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:18.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.389+0000 7f585a59c700 1 --1- 172.21.15.49:0/1013127733 >> v1:172.21.15.107:6789/0 conn(0x7f585c0b1e00 0x7f585c0d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:33640/0 (socket says 172.21.15.49:33640) 2023-07-18T03:39:18.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.389+0000 7f585b59e700 1 --2- 172.21.15.49:0/1013127733 >> v2:172.21.15.107:3300/0 conn(0x7f585c0d1220 0x7f585c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.389+0000 7f585b59e700 1 -- 172.21.15.49:0/1013127733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f585c15c950 msgr2=0x7f585c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.389+0000 7f585b59e700 1 --2- 172.21.15.49:0/1013127733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f585c15c950 0x7f585c0b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.389+0000 7f585b59e700 1 -- 172.21.15.49:0/1013127733 >> v1:172.21.15.107:6789/0 conn(0x7f585c0b1e00 legacy=0x7f585c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.389+0000 7f585b59e700 1 -- 172.21.15.49:0/1013127733 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c0b3910 con 0x7f585c0d1220 2023-07-18T03:39:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.389+0000 7f585ad9d700 1 --2- 172.21.15.49:0/1013127733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f585c15c950 0x7f585c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.390+0000 7f585b59e700 1 --2- 172.21.15.49:0/1013127733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f585c0d1220 0x7f585c0afac0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5854009d90 tx=0x7f585400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64b042f64d38eca2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.390+0000 7f5859d9b700 1 -- 172.21.15.49:0/1013127733 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585400e070 con 0x7f585c0d1220 2023-07-18T03:39:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.390+0000 7f5859d9b700 1 -- 172.21.15.49:0/1013127733 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585400bb20 con 0x7f585c0d1220 2023-07-18T03:39:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.390+0000 7f5859d9b700 1 -- 172.21.15.49:0/1013127733 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854012430 con 0x7f585c0d1220 2023-07-18T03:39:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.390+0000 7f58611ff700 1 -- 172.21.15.49:0/1013127733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f585c0d1220 msgr2=0x7f585c0afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.390+0000 7f58611ff700 1 --2- 172.21.15.49:0/1013127733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f585c0d1220 0x7f585c0afac0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5854009d90 tx=0x7f585400b330 comp rx=0 tx=0).stop 2023-07-18T03:39:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.390+0000 7f58611ff700 1 -- 172.21.15.49:0/1013127733 shutdown_connections 2023-07-18T03:39:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.390+0000 7f58611ff700 1 --2- 172.21.15.49:0/1013127733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f585c0d1220 0x7f585c0afac0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.390+0000 7f58611ff700 1 --2- 172.21.15.49:0/1013127733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f585c15c950 0x7f585c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.390+0000 7f58611ff700 1 -- 172.21.15.49:0/1013127733 >> 172.21.15.49:0/1013127733 conn(0x7f585c09c1d0 msgr2=0x7f585c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:18.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.391+0000 7f58611ff700 1 -- 172.21.15.49:0/1013127733 shutdown_connections 2023-07-18T03:39:18.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.391+0000 7f58611ff700 1 -- 172.21.15.49:0/1013127733 wait complete. 2023-07-18T03:39:18.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.392+0000 7f58611ff700 1 Processor -- start 2023-07-18T03:39:18.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.392+0000 7f58611ff700 1 -- start start 2023-07-18T03:39:18.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.393+0000 7f58611ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f585c0b1e00 0x7f585c0b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:18.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.393+0000 7f58611ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f585c0d1220 0x7f585c1f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:18.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.393+0000 7f58611ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f585c15c950 0x7f585c1452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:18.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.393+0000 7f58611ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585c0b65a0 con 0x7f585c0d1220 2023-07-18T03:39:18.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.393+0000 7f58611ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585c0d0430 con 0x7f585c15c950 2023-07-18T03:39:18.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.393+0000 7f58611ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585c0c9430 con 0x7f585c0b1e00 2023-07-18T03:39:18.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.394+0000 7f585a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f585c0d1220 0x7f585c1f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:18.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.394+0000 7f585a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f585c0d1220 0x7f585c1f1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54782/0 (socket says 172.21.15.49:54782) 2023-07-18T03:39:18.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.394+0000 7f585a59c700 1 -- 172.21.15.49:0/3306510830 learned_addr learned my addr 172.21.15.49:0/3306510830 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:18.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.394+0000 7f585ad9d700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f585c0b1e00 0x7f585c0b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:18.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.394+0000 7f585b59e700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f585c15c950 0x7f585c1452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:18.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.394+0000 7f585a59c700 1 -- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f585c0b1e00 msgr2=0x7f585c0b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:18.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.394+0000 7f585a59c700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f585c0b1e00 0x7f585c0b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.394+0000 7f585a59c700 1 -- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f585c15c950 msgr2=0x7f585c1452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:18.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.394+0000 7f585a59c700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f585c15c950 0x7f585c1452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.394+0000 7f585a59c700 1 -- 172.21.15.49:0/3306510830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c009500 con 0x7f585c0d1220 2023-07-18T03:39:18.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.395+0000 7f585a59c700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f585c0d1220 0x7f585c1f1da0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f585000c550 tx=0x7f585000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:18.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.395+0000 7f58437fe700 1 -- 172.21.15.49:0/3306510830 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850005cb0 con 0x7f585c0d1220 2023-07-18T03:39:18.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.395+0000 7f58437fe700 1 -- 172.21.15.49:0/3306510830 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5850005e10 con 0x7f585c0d1220 2023-07-18T03:39:18.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.395+0000 7f58611ff700 1 -- 172.21.15.49:0/3306510830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5854009a70 con 0x7f585c0d1220 2023-07-18T03:39:18.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.395+0000 7f58437fe700 1 -- 172.21.15.49:0/3306510830 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58500072d0 con 0x7f585c0d1220 2023-07-18T03:39:18.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.395+0000 7f58611ff700 1 -- 172.21.15.49:0/3306510830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585c0c8950 con 0x7f585c0d1220 2023-07-18T03:39:18.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.396+0000 7f58611ff700 1 -- 172.21.15.49:0/3306510830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585c04e280 con 0x7f585c0d1220 2023-07-18T03:39:18.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.397+0000 7f58437fe700 1 -- 172.21.15.49:0/3306510830 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5850007430 con 0x7f585c0d1220 2023-07-18T03:39:18.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.398+0000 7f58437fe700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f584806ace0 0x7f584806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:18.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.398+0000 7f58437fe700 1 -- 172.21.15.49:0/3306510830 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5850088790 con 0x7f585c0d1220 2023-07-18T03:39:18.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.398+0000 7f585ad9d700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f584806ace0 0x7f584806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:18.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.399+0000 7f585ad9d700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f584806ace0 0x7f584806d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f584c006fd0 tx=0x7f584c009290 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:18.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.402+0000 7f58437fe700 1 -- 172.21.15.49:0/3306510830 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5850053850 con 0x7f585c0d1220 2023-07-18T03:39:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.694+0000 7f58611ff700 1 -- 172.21.15.49:0/3306510830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7f585c158b60 con 0x7f585c0d1220 2023-07-18T03:39:18.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.924+0000 7f58437fe700 1 -- 172.21.15.49:0/3306510830 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v59) v1 ==== 184+0+0 (secure 0 0 0) 0x7f5850056e70 con 0x7f585c0d1220 2023-07-18T03:39:18.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.927+0000 7f58611ff700 1 -- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f584806ace0 msgr2=0x7f584806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:18.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.927+0000 7f58611ff700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f584806ace0 0x7f584806d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f584c006fd0 tx=0x7f584c009290 comp rx=0 tx=0).stop 2023-07-18T03:39:18.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.927+0000 7f58611ff700 1 -- 172.21.15.49:0/3306510830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f585c0d1220 msgr2=0x7f585c1f1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:18.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.927+0000 7f58611ff700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f585c0d1220 0x7f585c1f1da0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f585000c550 tx=0x7f585000c860 comp rx=0 tx=0).stop 2023-07-18T03:39:18.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.927+0000 7f58611ff700 1 -- 172.21.15.49:0/3306510830 shutdown_connections 2023-07-18T03:39:18.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.927+0000 7f58611ff700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f585c15c950 0x7f585c1452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.927+0000 7f58611ff700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f585c0b1e00 0x7f585c0b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.927+0000 7f58611ff700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f584806ace0 0x7f584806d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.927+0000 7f58611ff700 1 --2- 172.21.15.49:0/3306510830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f585c0d1220 0x7f585c1f1da0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:18.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.927+0000 7f58611ff700 1 -- 172.21.15.49:0/3306510830 >> 172.21.15.49:0/3306510830 conn(0x7f585c09c1d0 msgr2=0x7f585c06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:18.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.927+0000 7f58611ff700 1 -- 172.21.15.49:0/3306510830 shutdown_connections 2023-07-18T03:39:18.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:18.927+0000 7f58611ff700 1 -- 172.21.15.49:0/3306510830 wait complete. 2023-07-18T03:39:18.941 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-18T03:39:19.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.154+0000 7f68083ff700 1 Processor -- start 2023-07-18T03:39:19.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.154+0000 7f68083ff700 1 -- start start 2023-07-18T03:39:19.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.155+0000 7f68083ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68000d0f90 0x7f68000b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:19.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.155+0000 7f68083ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68000b8e30 con 0x7f68000d0b10 2023-07-18T03:39:19.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.155+0000 7f68083ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68000afaa0 con 0x7f68000b8a60 2023-07-18T03:39:19.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.155+0000 7f68083ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68000afd80 con 0x7f68000d0f90 2023-07-18T03:39:19.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.155+0000 7f680599a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f68000d0b10 0x7f68000b0e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55440/0 (socket says 172.21.15.49:55440) 2023-07-18T03:39:19.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.156+0000 7f680599a700 1 -- 172.21.15.49:0/2418132801 learned_addr learned my addr 172.21.15.49:0/2418132801 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:19.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.156+0000 7f680699c700 1 --2- 172.21.15.49:0/2418132801 >> v2:172.21.15.49:3300/0 conn(0x7f68000d0f90 0x7f68000b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:19.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.156+0000 7f6805199700 1 -- 172.21.15.49:0/2418132801 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52220551 0 0) 0x7f68000b8e30 con 0x7f68000d0b10 2023-07-18T03:39:19.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.156+0000 7f680699c700 1 -- 172.21.15.49:0/2418132801 >> v1:172.21.15.107:6789/0 conn(0x7f68000b8a60 legacy=0x7f68000b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:19.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.156+0000 7f680699c700 1 -- 172.21.15.49:0/2418132801 >> v1:172.21.15.49:6789/0 conn(0x7f68000d0b10 legacy=0x7f68000b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:19.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.156+0000 7f680699c700 1 -- 172.21.15.49:0/2418132801 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68000b0060 con 0x7f68000d0f90 2023-07-18T03:39:19.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.156+0000 7f680699c700 1 --2- 172.21.15.49:0/2418132801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68000d0f90 0x7f68000b1530 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009d90 tx=0x7f67f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26d09f25f9ebbcf server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:19.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.156+0000 7f6805199700 1 -- 172.21.15.49:0/2418132801 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800e070 con 0x7f68000d0f90 2023-07-18T03:39:19.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.157+0000 7f6805199700 1 -- 172.21.15.49:0/2418132801 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f800bbd0 con 0x7f68000d0f90 2023-07-18T03:39:19.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.157+0000 7f6805199700 1 -- 172.21.15.49:0/2418132801 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8012430 con 0x7f68000d0f90 2023-07-18T03:39:19.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.157+0000 7f68083ff700 1 -- 172.21.15.49:0/2418132801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68000d0f90 msgr2=0x7f68000b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:19.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.157+0000 7f68083ff700 1 --2- 172.21.15.49:0/2418132801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68000d0f90 0x7f68000b1530 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009d90 tx=0x7f67f800b330 comp rx=0 tx=0).stop 2023-07-18T03:39:19.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.157+0000 7f68083ff700 1 -- 172.21.15.49:0/2418132801 shutdown_connections 2023-07-18T03:39:19.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.157+0000 7f68083ff700 1 --2- 172.21.15.49:0/2418132801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68000d0f90 0x7f68000b1530 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:19.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.157+0000 7f68083ff700 1 -- 172.21.15.49:0/2418132801 >> 172.21.15.49:0/2418132801 conn(0x7f680009c1b0 msgr2=0x7f68000d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:19.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.157+0000 7f68083ff700 1 -- 172.21.15.49:0/2418132801 shutdown_connections 2023-07-18T03:39:19.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.157+0000 7f68083ff700 1 -- 172.21.15.49:0/2418132801 wait complete. 2023-07-18T03:39:19.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.160+0000 7f68083ff700 1 Processor -- start 2023-07-18T03:39:19.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.160+0000 7f68083ff700 1 -- start start 2023-07-18T03:39:19.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.161+0000 7f68083ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68000b8a60 0x7f68001470a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.161+0000 7f68083ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68000d0b10 0x7f68000b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.161+0000 7f68083ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68000d0f90 0x7f68000b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.161+0000 7f68083ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680015f7f0 con 0x7f68000d0f90 2023-07-18T03:39:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.161+0000 7f68083ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68000d01d0 con 0x7f68000b8a60 2023-07-18T03:39:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.161+0000 7f68083ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680015fa40 con 0x7f68000d0b10 2023-07-18T03:39:19.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.161+0000 7f680699c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68000d0f90 0x7f68000b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.161+0000 7f680699c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68000d0f90 0x7f68000b6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37714/0 (socket says 172.21.15.49:37714) 2023-07-18T03:39:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.161+0000 7f680699c700 1 -- 172.21.15.49:0/1452453227 learned_addr learned my addr 172.21.15.49:0/1452453227 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.162+0000 7f680619b700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68000b8a60 0x7f68001470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.162+0000 7f680699c700 1 -- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68000d0b10 msgr2=0x7f68000b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.162+0000 7f680599a700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68000d0b10 0x7f68000b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:19.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.162+0000 7f680699c700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68000d0b10 0x7f68000b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.162+0000 7f680699c700 1 -- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68000b8a60 msgr2=0x7f68001470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.162+0000 7f680699c700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68000b8a60 0x7f68001470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.162+0000 7f680699c700 1 -- 172.21.15.49:0/1452453227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f8009a70 con 0x7f68000d0f90 2023-07-18T03:39:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.162+0000 7f680599a700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68000d0b10 0x7f68000b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:19.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.162+0000 7f680699c700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68000d0f90 0x7f68000b6ca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f67f8014040 tx=0x7f67f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:19.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.162+0000 7f67f6ffd700 1 -- 172.21.15.49:0/1452453227 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f801b810 con 0x7f68000d0f90 2023-07-18T03:39:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.163+0000 7f67f6ffd700 1 -- 172.21.15.49:0/1452453227 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f8020070 con 0x7f68000d0f90 2023-07-18T03:39:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.163+0000 7f68083ff700 1 -- 172.21.15.49:0/1452453227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680015fd20 con 0x7f68000d0f90 2023-07-18T03:39:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.163+0000 7f67f6ffd700 1 -- 172.21.15.49:0/1452453227 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f801bbb0 con 0x7f68000d0f90 2023-07-18T03:39:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.163+0000 7f68083ff700 1 -- 172.21.15.49:0/1452453227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68001477c0 con 0x7f68000d0f90 2023-07-18T03:39:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.164+0000 7f68083ff700 1 -- 172.21.15.49:0/1452453227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6800074350 con 0x7f68000d0f90 2023-07-18T03:39:19.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.165+0000 7f67f6ffd700 1 -- 172.21.15.49:0/1452453227 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f67f801bd10 con 0x7f68000d0f90 2023-07-18T03:39:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.166+0000 7f67f6ffd700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f67e406ace0 0x7f67e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.166+0000 7f67f6ffd700 1 -- 172.21.15.49:0/1452453227 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67f8022070 con 0x7f68000d0f90 2023-07-18T03:39:19.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.166+0000 7f680619b700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f67e406ace0 0x7f67e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.167+0000 7f680619b700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f67e406ace0 0x7f67e406d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f67fc000f60 tx=0x7f67fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:19.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.170+0000 7f67f6ffd700 1 -- 172.21.15.49:0/1452453227 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f67f805a3e0 con 0x7f68000d0f90 2023-07-18T03:39:19.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.379+0000 7f68083ff700 1 -- 172.21.15.49:0/1452453227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f68001477c0 con 0x7f68000d0f90 2023-07-18T03:39:19.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.383+0000 7f67f6ffd700 1 -- 172.21.15.49:0/1452453227 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f68001477c0 con 0x7f68000d0f90 2023-07-18T03:39:19.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.385+0000 7f68083ff700 1 -- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f67e406ace0 msgr2=0x7f67e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:19.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.385+0000 7f68083ff700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f67e406ace0 0x7f67e406d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f67fc000f60 tx=0x7f67fc009040 comp rx=0 tx=0).stop 2023-07-18T03:39:19.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.385+0000 7f68083ff700 1 -- 172.21.15.49:0/1452453227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68000d0f90 msgr2=0x7f68000b6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:19.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.385+0000 7f68083ff700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68000d0f90 0x7f68000b6ca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f67f8014040 tx=0x7f67f8004ad0 comp rx=0 tx=0).stop 2023-07-18T03:39:19.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.385+0000 7f68083ff700 1 -- 172.21.15.49:0/1452453227 shutdown_connections 2023-07-18T03:39:19.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.386+0000 7f68083ff700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f67e406ace0 0x7f67e406d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:19.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.386+0000 7f68083ff700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68000d0f90 0x7f68000b6ca0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:19.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.386+0000 7f68083ff700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68000b8a60 0x7f68001470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:19.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.386+0000 7f68083ff700 1 --2- 172.21.15.49:0/1452453227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68000d0b10 0x7f68000b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:19.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.386+0000 7f68083ff700 1 -- 172.21.15.49:0/1452453227 >> 172.21.15.49:0/1452453227 conn(0x7f680009c1b0 msgr2=0x7f680015c260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:19.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.386+0000 7f68083ff700 1 -- 172.21.15.49:0/1452453227 shutdown_connections 2023-07-18T03:39:19.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:19.386+0000 7f68083ff700 1 -- 172.21.15.49:0/1452453227 wait complete. 2023-07-18T03:39:19.399 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-18T03:39:19.400 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-18T03:39:19.511 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-18T03:39:19.617 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-18T03:39:19.617 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:39:19.618 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:19.662 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:39:19.662 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:19.746 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:39:19.746 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:19.830 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:20.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.083+0000 7f19eefff700 1 Processor -- start 2023-07-18T03:39:20.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.083+0000 7f19eefff700 1 -- start start 2023-07-18T03:39:20.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.084+0000 7f19eefff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19e80b5d90 0x7f19e80b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.084+0000 7f19eefff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19e80d0b10 0x7f19e80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.084+0000 7f19eefff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e80b7f50 con 0x7f19e80b4510 2023-07-18T03:39:20.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.084+0000 7f19eefff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e80b48e0 con 0x7f19e80b5d90 2023-07-18T03:39:20.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.084+0000 7f19eefff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80d14d0 con 0x7f19e80d0b10 2023-07-18T03:39:20.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.084+0000 7f19e7fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19e80b5d90 0x7f19e80b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.084+0000 7f19ecd9b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f19e80b4510 0x7f19e80cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55448/0 (socket says 172.21.15.49:55448) 2023-07-18T03:39:20.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.085+0000 7f19ecd9b700 1 -- 172.21.15.49:0/2590018648 learned_addr learned my addr 172.21.15.49:0/2590018648 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:20.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.085+0000 7f19ed59c700 1 --2- 172.21.15.49:0/2590018648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19e80d0b10 0x7f19e80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.085+0000 7f19e7fff700 1 -- 172.21.15.49:0/2590018648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19e80d0b10 msgr2=0x7f19e80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:20.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.085+0000 7f19e7fff700 1 --2- 172.21.15.49:0/2590018648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19e80d0b10 0x7f19e80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.085+0000 7f19e7fff700 1 -- 172.21.15.49:0/2590018648 >> v1:172.21.15.49:6789/0 conn(0x7f19e80b4510 legacy=0x7f19e80cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.085+0000 7f19e7fff700 1 -- 172.21.15.49:0/2590018648 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e80cdca0 con 0x7f19e80b5d90 2023-07-18T03:39:20.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.085+0000 7f19e77fe700 1 -- 172.21.15.49:0/2590018648 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160302973 0 0) 0x7f19e80b7f50 con 0x7f19e80b4510 2023-07-18T03:39:20.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.085+0000 7f19e7fff700 1 --2- 172.21.15.49:0/2590018648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e80b5d90 0x7f19e80b6c80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009840 tx=0x7f19d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f22f9c7060281c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:20.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.086+0000 7f19e77fe700 1 -- 172.21.15.49:0/2590018648 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4011830 con 0x7f19e80b5d90 2023-07-18T03:39:20.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.086+0000 7f19e77fe700 1 -- 172.21.15.49:0/2590018648 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4011990 con 0x7f19e80b5d90 2023-07-18T03:39:20.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.086+0000 7f19e77fe700 1 -- 172.21.15.49:0/2590018648 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4011d20 con 0x7f19e80b5d90 2023-07-18T03:39:20.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.086+0000 7f19eefff700 1 -- 172.21.15.49:0/2590018648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e80b5d90 msgr2=0x7f19e80b6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.086+0000 7f19eefff700 1 --2- 172.21.15.49:0/2590018648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e80b5d90 0x7f19e80b6c80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009840 tx=0x7f19d4009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:20.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.086+0000 7f19eefff700 1 -- 172.21.15.49:0/2590018648 shutdown_connections 2023-07-18T03:39:20.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.086+0000 7f19eefff700 1 --2- 172.21.15.49:0/2590018648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19e80d0b10 0x7f19e80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.086+0000 7f19eefff700 1 --2- 172.21.15.49:0/2590018648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e80b5d90 0x7f19e80b6c80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.086+0000 7f19eefff700 1 -- 172.21.15.49:0/2590018648 >> 172.21.15.49:0/2590018648 conn(0x7f19e809c1d0 msgr2=0x7f19e80d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:20.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.086+0000 7f19eefff700 1 -- 172.21.15.49:0/2590018648 shutdown_connections 2023-07-18T03:39:20.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.086+0000 7f19eefff700 1 -- 172.21.15.49:0/2590018648 wait complete. 2023-07-18T03:39:20.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.088+0000 7f19eefff700 1 Processor -- start 2023-07-18T03:39:20.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.088+0000 7f19eefff700 1 -- start start 2023-07-18T03:39:20.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.089+0000 7f19eefff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e80b4510 0x7f19e81524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.089+0000 7f19eefff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19e80b5d90 0x7f19e80c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.089+0000 7f19eefff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e80d0b10 0x7f19e815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.089+0000 7f19eefff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e8152a10 con 0x7f19e80d0b10 2023-07-18T03:39:20.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.089+0000 7f19eefff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80ca520 con 0x7f19e80b4510 2023-07-18T03:39:20.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.089+0000 7f19eefff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80ca800 con 0x7f19e80b5d90 2023-07-18T03:39:20.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.089+0000 7f19ed59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e80d0b10 0x7f19e815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.089+0000 7f19ed59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e80d0b10 0x7f19e815ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37736/0 (socket says 172.21.15.49:37736) 2023-07-18T03:39:20.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.089+0000 7f19ed59c700 1 -- 172.21.15.49:0/3702148205 learned_addr learned my addr 172.21.15.49:0/3702148205 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:20.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19ecd9b700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e80b4510 0x7f19e81524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19e7fff700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19e80b5d90 0x7f19e80c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19ed59c700 1 -- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19e80b5d90 msgr2=0x7f19e80c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19ed59c700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19e80b5d90 0x7f19e80c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19ed59c700 1 -- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e80b4510 msgr2=0x7f19e81524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19ed59c700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e80b4510 0x7f19e81524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19ed59c700 1 -- 172.21.15.49:0/3702148205 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40094a0 con 0x7f19e80d0b10 2023-07-18T03:39:20.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19ecd9b700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e80b4510 0x7f19e81524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:20.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19ed59c700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e80d0b10 0x7f19e815ab70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f19e0007e40 tx=0x7f19e000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:20.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19e57fa700 1 -- 172.21.15.49:0/3702148205 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e000ede0 con 0x7f19e80d0b10 2023-07-18T03:39:20.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19e57fa700 1 -- 172.21.15.49:0/3702148205 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e0014420 con 0x7f19e80d0b10 2023-07-18T03:39:20.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19eefff700 1 -- 172.21.15.49:0/3702148205 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e80caae0 con 0x7f19e80d0b10 2023-07-18T03:39:20.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.090+0000 7f19e57fa700 1 -- 172.21.15.49:0/3702148205 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e00147d0 con 0x7f19e80d0b10 2023-07-18T03:39:20.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.091+0000 7f19eefff700 1 -- 172.21.15.49:0/3702148205 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e80cb0a0 con 0x7f19e80d0b10 2023-07-18T03:39:20.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.092+0000 7f19eefff700 1 -- 172.21.15.49:0/3702148205 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e806a3e0 con 0x7f19e80d0b10 2023-07-18T03:39:20.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.093+0000 7f19e57fa700 1 -- 172.21.15.49:0/3702148205 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f19e0014930 con 0x7f19e80d0b10 2023-07-18T03:39:20.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.094+0000 7f19e57fa700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f19cc06ace0 0x7f19cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.094+0000 7f19e57fa700 1 -- 172.21.15.49:0/3702148205 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f19e0089b90 con 0x7f19e80d0b10 2023-07-18T03:39:20.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.094+0000 7f19ecd9b700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f19cc06ace0 0x7f19cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.095+0000 7f19ecd9b700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f19cc06ace0 0x7f19cc06d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0017b0 tx=0x7f19dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:20.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.099+0000 7f19e57fa700 1 -- 172.21.15.49:0/3702148205 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f19e0058810 con 0x7f19e80d0b10 2023-07-18T03:39:20.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.312+0000 7f19eefff700 1 -- 172.21.15.49:0/3702148205 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19e80cb290 con 0x7f19e80d0b10 2023-07-18T03:39:20.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.313+0000 7f19e57fa700 1 -- 172.21.15.49:0/3702148205 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f19e00583a0 con 0x7f19e80d0b10 2023-07-18T03:39:20.309 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:20.309 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:38:37.510006+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:20.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.315+0000 7f19eefff700 1 -- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f19cc06ace0 msgr2=0x7f19cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.316+0000 7f19eefff700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f19cc06ace0 0x7f19cc06d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0017b0 tx=0x7f19dc009040 comp rx=0 tx=0).stop 2023-07-18T03:39:20.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.316+0000 7f19eefff700 1 -- 172.21.15.49:0/3702148205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e80d0b10 msgr2=0x7f19e815ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.316+0000 7f19eefff700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e80d0b10 0x7f19e815ab70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f19e0007e40 tx=0x7f19e000e3c0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.316+0000 7f19eefff700 1 -- 172.21.15.49:0/3702148205 shutdown_connections 2023-07-18T03:39:20.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.316+0000 7f19eefff700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f19cc06ace0 0x7f19cc06d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.316+0000 7f19eefff700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19e80d0b10 0x7f19e815ab70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.316+0000 7f19eefff700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e80b4510 0x7f19e81524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.316+0000 7f19eefff700 1 --2- 172.21.15.49:0/3702148205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19e80b5d90 0x7f19e80c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.316+0000 7f19eefff700 1 -- 172.21.15.49:0/3702148205 >> 172.21.15.49:0/3702148205 conn(0x7f19e809c1d0 msgr2=0x7f19e80cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:20.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.316+0000 7f19eefff700 1 -- 172.21.15.49:0/3702148205 shutdown_connections 2023-07-18T03:39:20.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.316+0000 7f19eefff700 1 -- 172.21.15.49:0/3702148205 wait complete. 2023-07-18T03:39:20.329 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:38:33.773236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:38:35.023692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:39:20.331 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:39:20.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.547+0000 7f1d029ff700 1 Processor -- start 2023-07-18T03:39:20.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.547+0000 7f1d029ff700 1 -- start start 2023-07-18T03:39:20.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.548+0000 7f1d029ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc06c9f0 0x7f1cfc0b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.548+0000 7f1d029ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cfc06bcc0 con 0x7f1cfc15d9d0 2023-07-18T03:39:20.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.548+0000 7f1d029ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cfc06c4e0 con 0x7f1cfc0b1e50 2023-07-18T03:39:20.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.548+0000 7f1d029ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc15dda0 con 0x7f1cfc06c9f0 2023-07-18T03:39:20.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.548+0000 7f1cfbfff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1cfc15d9d0 0x7f1cfc06b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55458/0 (socket says 172.21.15.49:55458) 2023-07-18T03:39:20.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.548+0000 7f1cfbfff700 1 -- 172.21.15.49:0/736502507 learned_addr learned my addr 172.21.15.49:0/736502507 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:20.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfb7fe700 1 --2- 172.21.15.49:0/736502507 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc06c9f0 0x7f1cfc0b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341981571 0 0) 0x7f1cfc06bcc0 con 0x7f1cfc15d9d0 2023-07-18T03:39:20.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ce00035b0 con 0x7f1cfc15d9d0 2023-07-18T03:39:20.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722104715 0 0) 0x7f1cfc06c4e0 con 0x7f1cfc0b1e50 2023-07-18T03:39:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cfc06bcc0 con 0x7f1cfc0b1e50 2023-07-18T03:39:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2406476812 0 0) 0x7f1ce00035b0 con 0x7f1cfc15d9d0 2023-07-18T03:39:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc06c9f0 msgr2=0x7f1cfc0b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 --2- 172.21.15.49:0/736502507 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc06c9f0 0x7f1cfc0b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 >> v1:172.21.15.107:6789/0 conn(0x7f1cfc0b1e50 legacy=0x7f1cfc071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc06bf20 con 0x7f1cfc15d9d0 2023-07-18T03:39:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f1cec001670 con 0x7f1cfc15d9d0 2023-07-18T03:39:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 >> v1:172.21.15.49:6789/0 conn(0x7f1cfc15d9d0 legacy=0x7f1cfc06b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 --2- 172.21.15.49:0/736502507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce0003bc0 0x7f1ce0003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 --2- 172.21.15.49:0/736502507 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc06c9f0 0x7f1ce0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 --2- 172.21.15.49:0/736502507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ce0004d90 0x7f1ce0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc15dda0 con 0x7f1ce0003bc0 2023-07-18T03:39:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce000bba0 con 0x7f1ce0004d90 2023-07-18T03:39:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.549+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce000bd30 con 0x7f1cfc06c9f0 2023-07-18T03:39:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.550+0000 7f1cfbfff700 1 --2- 172.21.15.49:0/736502507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce0003bc0 0x7f1ce0003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.550+0000 7f1cfb7fe700 1 --2- 172.21.15.49:0/736502507 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc06c9f0 0x7f1ce0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.550+0000 7f1d00f9c700 1 --2- 172.21.15.49:0/736502507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ce0004d90 0x7f1ce0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.550+0000 7f1cfbfff700 1 -- 172.21.15.49:0/736502507 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc06c9f0 msgr2=0x7f1ce0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.550+0000 7f1cfbfff700 1 --2- 172.21.15.49:0/736502507 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc06c9f0 0x7f1ce0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.550+0000 7f1cfbfff700 1 -- 172.21.15.49:0/736502507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ce0004d90 msgr2=0x7f1ce0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.550+0000 7f1cfbfff700 1 --2- 172.21.15.49:0/736502507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ce0004d90 0x7f1ce0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.550+0000 7f1cfbfff700 1 -- 172.21.15.49:0/736502507 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce00035b0 con 0x7f1ce0003bc0 2023-07-18T03:39:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.550+0000 7f1cfbfff700 1 --2- 172.21.15.49:0/736502507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce0003bc0 0x7f1ce0003fd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00c160 tx=0x7f1ce0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783753380f9579be server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.551+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00cca0 con 0x7f1ce0003bc0 2023-07-18T03:39:20.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.551+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec00ce00 con 0x7f1ce0003bc0 2023-07-18T03:39:20.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.551+0000 7f1cfaffd700 1 -- 172.21.15.49:0/736502507 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec005ba0 con 0x7f1ce0003bc0 2023-07-18T03:39:20.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.551+0000 7f1d029ff700 1 -- 172.21.15.49:0/736502507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce0003bc0 msgr2=0x7f1ce0003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.551+0000 7f1d029ff700 1 --2- 172.21.15.49:0/736502507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce0003bc0 0x7f1ce0003fd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00c160 tx=0x7f1ce0005880 comp rx=0 tx=0).stop 2023-07-18T03:39:20.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.551+0000 7f1d00f9c700 1 -- 172.21.15.49:0/736502507 reap_dead start 2023-07-18T03:39:20.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.551+0000 7f1d029ff700 1 -- 172.21.15.49:0/736502507 shutdown_connections 2023-07-18T03:39:20.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.551+0000 7f1d029ff700 1 -- 172.21.15.49:0/736502507 >> 172.21.15.49:0/736502507 conn(0x7f1cfc0b05d0 msgr2=0x7f1cfc0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:20.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.551+0000 7f1d029ff700 1 -- 172.21.15.49:0/736502507 shutdown_connections 2023-07-18T03:39:20.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.551+0000 7f1d029ff700 1 -- 172.21.15.49:0/736502507 wait complete. 2023-07-18T03:39:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.553+0000 7f1d029ff700 1 Processor -- start 2023-07-18T03:39:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.553+0000 7f1d029ff700 1 -- start start 2023-07-18T03:39:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.554+0000 7f1d029ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce0003bc0 0x7f1cfc1ef720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.554+0000 7f1d029ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cfc0c2a90 0x7f1cfc1f3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.554+0000 7f1d029ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc1f3890 0x7f1cfc158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.554+0000 7f1d029ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc1f3d10 con 0x7f1ce0003bc0 2023-07-18T03:39:20.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.554+0000 7f1d029ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc1e4360 con 0x7f1cfc0c2a90 2023-07-18T03:39:20.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.554+0000 7f1d029ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc1efc60 con 0x7f1cfc1f3890 2023-07-18T03:39:20.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.554+0000 7f1cfbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce0003bc0 0x7f1cfc1ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.554+0000 7f1cfbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce0003bc0 0x7f1cfc1ef720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37764/0 (socket says 172.21.15.49:37764) 2023-07-18T03:39:20.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.554+0000 7f1cfbfff700 1 -- 172.21.15.49:0/1262664704 learned_addr learned my addr 172.21.15.49:0/1262664704 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:20.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.555+0000 7f1d00f9c700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc1f3890 0x7f1cfc158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.555+0000 7f1cfb7fe700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cfc0c2a90 0x7f1cfc1f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.555+0000 7f1cfbfff700 1 -- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc1f3890 msgr2=0x7f1cfc158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.555+0000 7f1cfbfff700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc1f3890 0x7f1cfc158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.555+0000 7f1cfbfff700 1 -- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cfc0c2a90 msgr2=0x7f1cfc1f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.555+0000 7f1cfbfff700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cfc0c2a90 0x7f1cfc1f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.555+0000 7f1cfbfff700 1 -- 172.21.15.49:0/1262664704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc1e3d30 con 0x7f1ce0003bc0 2023-07-18T03:39:20.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.555+0000 7f1cfbfff700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce0003bc0 0x7f1cfc1ef720 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00c750 tx=0x7f1cec001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:20.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.555+0000 7f1cf8ff9700 1 -- 172.21.15.49:0/1262664704 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec004c80 con 0x7f1ce0003bc0 2023-07-18T03:39:20.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.555+0000 7f1cf8ff9700 1 -- 172.21.15.49:0/1262664704 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec004de0 con 0x7f1ce0003bc0 2023-07-18T03:39:20.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.555+0000 7f1cf8ff9700 1 -- 172.21.15.49:0/1262664704 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0072d0 con 0x7f1ce0003bc0 2023-07-18T03:39:20.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.555+0000 7f1d029ff700 1 -- 172.21.15.49:0/1262664704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cfc159a50 con 0x7f1ce0003bc0 2023-07-18T03:39:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.556+0000 7f1d029ff700 1 -- 172.21.15.49:0/1262664704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc1e3ff0 con 0x7f1ce0003bc0 2023-07-18T03:39:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.557+0000 7f1d029ff700 1 -- 172.21.15.49:0/1262664704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cfc074350 con 0x7f1ce0003bc0 2023-07-18T03:39:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.558+0000 7f1cf8ff9700 1 -- 172.21.15.49:0/1262664704 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1cec019070 con 0x7f1ce0003bc0 2023-07-18T03:39:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.558+0000 7f1cf8ff9700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1ce406ace0 0x7f1ce406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.559+0000 7f1cf8ff9700 1 -- 172.21.15.49:0/1262664704 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1cec08a3d0 con 0x7f1ce0003bc0 2023-07-18T03:39:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.559+0000 7f1cfb7fe700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1ce406ace0 0x7f1ce406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.560+0000 7f1cfb7fe700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1ce406ace0 0x7f1ce406d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0c5710 tx=0x7f1ce8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:20.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.563+0000 7f1cf8ff9700 1 -- 172.21.15.49:0/1262664704 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1cec055470 con 0x7f1ce0003bc0 2023-07-18T03:39:20.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.767+0000 7f1d029ff700 1 -- 172.21.15.49:0/1262664704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1cfc0b3da0 con 0x7f1ce0003bc0 2023-07-18T03:39:20.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.768+0000 7f1cf8ff9700 1 -- 172.21.15.49:0/1262664704 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1cec058a90 con 0x7f1ce0003bc0 2023-07-18T03:39:20.763 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:39:20.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.770+0000 7f1d029ff700 1 -- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1ce406ace0 msgr2=0x7f1ce406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.770+0000 7f1d029ff700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1ce406ace0 0x7f1ce406d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0c5710 tx=0x7f1ce8008040 comp rx=0 tx=0).stop 2023-07-18T03:39:20.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.770+0000 7f1d029ff700 1 -- 172.21.15.49:0/1262664704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce0003bc0 msgr2=0x7f1cfc1ef720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.770+0000 7f1d029ff700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce0003bc0 0x7f1cfc1ef720 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00c750 tx=0x7f1cec001930 comp rx=0 tx=0).stop 2023-07-18T03:39:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.770+0000 7f1d029ff700 1 -- 172.21.15.49:0/1262664704 shutdown_connections 2023-07-18T03:39:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.770+0000 7f1d029ff700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cfc1f3890 0x7f1cfc158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.770+0000 7f1d029ff700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1ce406ace0 0x7f1ce406d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.770+0000 7f1d029ff700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ce0003bc0 0x7f1cfc1ef720 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.770+0000 7f1d029ff700 1 --2- 172.21.15.49:0/1262664704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cfc0c2a90 0x7f1cfc1f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:20.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.770+0000 7f1d029ff700 1 -- 172.21.15.49:0/1262664704 >> 172.21.15.49:0/1262664704 conn(0x7f1cfc0b05d0 msgr2=0x7f1cfc0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:20.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.770+0000 7f1d029ff700 1 -- 172.21.15.49:0/1262664704 shutdown_connections 2023-07-18T03:39:20.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:20.770+0000 7f1d029ff700 1 -- 172.21.15.49:0/1262664704 wait complete. 2023-07-18T03:39:20.783 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:39:20.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.000+0000 7f5019bff700 1 Processor -- start 2023-07-18T03:39:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.000+0000 7f5019bff700 1 -- start start 2023-07-18T03:39:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.001+0000 7f5019bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f50140d0b10 0x7f50140d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.001+0000 7f5019bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50140b0e20 0x7f50140b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.001+0000 7f5019bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50140b8e30 con 0x7f50140b8a60 2023-07-18T03:39:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.001+0000 7f5019bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50140d14d0 con 0x7f50140d0b10 2023-07-18T03:39:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.001+0000 7f5019bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50140afaa0 con 0x7f50140b0e20 2023-07-18T03:39:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f50137fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f50140b8a60 0x7f50140b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55460/0 (socket says 172.21.15.49:55460) 2023-07-18T03:39:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f50137fe700 1 -- 172.21.15.49:0/419405186 learned_addr learned my addr 172.21.15.49:0/419405186 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f5012ffd700 1 --2- 172.21.15.49:0/419405186 >> v2:172.21.15.107:3300/0 conn(0x7f50140d0b10 0x7f50140d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f5013fff700 1 --2- 172.21.15.49:0/419405186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50140b0e20 0x7f50140b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57687244 0 0) 0x7f50140b8e30 con 0x7f50140b8a60 2023-07-18T03:39:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ffc0035b0 con 0x7f50140b8a60 2023-07-18T03:39:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1392439577 0 0) 0x7f4ffc0035b0 con 0x7f50140b8a60 2023-07-18T03:39:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50140b0e20 msgr2=0x7f50140b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f50127fc700 1 --2- 172.21.15.49:0/419405186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50140b0e20 0x7f50140b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 >> v2:172.21.15.107:3300/0 conn(0x7f50140d0b10 msgr2=0x7f50140d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f50127fc700 1 --2- 172.21.15.49:0/419405186 >> v2:172.21.15.107:3300/0 conn(0x7f50140d0b10 0x7f50140d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50140afd80 con 0x7f50140b8a60 2023-07-18T03:39:21.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.002+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f5004001670 con 0x7f50140b8a60 2023-07-18T03:39:21.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 >> v1:172.21.15.49:6789/0 conn(0x7f50140b8a60 legacy=0x7f50140b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50127fc700 1 --2- 172.21.15.49:0/419405186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc003a00 0x7f4ffc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50127fc700 1 --2- 172.21.15.49:0/419405186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc004430 0x7f4ffc0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50127fc700 1 --2- 172.21.15.49:0/419405186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50140b0e20 0x7f4ffc004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50140d14d0 con 0x7f4ffc003a00 2023-07-18T03:39:21.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50140afaa0 con 0x7f4ffc004430 2023-07-18T03:39:21.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc00ba10 con 0x7f50140b0e20 2023-07-18T03:39:21.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50137fe700 1 --2- 172.21.15.49:0/419405186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc003a00 0x7f4ffc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f5012ffd700 1 --2- 172.21.15.49:0/419405186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc004430 0x7f4ffc0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f5013fff700 1 --2- 172.21.15.49:0/419405186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50140b0e20 0x7f4ffc004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50137fe700 1 -- 172.21.15.49:0/419405186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50140b0e20 msgr2=0x7f4ffc004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50137fe700 1 --2- 172.21.15.49:0/419405186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50140b0e20 0x7f4ffc004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50137fe700 1 -- 172.21.15.49:0/419405186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc004430 msgr2=0x7f4ffc0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50137fe700 1 --2- 172.21.15.49:0/419405186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc004430 0x7f4ffc0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.003+0000 7f50137fe700 1 -- 172.21.15.49:0/419405186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc0035b0 con 0x7f4ffc003a00 2023-07-18T03:39:21.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.004+0000 7f50137fe700 1 --2- 172.21.15.49:0/419405186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc003a00 0x7f4ffc003e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f500400c320 tx=0x7f4ffc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670351d6f9d85f0f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.004+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500400ce60 con 0x7f4ffc003a00 2023-07-18T03:39:21.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.004+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5004005960 con 0x7f4ffc003a00 2023-07-18T03:39:21.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.004+0000 7f50127fc700 1 -- 172.21.15.49:0/419405186 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004005cf0 con 0x7f4ffc003a00 2023-07-18T03:39:21.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.004+0000 7f5019bff700 1 -- 172.21.15.49:0/419405186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc003a00 msgr2=0x7f4ffc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.004+0000 7f5019bff700 1 --2- 172.21.15.49:0/419405186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc003a00 0x7f4ffc003e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f500400c320 tx=0x7f4ffc005880 comp rx=0 tx=0).stop 2023-07-18T03:39:21.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.004+0000 7f5013fff700 1 -- 172.21.15.49:0/419405186 reap_dead start 2023-07-18T03:39:21.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.004+0000 7f5019bff700 1 -- 172.21.15.49:0/419405186 shutdown_connections 2023-07-18T03:39:21.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.004+0000 7f5019bff700 1 -- 172.21.15.49:0/419405186 >> 172.21.15.49:0/419405186 conn(0x7f501409c1b0 msgr2=0x7f50140d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:21.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.005+0000 7f5019bff700 1 -- 172.21.15.49:0/419405186 shutdown_connections 2023-07-18T03:39:21.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.005+0000 7f5019bff700 1 -- 172.21.15.49:0/419405186 wait complete. 2023-07-18T03:39:21.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.007+0000 7f5019bff700 1 Processor -- start 2023-07-18T03:39:21.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.007+0000 7f5019bff700 1 -- start start 2023-07-18T03:39:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.008+0000 7f5019bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc003a00 0x7f50140cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.008+0000 7f5019bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f501415f720 0x7f50141f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5019bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50141f5a70 0x7f501415d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5019bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50140ce6d0 con 0x7f501415f720 2023-07-18T03:39:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5019bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50141f5ef0 con 0x7f4ffc003a00 2023-07-18T03:39:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5019bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50140cdb50 con 0x7f50141f5a70 2023-07-18T03:39:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5012ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f501415f720 0x7f50141f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5012ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f501415f720 0x7f50141f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37784/0 (socket says 172.21.15.49:37784) 2023-07-18T03:39:21.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5012ffd700 1 -- 172.21.15.49:0/352543604 learned_addr learned my addr 172.21.15.49:0/352543604 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:21.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f50137fe700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc003a00 0x7f50140cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5013fff700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50141f5a70 0x7f501415d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5012ffd700 1 -- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50141f5a70 msgr2=0x7f501415d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5012ffd700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50141f5a70 0x7f501415d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5012ffd700 1 -- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc003a00 msgr2=0x7f50140cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5012ffd700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc003a00 0x7f50140cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5012ffd700 1 -- 172.21.15.49:0/352543604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50140cde30 con 0x7f501415f720 2023-07-18T03:39:21.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.009+0000 7f5013fff700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50141f5a70 0x7f501415d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:21.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.010+0000 7f5012ffd700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f501415f720 0x7f50141f5530 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5008011aa0 tx=0x7f5008011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.010+0000 7f4ffbfff700 1 -- 172.21.15.49:0/352543604 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500800f8e0 con 0x7f501415f720 2023-07-18T03:39:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.010+0000 7f4ffbfff700 1 -- 172.21.15.49:0/352543604 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500800fa60 con 0x7f501415f720 2023-07-18T03:39:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.010+0000 7f4ffbfff700 1 -- 172.21.15.49:0/352543604 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500800fe10 con 0x7f501415f720 2023-07-18T03:39:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.010+0000 7f5019bff700 1 -- 172.21.15.49:0/352543604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50140ce110 con 0x7f501415f720 2023-07-18T03:39:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.010+0000 7f5019bff700 1 -- 172.21.15.49:0/352543604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501415fb70 con 0x7f501415f720 2023-07-18T03:39:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.011+0000 7f5019bff700 1 -- 172.21.15.49:0/352543604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5014074350 con 0x7f501415f720 2023-07-18T03:39:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.012+0000 7f4ffbfff700 1 -- 172.21.15.49:0/352543604 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5008014440 con 0x7f501415f720 2023-07-18T03:39:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.013+0000 7f4ffbfff700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f500006ad10 0x7f500006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.014+0000 7f4ffbfff700 1 -- 172.21.15.49:0/352543604 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5008089a10 con 0x7f501415f720 2023-07-18T03:39:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.014+0000 7f50137fe700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f500006ad10 0x7f500006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.015+0000 7f50137fe700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f500006ad10 0x7f500006d1c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5004000c00 tx=0x7f500400a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.018+0000 7f4ffbfff700 1 -- 172.21.15.49:0/352543604 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5008054b80 con 0x7f501415f720 2023-07-18T03:39:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.223+0000 7f5019bff700 1 -- 172.21.15.49:0/352543604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f501415fdf0 con 0x7f501415f720 2023-07-18T03:39:21.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.224+0000 7f4ffbfff700 1 -- 172.21.15.49:0/352543604 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50080220a0 con 0x7f501415f720 2023-07-18T03:39:21.218 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:39:21.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.226+0000 7f5019bff700 1 -- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f500006ad10 msgr2=0x7f500006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.226+0000 7f5019bff700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f500006ad10 0x7f500006d1c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5004000c00 tx=0x7f500400a000 comp rx=0 tx=0).stop 2023-07-18T03:39:21.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.226+0000 7f5019bff700 1 -- 172.21.15.49:0/352543604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f501415f720 msgr2=0x7f50141f5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.226+0000 7f5019bff700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f501415f720 0x7f50141f5530 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5008011aa0 tx=0x7f5008011db0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.226+0000 7f5019bff700 1 -- 172.21.15.49:0/352543604 shutdown_connections 2023-07-18T03:39:21.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.226+0000 7f5019bff700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50141f5a70 0x7f501415d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.226+0000 7f5019bff700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc003a00 0x7f50140cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.226+0000 7f5019bff700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f500006ad10 0x7f500006d1c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.226+0000 7f5019bff700 1 --2- 172.21.15.49:0/352543604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f501415f720 0x7f50141f5530 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.226+0000 7f5019bff700 1 -- 172.21.15.49:0/352543604 >> 172.21.15.49:0/352543604 conn(0x7f501409c1b0 msgr2=0x7f50140d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:21.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.226+0000 7f5019bff700 1 -- 172.21.15.49:0/352543604 shutdown_connections 2023-07-18T03:39:21.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.226+0000 7f5019bff700 1 -- 172.21.15.49:0/352543604 wait complete. 2023-07-18T03:39:21.239 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:39:21.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.451+0000 7ffaa19ff700 1 Processor -- start 2023-07-18T03:39:21.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.452+0000 7ffaa19ff700 1 -- start start 2023-07-18T03:39:21.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.453+0000 7ffaa19ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ffa9c0b4600 0x7ffa9c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.453+0000 7ffaa19ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9c0cfe40 0x7ffa9c0d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.453+0000 7ffaa19ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c15cd60 con 0x7ffa9c0cd540 2023-07-18T03:39:21.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.453+0000 7ffaa19ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9c0d1550 con 0x7ffa9c0b4600 2023-07-18T03:39:21.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.453+0000 7ffaa19ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c0d2d90 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.453+0000 7ffa9affd700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ffa9c0b4600 0x7ffa9c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.453+0000 7ffa9b7fe700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7ffa9c0cd540 0x7ffa9c0cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:38826/0 (socket says 172.21.15.49:38826) 2023-07-18T03:39:21.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.453+0000 7ffa9b7fe700 1 -- 172.21.15.49:0/1614586052 learned_addr learned my addr 172.21.15.49:0/1614586052 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:21.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.453+0000 7ffa9a7fc700 1 --2- 172.21.15.49:0/1614586052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9c0cfe40 0x7ffa9c0d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.454+0000 7ffa9affd700 1 -- 172.21.15.49:0/1614586052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9c0cfe40 msgr2=0x7ffa9c0d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.454+0000 7ffa9affd700 1 --2- 172.21.15.49:0/1614586052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9c0cfe40 0x7ffa9c0d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.454+0000 7ffa9affd700 1 -- 172.21.15.49:0/1614586052 >> v1:172.21.15.107:6789/0 conn(0x7ffa9c0cd540 legacy=0x7ffa9c0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.454+0000 7ffa9affd700 1 -- 172.21.15.49:0/1614586052 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c0cf340 con 0x7ffa9c0b4600 2023-07-18T03:39:21.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.454+0000 7ffa9affd700 1 --2- 172.21.15.49:0/1614586052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c0b4600 0x7ffa9c0cf900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009870 tx=0x7ffa8c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba9cf6e243b81195 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.454+0000 7ffa99ffb700 1 -- 172.21.15.49:0/1614586052 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c011830 con 0x7ffa9c0b4600 2023-07-18T03:39:21.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.454+0000 7ffa99ffb700 1 -- 172.21.15.49:0/1614586052 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c011990 con 0x7ffa9c0b4600 2023-07-18T03:39:21.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.454+0000 7ffa99ffb700 1 -- 172.21.15.49:0/1614586052 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c011d20 con 0x7ffa9c0b4600 2023-07-18T03:39:21.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.455+0000 7ffaa19ff700 1 -- 172.21.15.49:0/1614586052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c0b4600 msgr2=0x7ffa9c0cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.455+0000 7ffaa19ff700 1 --2- 172.21.15.49:0/1614586052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c0b4600 0x7ffa9c0cf900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009870 tx=0x7ffa8c009b80 comp rx=0 tx=0).stop 2023-07-18T03:39:21.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.455+0000 7ffaa19ff700 1 -- 172.21.15.49:0/1614586052 shutdown_connections 2023-07-18T03:39:21.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.455+0000 7ffaa19ff700 1 --2- 172.21.15.49:0/1614586052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c0b4600 0x7ffa9c0cf900 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.455+0000 7ffaa19ff700 1 --2- 172.21.15.49:0/1614586052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9c0cfe40 0x7ffa9c0d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.455+0000 7ffaa19ff700 1 -- 172.21.15.49:0/1614586052 >> 172.21.15.49:0/1614586052 conn(0x7ffa9c09c200 msgr2=0x7ffa9c0d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:21.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.455+0000 7ffaa19ff700 1 -- 172.21.15.49:0/1614586052 shutdown_connections 2023-07-18T03:39:21.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.455+0000 7ffaa19ff700 1 -- 172.21.15.49:0/1614586052 wait complete. 2023-07-18T03:39:21.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.457+0000 7ffaa19ff700 1 Processor -- start 2023-07-18T03:39:21.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.457+0000 7ffaa19ff700 1 -- start start 2023-07-18T03:39:21.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.458+0000 7ffaa19ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9c0cd540 0x7ffa9c15ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.458+0000 7ffaa19ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c0cfe40 0x7ffa9c158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.458+0000 7ffaa19ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c158c50 0x7ffa9c156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.458+0000 7ffaa19ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c0c7b10 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.458+0000 7ffaa19ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c159110 con 0x7ffa9c158c50 2023-07-18T03:39:21.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.458+0000 7ffaa19ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c1593f0 con 0x7ffa9c0cd540 2023-07-18T03:39:21.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.458+0000 7ffa9a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c0cfe40 0x7ffa9c158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.458+0000 7ffa9a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c0cfe40 0x7ffa9c158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37792/0 (socket says 172.21.15.49:37792) 2023-07-18T03:39:21.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.458+0000 7ffa9a7fc700 1 -- 172.21.15.49:0/499545016 learned_addr learned my addr 172.21.15.49:0/499545016 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:21.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.458+0000 7ffa9affd700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9c0cd540 0x7ffa9c15ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.458+0000 7ffa9b7fe700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c158c50 0x7ffa9c156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.459+0000 7ffa9a7fc700 1 -- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9c0cd540 msgr2=0x7ffa9c15ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.459+0000 7ffa9a7fc700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9c0cd540 0x7ffa9c15ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.459+0000 7ffa9a7fc700 1 -- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c158c50 msgr2=0x7ffa9c156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.459+0000 7ffa9a7fc700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c158c50 0x7ffa9c156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.459+0000 7ffa9a7fc700 1 -- 172.21.15.49:0/499545016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c0094a0 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.459+0000 7ffa9a7fc700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c0cfe40 0x7ffa9c158710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000e9b0 tx=0x7ffa9000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.459+0000 7ffa83fff700 1 -- 172.21.15.49:0/499545016 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90009710 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.459+0000 7ffa83fff700 1 -- 172.21.15.49:0/499545016 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9000ee50 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.459+0000 7ffaa19ff700 1 -- 172.21.15.49:0/499545016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9c0c74c0 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.459+0000 7ffa83fff700 1 -- 172.21.15.49:0/499545016 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000ca50 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.460+0000 7ffaa19ff700 1 -- 172.21.15.49:0/499545016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c15f250 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.461+0000 7ffaa19ff700 1 -- 172.21.15.49:0/499545016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9c0c7c80 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.462+0000 7ffa83fff700 1 -- 172.21.15.49:0/499545016 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffa9000cbb0 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.462+0000 7ffa83fff700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa8806ace0 0x7ffa8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.463+0000 7ffa83fff700 1 -- 172.21.15.49:0/499545016 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffa90089450 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.463+0000 7ffa9affd700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa8806ace0 0x7ffa8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.464+0000 7ffa9affd700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa8806ace0 0x7ffa8806d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0090b0 tx=0x7ffa8c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.467+0000 7ffa83fff700 1 -- 172.21.15.49:0/499545016 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffa90057f90 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.672+0000 7ffaa19ff700 1 -- 172.21.15.49:0/499545016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa9c074350 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.673+0000 7ffa83fff700 1 -- 172.21.15.49:0/499545016 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa9000ce60 con 0x7ffa9c0cfe40 2023-07-18T03:39:21.668 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:39:21.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.675+0000 7ffaa19ff700 1 -- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa8806ace0 msgr2=0x7ffa8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.675+0000 7ffaa19ff700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa8806ace0 0x7ffa8806d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0090b0 tx=0x7ffa8c01c040 comp rx=0 tx=0).stop 2023-07-18T03:39:21.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.675+0000 7ffaa19ff700 1 -- 172.21.15.49:0/499545016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c0cfe40 msgr2=0x7ffa9c158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.675+0000 7ffaa19ff700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c0cfe40 0x7ffa9c158710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000e9b0 tx=0x7ffa9000ecc0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.675+0000 7ffaa19ff700 1 -- 172.21.15.49:0/499545016 shutdown_connections 2023-07-18T03:39:21.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.675+0000 7ffaa19ff700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa9c158c50 0x7ffa9c156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.675+0000 7ffaa19ff700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa9c0cd540 0x7ffa9c15ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.675+0000 7ffaa19ff700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ffa8806ace0 0x7ffa8806d190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.675+0000 7ffaa19ff700 1 --2- 172.21.15.49:0/499545016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa9c0cfe40 0x7ffa9c158710 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.675+0000 7ffaa19ff700 1 -- 172.21.15.49:0/499545016 >> 172.21.15.49:0/499545016 conn(0x7ffa9c09c200 msgr2=0x7ffa9c0b5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:21.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.675+0000 7ffaa19ff700 1 -- 172.21.15.49:0/499545016 shutdown_connections 2023-07-18T03:39:21.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.675+0000 7ffaa19ff700 1 -- 172.21.15.49:0/499545016 wait complete. 2023-07-18T03:39:21.688 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:39:21.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.902+0000 7fd72ddff700 1 Processor -- start 2023-07-18T03:39:21.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.902+0000 7fd72ddff700 1 -- start start 2023-07-18T03:39:21.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.903+0000 7fd72ddff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd7280b8a60 0x7fd7280b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.903+0000 7fd72ddff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7280b8e30 con 0x7fd7280d1220 2023-07-18T03:39:21.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.903+0000 7fd72ddff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7280afaa0 con 0x7fd7280b4530 2023-07-18T03:39:21.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.903+0000 7fd72ddff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7280afd80 con 0x7fd7280b8a60 2023-07-18T03:39:21.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.904+0000 7fd727fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd7280d1220 0x7fd7280b0e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55466/0 (socket says 172.21.15.49:55466) 2023-07-18T03:39:21.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.904+0000 7fd727fff700 1 -- 172.21.15.49:0/1834765154 learned_addr learned my addr 172.21.15.49:0/1834765154 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:21.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.904+0000 7fd7277fe700 1 --2- 172.21.15.49:0/1834765154 >> v2:172.21.15.107:3300/0 conn(0x7fd7280b8a60 0x7fd7280b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.904+0000 7fd7267fc700 1 -- 172.21.15.49:0/1834765154 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862204218 0 0) 0x7fd7280b8e30 con 0x7fd7280d1220 2023-07-18T03:39:21.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.904+0000 7fd7267fc700 1 -- 172.21.15.49:0/1834765154 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7100035b0 con 0x7fd7280d1220 2023-07-18T03:39:21.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.904+0000 7fd7267fc700 1 -- 172.21.15.49:0/1834765154 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853563445 0 0) 0x7fd7280afaa0 con 0x7fd7280b4530 2023-07-18T03:39:21.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.904+0000 7fd7267fc700 1 -- 172.21.15.49:0/1834765154 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7280b8e30 con 0x7fd7280b4530 2023-07-18T03:39:21.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.904+0000 7fd7277fe700 1 -- 172.21.15.49:0/1834765154 >> v1:172.21.15.107:6789/0 conn(0x7fd7280b4530 legacy=0x7fd7280d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.904+0000 7fd7277fe700 1 -- 172.21.15.49:0/1834765154 >> v1:172.21.15.49:6789/0 conn(0x7fd7280d1220 legacy=0x7fd7280b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.904+0000 7fd7277fe700 1 -- 172.21.15.49:0/1834765154 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7280b0060 con 0x7fd7280b8a60 2023-07-18T03:39:21.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.905+0000 7fd7277fe700 1 --2- 172.21.15.49:0/1834765154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7280b8a60 0x7fd7280b3ff0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd718009840 tx=0x7fd718009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6041452d3945f2b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.905+0000 7fd7267fc700 1 -- 172.21.15.49:0/1834765154 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718011830 con 0x7fd7280b8a60 2023-07-18T03:39:21.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.905+0000 7fd7267fc700 1 -- 172.21.15.49:0/1834765154 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd718011990 con 0x7fd7280b8a60 2023-07-18T03:39:21.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.905+0000 7fd7267fc700 1 -- 172.21.15.49:0/1834765154 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718011d00 con 0x7fd7280b8a60 2023-07-18T03:39:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.906+0000 7fd72ddff700 1 -- 172.21.15.49:0/1834765154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7280b8a60 msgr2=0x7fd7280b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.906+0000 7fd72ddff700 1 --2- 172.21.15.49:0/1834765154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7280b8a60 0x7fd7280b3ff0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd718009840 tx=0x7fd718009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.906+0000 7fd72ddff700 1 -- 172.21.15.49:0/1834765154 shutdown_connections 2023-07-18T03:39:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.906+0000 7fd72ddff700 1 --2- 172.21.15.49:0/1834765154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7280b8a60 0x7fd7280b3ff0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.906+0000 7fd72ddff700 1 -- 172.21.15.49:0/1834765154 >> 172.21.15.49:0/1834765154 conn(0x7fd72809c1b0 msgr2=0x7fd7280d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.906+0000 7fd72ddff700 1 -- 172.21.15.49:0/1834765154 shutdown_connections 2023-07-18T03:39:21.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.906+0000 7fd72ddff700 1 -- 172.21.15.49:0/1834765154 wait complete. 2023-07-18T03:39:21.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.907+0000 7fd72ddff700 1 Processor -- start 2023-07-18T03:39:21.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.908+0000 7fd72ddff700 1 -- start start 2023-07-18T03:39:21.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd72ddff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7280b4530 0x7fd728150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd72ddff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7280b8a60 0x7fd72814f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd72ddff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7280d1220 0x7fd7281f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd72ddff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280cfcd0 con 0x7fd7280d1220 2023-07-18T03:39:21.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd72ddff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72814feb0 con 0x7fd7280b4530 2023-07-18T03:39:21.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd72ddff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280cf050 con 0x7fd7280b8a60 2023-07-18T03:39:21.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd727fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7280d1220 0x7fd7281f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd7277fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7280b4530 0x7fd728150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd727fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7280d1220 0x7fd7281f4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37796/0 (socket says 172.21.15.49:37796) 2023-07-18T03:39:21.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd727fff700 1 -- 172.21.15.49:0/109849167 learned_addr learned my addr 172.21.15.49:0/109849167 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:21.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd726ffd700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7280b8a60 0x7fd72814f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd727fff700 1 -- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7280b8a60 msgr2=0x7fd72814f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd727fff700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7280b8a60 0x7fd72814f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd727fff700 1 -- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7280b4530 msgr2=0x7fd728150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd727fff700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7280b4530 0x7fd728150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.909+0000 7fd727fff700 1 -- 172.21.15.49:0/109849167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7180094a0 con 0x7fd7280d1220 2023-07-18T03:39:21.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.910+0000 7fd727fff700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7280d1220 0x7fd7281f4d60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd72000ca70 tx=0x7fd72000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.910+0000 7fd70ffff700 1 -- 172.21.15.49:0/109849167 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72000a9e0 con 0x7fd7280d1220 2023-07-18T03:39:21.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.910+0000 7fd70ffff700 1 -- 172.21.15.49:0/109849167 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72000ab40 con 0x7fd7280d1220 2023-07-18T03:39:21.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.910+0000 7fd70ffff700 1 -- 172.21.15.49:0/109849167 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720012400 con 0x7fd7280d1220 2023-07-18T03:39:21.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.910+0000 7fd72ddff700 1 -- 172.21.15.49:0/109849167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7280cf330 con 0x7fd7280d1220 2023-07-18T03:39:21.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.910+0000 7fd72ddff700 1 -- 172.21.15.49:0/109849167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7280cf8f0 con 0x7fd7280d1220 2023-07-18T03:39:21.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.911+0000 7fd72ddff700 1 -- 172.21.15.49:0/109849167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd728074350 con 0x7fd7280d1220 2023-07-18T03:39:21.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.912+0000 7fd70ffff700 1 -- 172.21.15.49:0/109849167 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd72001b3b0 con 0x7fd7280d1220 2023-07-18T03:39:21.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.913+0000 7fd70ffff700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd71406ace0 0x7fd71406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.913+0000 7fd70ffff700 1 -- 172.21.15.49:0/109849167 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd720088e70 con 0x7fd7280d1220 2023-07-18T03:39:21.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.913+0000 7fd7277fe700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd71406ace0 0x7fd71406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.914+0000 7fd7277fe700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd71406ace0 0x7fd71406d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd718009b90 tx=0x7fd71801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:21.918+0000 7fd70ffff700 1 -- 172.21.15.49:0/109849167 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd720054060 con 0x7fd7280d1220 2023-07-18T03:39:22.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.125+0000 7fd72ddff700 1 -- 172.21.15.49:0/109849167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7280d23e0 con 0x7fd7280d1220 2023-07-18T03:39:22.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.126+0000 7fd70ffff700 1 -- 172.21.15.49:0/109849167 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd720017020 con 0x7fd7280d1220 2023-07-18T03:39:22.121 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:39:22.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.128+0000 7fd72ddff700 1 -- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd71406ace0 msgr2=0x7fd71406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.128+0000 7fd72ddff700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd71406ace0 0x7fd71406d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd718009b90 tx=0x7fd71801c040 comp rx=0 tx=0).stop 2023-07-18T03:39:22.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.128+0000 7fd72ddff700 1 -- 172.21.15.49:0/109849167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7280d1220 msgr2=0x7fd7281f4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.128+0000 7fd72ddff700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7280d1220 0x7fd7281f4d60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd72000ca70 tx=0x7fd72000cd80 comp rx=0 tx=0).stop 2023-07-18T03:39:22.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.128+0000 7fd72ddff700 1 -- 172.21.15.49:0/109849167 shutdown_connections 2023-07-18T03:39:22.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.128+0000 7fd72ddff700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd71406ace0 0x7fd71406d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.128+0000 7fd72ddff700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7280d1220 0x7fd7281f4d60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.128+0000 7fd72ddff700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7280b4530 0x7fd728150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.128+0000 7fd72ddff700 1 --2- 172.21.15.49:0/109849167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7280b8a60 0x7fd72814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.128+0000 7fd72ddff700 1 -- 172.21.15.49:0/109849167 >> 172.21.15.49:0/109849167 conn(0x7fd72809c1b0 msgr2=0x7fd72815c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:22.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.128+0000 7fd72ddff700 1 -- 172.21.15.49:0/109849167 shutdown_connections 2023-07-18T03:39:22.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.129+0000 7fd72ddff700 1 -- 172.21.15.49:0/109849167 wait complete. 2023-07-18T03:39:22.141 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-18T03:39:22.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.352+0000 7f7a2f7ff700 1 Processor -- start 2023-07-18T03:39:22.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.352+0000 7f7a2f7ff700 1 -- start start 2023-07-18T03:39:22.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.353+0000 7f7a2f7ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7a280b4860 0x7f7a280b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.353+0000 7f7a2f7ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a280d02a0 con 0x7f7a280b8660 2023-07-18T03:39:22.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.353+0000 7f7a2f7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a280cfce0 con 0x7f7a280d12c0 2023-07-18T03:39:22.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.353+0000 7f7a2f7ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a280b5240 con 0x7f7a280b4860 2023-07-18T03:39:22.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.354+0000 7f7a2cd9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7a280b4860 0x7f7a280b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.354+0000 7f7a2d59b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7a280b8660 0x7f7a280cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55472/0 (socket says 172.21.15.49:55472) 2023-07-18T03:39:22.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.354+0000 7f7a2d59b700 1 -- 172.21.15.49:0/926560894 learned_addr learned my addr 172.21.15.49:0/926560894 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:22.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.354+0000 7f7a1ffff700 1 -- 172.21.15.49:0/926560894 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778108873 0 0) 0x7f7a280d02a0 con 0x7f7a280b8660 2023-07-18T03:39:22.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.354+0000 7f7a1ffff700 1 -- 172.21.15.49:0/926560894 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a140035b0 con 0x7f7a280b8660 2023-07-18T03:39:22.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.354+0000 7f7a2cd9a700 1 -- 172.21.15.49:0/926560894 >> v1:172.21.15.107:6789/0 conn(0x7f7a280d12c0 legacy=0x7f7a280d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.354+0000 7f7a2cd9a700 1 -- 172.21.15.49:0/926560894 >> v1:172.21.15.49:6789/0 conn(0x7f7a280b8660 legacy=0x7f7a280cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.354+0000 7f7a2cd9a700 1 -- 172.21.15.49:0/926560894 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a280cffc0 con 0x7f7a280b4860 2023-07-18T03:39:22.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.355+0000 7f7a2cd9a700 1 --2- 172.21.15.49:0/926560894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a280b4860 0x7f7a280b4d00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009840 tx=0x7f7a18009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b7d17755552ba6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:22.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.355+0000 7f7a1ffff700 1 -- 172.21.15.49:0/926560894 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18011830 con 0x7f7a280b4860 2023-07-18T03:39:22.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.355+0000 7f7a1ffff700 1 -- 172.21.15.49:0/926560894 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a18011990 con 0x7f7a280b4860 2023-07-18T03:39:22.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.355+0000 7f7a1ffff700 1 -- 172.21.15.49:0/926560894 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18011d20 con 0x7f7a280b4860 2023-07-18T03:39:22.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.355+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/926560894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a280b4860 msgr2=0x7f7a280b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.355+0000 7f7a2f7ff700 1 --2- 172.21.15.49:0/926560894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a280b4860 0x7f7a280b4d00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009840 tx=0x7f7a18009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:22.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.355+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/926560894 shutdown_connections 2023-07-18T03:39:22.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.355+0000 7f7a2f7ff700 1 --2- 172.21.15.49:0/926560894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a280b4860 0x7f7a280b4d00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.355+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/926560894 >> 172.21.15.49:0/926560894 conn(0x7f7a28066860 msgr2=0x7f7a2806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:22.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.356+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/926560894 shutdown_connections 2023-07-18T03:39:22.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.356+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/926560894 wait complete. 2023-07-18T03:39:22.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.357+0000 7f7a2f7ff700 1 Processor -- start 2023-07-18T03:39:22.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.357+0000 7f7a2f7ff700 1 -- start start 2023-07-18T03:39:22.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.358+0000 7f7a2f7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a280b4860 0x7f7a281f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.358+0000 7f7a2f7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a280b8660 0x7f7a28157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.358+0000 7f7a2f7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a280d12c0 0x7f7a2815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.358+0000 7f7a2f7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a28159490 con 0x7f7a280b8660 2023-07-18T03:39:22.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.358+0000 7f7a2f7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a28158880 con 0x7f7a280d12c0 2023-07-18T03:39:22.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.358+0000 7f7a2f7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a28158b60 con 0x7f7a280b4860 2023-07-18T03:39:22.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.359+0000 7f7a2cd9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a280b8660 0x7f7a28157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.359+0000 7f7a2cd9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a280b8660 0x7f7a28157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37810/0 (socket says 172.21.15.49:37810) 2023-07-18T03:39:22.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.359+0000 7f7a2cd9a700 1 -- 172.21.15.49:0/93931219 learned_addr learned my addr 172.21.15.49:0/93931219 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:22.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.359+0000 7f7a2dd9c700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a280d12c0 0x7f7a2815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.359+0000 7f7a2d59b700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a280b4860 0x7f7a281f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.359+0000 7f7a2cd9a700 1 -- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a280b4860 msgr2=0x7f7a281f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.359+0000 7f7a2cd9a700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a280b4860 0x7f7a281f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.359+0000 7f7a2cd9a700 1 -- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a280d12c0 msgr2=0x7f7a2815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.359+0000 7f7a2cd9a700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a280d12c0 0x7f7a2815a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.359+0000 7f7a2cd9a700 1 -- 172.21.15.49:0/93931219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a180094a0 con 0x7f7a280b8660 2023-07-18T03:39:22.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.359+0000 7f7a2cd9a700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a280b8660 0x7f7a28157ea0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009fd0 tx=0x7f7a18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:22.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.360+0000 7f7a1dffb700 1 -- 172.21.15.49:0/93931219 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a180038a0 con 0x7f7a280b8660 2023-07-18T03:39:22.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.360+0000 7f7a1dffb700 1 -- 172.21.15.49:0/93931219 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1801f070 con 0x7f7a280b8660 2023-07-18T03:39:22.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.360+0000 7f7a1dffb700 1 -- 172.21.15.49:0/93931219 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18003c40 con 0x7f7a280b8660 2023-07-18T03:39:22.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.360+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/93931219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a28158e40 con 0x7f7a280b8660 2023-07-18T03:39:22.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.360+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/93931219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a28159920 con 0x7f7a280b8660 2023-07-18T03:39:22.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.361+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/93931219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a28074350 con 0x7f7a280b8660 2023-07-18T03:39:22.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.362+0000 7f7a1dffb700 1 -- 172.21.15.49:0/93931219 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7a18026480 con 0x7f7a280b8660 2023-07-18T03:39:22.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.363+0000 7f7a1dffb700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7a0c06ace0 0x7f7a0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.363+0000 7f7a1dffb700 1 -- 172.21.15.49:0/93931219 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7a18021070 con 0x7f7a280b8660 2023-07-18T03:39:22.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.363+0000 7f7a2d59b700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7a0c06ace0 0x7f7a0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.364+0000 7f7a2d59b700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7a0c06ace0 0x7f7a0c06d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a240011e0 tx=0x7f7a24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:22.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.367+0000 7f7a1dffb700 1 -- 172.21.15.49:0/93931219 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7a18055f00 con 0x7f7a280b8660 2023-07-18T03:39:22.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.595+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/93931219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7a280cdb10 con 0x7f7a280b8660 2023-07-18T03:39:22.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.596+0000 7f7a1dffb700 1 -- 172.21.15.49:0/93931219 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7a18015080 con 0x7f7a280b8660 2023-07-18T03:39:22.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.598+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7a0c06ace0 msgr2=0x7f7a0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.598+0000 7f7a2f7ff700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7a0c06ace0 0x7f7a0c06d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a240011e0 tx=0x7f7a24009040 comp rx=0 tx=0).stop 2023-07-18T03:39:22.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.598+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/93931219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a280b8660 msgr2=0x7f7a28157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.598+0000 7f7a2f7ff700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a280b8660 0x7f7a28157ea0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009fd0 tx=0x7f7a18004750 comp rx=0 tx=0).stop 2023-07-18T03:39:22.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.598+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/93931219 shutdown_connections 2023-07-18T03:39:22.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.598+0000 7f7a2f7ff700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a280d12c0 0x7f7a2815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.598+0000 7f7a2f7ff700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a280b4860 0x7f7a281f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.598+0000 7f7a2f7ff700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7a0c06ace0 0x7f7a0c06d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.598+0000 7f7a2f7ff700 1 --2- 172.21.15.49:0/93931219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a280b8660 0x7f7a28157ea0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.598+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/93931219 >> 172.21.15.49:0/93931219 conn(0x7f7a28066860 msgr2=0x7f7a280c8920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:22.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.598+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/93931219 shutdown_connections 2023-07-18T03:39:22.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:22.599+0000 7f7a2f7ff700 1 -- 172.21.15.49:0/93931219 wait complete. 2023-07-18T03:39:22.597 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-07-18T03:39:22.612 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-18T03:39:22.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:22.613 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-07-18T03:39:22.730 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:39:22.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:22.731 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-18T03:39:22.794 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:39:22.794 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:22.844 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:39:22.845 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:22.930 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:39:22.931 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:23.015 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:23.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.227+0000 7f1e76dff700 1 Processor -- start 2023-07-18T03:39:23.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.227+0000 7f1e76dff700 1 -- start start 2023-07-18T03:39:23.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.229+0000 7f1e76dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e701500d0 0x7f1e70070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.229+0000 7f1e76dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1e700cea50 0x7f1e700ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.229+0000 7f1e76dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e700cdf90 con 0x7f1e70070890 2023-07-18T03:39:23.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.229+0000 7f1e76dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7015a2f0 con 0x7f1e700cea50 2023-07-18T03:39:23.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.229+0000 7f1e76dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7015a5d0 con 0x7f1e701500d0 2023-07-18T03:39:23.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.229+0000 7f1e74b9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e701500d0 0x7f1e70070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.229+0000 7f1e74b9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e701500d0 0x7f1e70070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:35206/0 (socket says 172.21.15.49:35206) 2023-07-18T03:39:23.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.230+0000 7f1e74b9b700 1 -- 172.21.15.49:0/3874742935 learned_addr learned my addr 172.21.15.49:0/3874742935 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:23.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.229+0000 7f1e7539c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1e700cea50 0x7f1e700ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.229+0000 7f1e6ffff700 1 --1- 172.21.15.49:0/3874742935 >> v1:172.21.15.107:6789/0 conn(0x7f1e70070890 0x7f1e701597c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:38866/0 (socket says 172.21.15.49:38866) 2023-07-18T03:39:23.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.230+0000 7f1e74b9b700 1 -- 172.21.15.49:0/3874742935 >> v2:172.21.15.107:3300/0 conn(0x7f1e700cea50 msgr2=0x7f1e700ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:23.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.230+0000 7f1e74b9b700 1 --2- 172.21.15.49:0/3874742935 >> v2:172.21.15.107:3300/0 conn(0x7f1e700cea50 0x7f1e700ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.230+0000 7f1e74b9b700 1 -- 172.21.15.49:0/3874742935 >> v1:172.21.15.107:6789/0 conn(0x7f1e70070890 legacy=0x7f1e701597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.230+0000 7f1e74b9b700 1 -- 172.21.15.49:0/3874742935 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700ce250 con 0x7f1e701500d0 2023-07-18T03:39:23.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.230+0000 7f1e74b9b700 1 --2- 172.21.15.49:0/3874742935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e701500d0 0x7f1e70070350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400bb90 tx=0x7f1e6400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a26feb1a39b2258 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.230+0000 7f1e6f7fe700 1 -- 172.21.15.49:0/3874742935 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6400dd30 con 0x7f1e701500d0 2023-07-18T03:39:23.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.231+0000 7f1e6f7fe700 1 -- 172.21.15.49:0/3874742935 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6400de90 con 0x7f1e701500d0 2023-07-18T03:39:23.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.231+0000 7f1e6f7fe700 1 -- 172.21.15.49:0/3874742935 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64010690 con 0x7f1e701500d0 2023-07-18T03:39:23.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.231+0000 7f1e76dff700 1 -- 172.21.15.49:0/3874742935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e701500d0 msgr2=0x7f1e70070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.231+0000 7f1e76dff700 1 --2- 172.21.15.49:0/3874742935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e701500d0 0x7f1e70070350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400bb90 tx=0x7f1e6400bea0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.231+0000 7f1e76dff700 1 -- 172.21.15.49:0/3874742935 shutdown_connections 2023-07-18T03:39:23.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.231+0000 7f1e76dff700 1 --2- 172.21.15.49:0/3874742935 >> v2:172.21.15.107:3300/0 conn(0x7f1e700cea50 0x7f1e700ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.231+0000 7f1e76dff700 1 --2- 172.21.15.49:0/3874742935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e701500d0 0x7f1e70070350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.231+0000 7f1e76dff700 1 -- 172.21.15.49:0/3874742935 >> 172.21.15.49:0/3874742935 conn(0x7f1e70069f30 msgr2=0x7f1e7006a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.231+0000 7f1e76dff700 1 -- 172.21.15.49:0/3874742935 shutdown_connections 2023-07-18T03:39:23.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.231+0000 7f1e76dff700 1 -- 172.21.15.49:0/3874742935 wait complete. 2023-07-18T03:39:23.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.233+0000 7f1e76dff700 1 Processor -- start 2023-07-18T03:39:23.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.233+0000 7f1e76dff700 1 -- start start 2023-07-18T03:39:23.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.234+0000 7f1e76dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e70070890 0x7f1e70148430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.234+0000 7f1e76dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e700cea50 0x7f1e70153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.234+0000 7f1e76dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e701500d0 0x7f1e70137d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.234+0000 7f1e76dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e701508b0 con 0x7f1e70070890 2023-07-18T03:39:23.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.234+0000 7f1e76dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e701f2490 con 0x7f1e700cea50 2023-07-18T03:39:23.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.234+0000 7f1e76dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70138260 con 0x7f1e701500d0 2023-07-18T03:39:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.235+0000 7f1e74b9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e70070890 0x7f1e70148430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.235+0000 7f1e74b9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e70070890 0x7f1e70148430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37826/0 (socket says 172.21.15.49:37826) 2023-07-18T03:39:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.235+0000 7f1e74b9b700 1 -- 172.21.15.49:0/2818342604 learned_addr learned my addr 172.21.15.49:0/2818342604 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.235+0000 7f1e7539c700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e701500d0 0x7f1e70137d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.235+0000 7f1e6ffff700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e700cea50 0x7f1e70153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.235+0000 7f1e74b9b700 1 -- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e701500d0 msgr2=0x7f1e70137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.235+0000 7f1e74b9b700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e701500d0 0x7f1e70137d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.235+0000 7f1e74b9b700 1 -- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e700cea50 msgr2=0x7f1e70153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.235+0000 7f1e74b9b700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e700cea50 0x7f1e70153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.235+0000 7f1e74b9b700 1 -- 172.21.15.49:0/2818342604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6400b7f0 con 0x7f1e70070890 2023-07-18T03:39:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.235+0000 7f1e74b9b700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e70070890 0x7f1e70148430 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e64000c00 tx=0x7f1e64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.236+0000 7f1e6d7fa700 1 -- 172.21.15.49:0/2818342604 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64003b70 con 0x7f1e70070890 2023-07-18T03:39:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.236+0000 7f1e6d7fa700 1 -- 172.21.15.49:0/2818342604 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e64003cd0 con 0x7f1e70070890 2023-07-18T03:39:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.236+0000 7f1e6d7fa700 1 -- 172.21.15.49:0/2818342604 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e640106b0 con 0x7f1e70070890 2023-07-18T03:39:23.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.236+0000 7f1e76dff700 1 -- 172.21.15.49:0/2818342604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e70148b30 con 0x7f1e70070890 2023-07-18T03:39:23.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.236+0000 7f1e76dff700 1 -- 172.21.15.49:0/2818342604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e701490f0 con 0x7f1e70070890 2023-07-18T03:39:23.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.237+0000 7f1e5affd700 1 -- 172.21.15.49:0/2818342604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e480052f0 con 0x7f1e70070890 2023-07-18T03:39:23.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.238+0000 7f1e6d7fa700 1 -- 172.21.15.49:0/2818342604 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1e64019400 con 0x7f1e70070890 2023-07-18T03:39:23.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.239+0000 7f1e6d7fa700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e5406ace0 0x7f1e5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.239+0000 7f1e6d7fa700 1 -- 172.21.15.49:0/2818342604 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1e6408a6d0 con 0x7f1e70070890 2023-07-18T03:39:23.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.239+0000 7f1e6ffff700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e5406ace0 0x7f1e5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.240+0000 7f1e6ffff700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e5406ace0 0x7f1e5406d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e70152e50 tx=0x7f1e5c007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.243+0000 7f1e6d7fa700 1 -- 172.21.15.49:0/2818342604 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1e640557d0 con 0x7f1e70070890 2023-07-18T03:39:23.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.453+0000 7f1e5affd700 1 -- 172.21.15.49:0/2818342604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e48005870 con 0x7f1e70070890 2023-07-18T03:39:23.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.454+0000 7f1e6d7fa700 1 -- 172.21.15.49:0/2818342604 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f1e64058df0 con 0x7f1e70070890 2023-07-18T03:39:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:23.451 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:22.932147+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.457+0000 7f1e5affd700 1 -- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e5406ace0 msgr2=0x7f1e5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.457+0000 7f1e5affd700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e5406ace0 0x7f1e5406d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e70152e50 tx=0x7f1e5c007ed0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.457+0000 7f1e5affd700 1 -- 172.21.15.49:0/2818342604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e70070890 msgr2=0x7f1e70148430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.457+0000 7f1e5affd700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e70070890 0x7f1e70148430 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e64000c00 tx=0x7f1e64004750 comp rx=0 tx=0).stop 2023-07-18T03:39:23.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.457+0000 7f1e5affd700 1 -- 172.21.15.49:0/2818342604 shutdown_connections 2023-07-18T03:39:23.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.457+0000 7f1e5affd700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e701500d0 0x7f1e70137d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.457+0000 7f1e5affd700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e5406ace0 0x7f1e5406d190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.457+0000 7f1e5affd700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e70070890 0x7f1e70148430 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.457+0000 7f1e5affd700 1 --2- 172.21.15.49:0/2818342604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e700cea50 0x7f1e70153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.457+0000 7f1e5affd700 1 -- 172.21.15.49:0/2818342604 >> 172.21.15.49:0/2818342604 conn(0x7f1e70069f30 msgr2=0x7f1e70158200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.457+0000 7f1e5affd700 1 -- 172.21.15.49:0/2818342604 shutdown_connections 2023-07-18T03:39:23.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.457+0000 7f1e5affd700 1 -- 172.21.15.49:0/2818342604 wait complete. 2023-07-18T03:39:23.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:38:33.773236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:38:35.023692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:39:23.471 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:39:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.686+0000 7f56433ff700 1 Processor -- start 2023-07-18T03:39:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.686+0000 7f56433ff700 1 -- start start 2023-07-18T03:39:23.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.687+0000 7f56433ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f563c06b750 0x7f563c0cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.687+0000 7f56433ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f563c0cb330 0x7f563c0b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.687+0000 7f56433ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c0b0560 con 0x7f563c0d12c0 2023-07-18T03:39:23.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.687+0000 7f56433ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c0d1690 con 0x7f563c06b750 2023-07-18T03:39:23.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.687+0000 7f56433ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0cc760 con 0x7f563c0cb330 2023-07-18T03:39:23.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.688+0000 7f564099a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f563c06b750 0x7f563c0cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.688+0000 7f564119b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f563c0d12c0 0x7f563c06b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55474/0 (socket says 172.21.15.49:55474) 2023-07-18T03:39:23.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.688+0000 7f564119b700 1 -- 172.21.15.49:0/286047406 learned_addr learned my addr 172.21.15.49:0/286047406 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:23.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.688+0000 7f564199c700 1 --2- 172.21.15.49:0/286047406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f563c0cb330 0x7f563c0b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.688+0000 7f564099a700 1 -- 172.21.15.49:0/286047406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f563c0cb330 msgr2=0x7f563c0b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.688+0000 7f564099a700 1 --2- 172.21.15.49:0/286047406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f563c0cb330 0x7f563c0b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.688+0000 7f564099a700 1 -- 172.21.15.49:0/286047406 >> v1:172.21.15.49:6789/0 conn(0x7f563c0d12c0 legacy=0x7f563c06b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.688+0000 7f564099a700 1 -- 172.21.15.49:0/286047406 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c15ced0 con 0x7f563c06b750 2023-07-18T03:39:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.688+0000 7f5633fff700 1 -- 172.21.15.49:0/286047406 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874507145 0 0) 0x7f563c0b0560 con 0x7f563c0d12c0 2023-07-18T03:39:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.689+0000 7f564099a700 1 --2- 172.21.15.49:0/286047406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f563c06b750 0x7f563c0cc220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f562c009840 tx=0x7f562c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595708365c5da57e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.689+0000 7f5633fff700 1 -- 172.21.15.49:0/286047406 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c011830 con 0x7f563c06b750 2023-07-18T03:39:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.689+0000 7f5633fff700 1 -- 172.21.15.49:0/286047406 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c011990 con 0x7f563c06b750 2023-07-18T03:39:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.689+0000 7f5633fff700 1 -- 172.21.15.49:0/286047406 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c011d20 con 0x7f563c06b750 2023-07-18T03:39:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.689+0000 7f56433ff700 1 -- 172.21.15.49:0/286047406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f563c06b750 msgr2=0x7f563c0cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.689+0000 7f56433ff700 1 --2- 172.21.15.49:0/286047406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f563c06b750 0x7f563c0cc220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f562c009840 tx=0x7f562c009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.689+0000 7f56433ff700 1 -- 172.21.15.49:0/286047406 shutdown_connections 2023-07-18T03:39:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.689+0000 7f56433ff700 1 --2- 172.21.15.49:0/286047406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f563c0cb330 0x7f563c0b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.689+0000 7f56433ff700 1 --2- 172.21.15.49:0/286047406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f563c06b750 0x7f563c0cc220 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.689+0000 7f56433ff700 1 -- 172.21.15.49:0/286047406 >> 172.21.15.49:0/286047406 conn(0x7f563c0999e0 msgr2=0x7f563c099de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.689+0000 7f56433ff700 1 -- 172.21.15.49:0/286047406 shutdown_connections 2023-07-18T03:39:23.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.689+0000 7f56433ff700 1 -- 172.21.15.49:0/286047406 wait complete. 2023-07-18T03:39:23.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.691+0000 7f56433ff700 1 Processor -- start 2023-07-18T03:39:23.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.691+0000 7f56433ff700 1 -- start start 2023-07-18T03:39:23.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.692+0000 7f56433ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f563c06b750 0x7f563c145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.692+0000 7f56433ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f563c0cb330 0x7f563c0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.692+0000 7f56433ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f563c0d12c0 0x7f563c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.692+0000 7f56433ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0b4050 con 0x7f563c0d12c0 2023-07-18T03:39:23.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.692+0000 7f56433ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0b5f00 con 0x7f563c0cb330 2023-07-18T03:39:23.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.692+0000 7f56433ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1f23a0 con 0x7f563c06b750 2023-07-18T03:39:23.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f564199c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f563c0d12c0 0x7f563c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f564199c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f563c0d12c0 0x7f563c0b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37846/0 (socket says 172.21.15.49:37846) 2023-07-18T03:39:23.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f564199c700 1 -- 172.21.15.49:0/1182864700 learned_addr learned my addr 172.21.15.49:0/1182864700 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:23.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f564119b700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f563c06b750 0x7f563c145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f564099a700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f563c0cb330 0x7f563c0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f564199c700 1 -- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f563c06b750 msgr2=0x7f563c145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f564199c700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f563c06b750 0x7f563c145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f564199c700 1 -- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f563c0cb330 msgr2=0x7f563c0b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f564199c700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f563c0cb330 0x7f563c0b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f564199c700 1 -- 172.21.15.49:0/1182864700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c0094a0 con 0x7f563c0d12c0 2023-07-18T03:39:23.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f564099a700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f563c0cb330 0x7f563c0b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:23.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f564199c700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f563c0d12c0 0x7f563c0b6390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f563400efc0 tx=0x7f563400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f5631ffb700 1 -- 172.21.15.49:0/1182864700 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634009ba0 con 0x7f563c0d12c0 2023-07-18T03:39:23.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.693+0000 7f5631ffb700 1 -- 172.21.15.49:0/1182864700 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563400f040 con 0x7f563c0d12c0 2023-07-18T03:39:23.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.694+0000 7f5631ffb700 1 -- 172.21.15.49:0/1182864700 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634014480 con 0x7f563c0d12c0 2023-07-18T03:39:23.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.694+0000 7f56433ff700 1 -- 172.21.15.49:0/1182864700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c1f6a20 con 0x7f563c0d12c0 2023-07-18T03:39:23.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.694+0000 7f56433ff700 1 -- 172.21.15.49:0/1182864700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c0b58b0 con 0x7f563c0d12c0 2023-07-18T03:39:23.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.695+0000 7f56433ff700 1 -- 172.21.15.49:0/1182864700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c06c370 con 0x7f563c0d12c0 2023-07-18T03:39:23.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.696+0000 7f5631ffb700 1 -- 172.21.15.49:0/1182864700 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f56340145e0 con 0x7f563c0d12c0 2023-07-18T03:39:23.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.696+0000 7f5631ffb700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f562006ace0 0x7f562006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.697+0000 7f5631ffb700 1 -- 172.21.15.49:0/1182864700 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5634089d80 con 0x7f563c0d12c0 2023-07-18T03:39:23.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.697+0000 7f564119b700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f562006ace0 0x7f562006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.698+0000 7f564119b700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f562006ace0 0x7f562006d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5638000f60 tx=0x7f5638007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.701+0000 7f5631ffb700 1 -- 172.21.15.49:0/1182864700 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5634054df0 con 0x7f563c0d12c0 2023-07-18T03:39:23.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.909+0000 7f56433ff700 1 -- 172.21.15.49:0/1182864700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f563c0b6070 con 0x7f563c0d12c0 2023-07-18T03:39:23.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.909+0000 7f5631ffb700 1 -- 172.21.15.49:0/1182864700 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f56340180a0 con 0x7f563c0d12c0 2023-07-18T03:39:23.904 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:39:23.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.911+0000 7f56433ff700 1 -- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f562006ace0 msgr2=0x7f562006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.911+0000 7f56433ff700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f562006ace0 0x7f562006d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5638000f60 tx=0x7f5638007be0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.911+0000 7f56433ff700 1 -- 172.21.15.49:0/1182864700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f563c0d12c0 msgr2=0x7f563c0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.911+0000 7f56433ff700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f563c0d12c0 0x7f563c0b6390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f563400efc0 tx=0x7f563400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.911+0000 7f56433ff700 1 -- 172.21.15.49:0/1182864700 shutdown_connections 2023-07-18T03:39:23.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.911+0000 7f56433ff700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f562006ace0 0x7f562006d190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.911+0000 7f56433ff700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f563c0d12c0 0x7f563c0b6390 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.911+0000 7f56433ff700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f563c06b750 0x7f563c145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.911+0000 7f56433ff700 1 --2- 172.21.15.49:0/1182864700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f563c0cb330 0x7f563c0b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.911+0000 7f56433ff700 1 -- 172.21.15.49:0/1182864700 >> 172.21.15.49:0/1182864700 conn(0x7f563c0999e0 msgr2=0x7f563c06b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.911+0000 7f56433ff700 1 -- 172.21.15.49:0/1182864700 shutdown_connections 2023-07-18T03:39:23.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:23.912+0000 7f56433ff700 1 -- 172.21.15.49:0/1182864700 wait complete. 2023-07-18T03:39:23.924 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:39:24.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.140+0000 7f2662fff700 1 Processor -- start 2023-07-18T03:39:24.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.140+0000 7f2662fff700 1 -- start start 2023-07-18T03:39:24.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.142+0000 7f2662fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f265c0b8a60 0x7f265c0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.142+0000 7f2662fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f265c0b4530 0x7f265c0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.142+0000 7f2662fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f265c0d1050 0x7f265c0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.142+0000 7f2662fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265c0d14d0 con 0x7f265c0d1050 2023-07-18T03:39:24.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.142+0000 7f2662fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265c0b8e30 con 0x7f265c0b8a60 2023-07-18T03:39:24.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.142+0000 7f2662fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0afaa0 con 0x7f265c0b4530 2023-07-18T03:39:24.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.142+0000 7f266159c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f265c0d1050 0x7f265c0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.142+0000 7f266159c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f265c0d1050 0x7f265c0b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37860/0 (socket says 172.21.15.49:37860) 2023-07-18T03:39:24.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.142+0000 7f266159c700 1 -- 172.21.15.49:0/213503042 learned_addr learned my addr 172.21.15.49:0/213503042 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:24.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.142+0000 7f2660d9b700 1 --2- 172.21.15.49:0/213503042 >> v2:172.21.15.107:3300/0 conn(0x7f265c0b8a60 0x7f265c0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.142+0000 7f265bfff700 1 --2- 172.21.15.49:0/213503042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f265c0b4530 0x7f265c0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.143+0000 7f266159c700 1 -- 172.21.15.49:0/213503042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f265c0b4530 msgr2=0x7f265c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.143+0000 7f266159c700 1 --2- 172.21.15.49:0/213503042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f265c0b4530 0x7f265c0d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.143+0000 7f266159c700 1 -- 172.21.15.49:0/213503042 >> v2:172.21.15.107:3300/0 conn(0x7f265c0b8a60 msgr2=0x7f265c0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.143+0000 7f266159c700 1 --2- 172.21.15.49:0/213503042 >> v2:172.21.15.107:3300/0 conn(0x7f265c0b8a60 0x7f265c0b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.143+0000 7f266159c700 1 -- 172.21.15.49:0/213503042 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c0afd80 con 0x7f265c0d1050 2023-07-18T03:39:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.143+0000 7f2660d9b700 1 --2- 172.21.15.49:0/213503042 >> v2:172.21.15.107:3300/0 conn(0x7f265c0b8a60 0x7f265c0b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.143+0000 7f265bfff700 1 --2- 172.21.15.49:0/213503042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f265c0b4530 0x7f265c0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.143+0000 7f266159c700 1 --2- 172.21.15.49:0/213503042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f265c0d1050 0x7f265c0b25d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f265c0b5b00 tx=0x7f265400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccd7777f7895d12 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.143+0000 7f265b7fe700 1 -- 172.21.15.49:0/213503042 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265400e070 con 0x7f265c0d1050 2023-07-18T03:39:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.143+0000 7f265b7fe700 1 -- 172.21.15.49:0/213503042 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265400b9c0 con 0x7f265c0d1050 2023-07-18T03:39:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.143+0000 7f265b7fe700 1 -- 172.21.15.49:0/213503042 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265400bd70 con 0x7f265c0d1050 2023-07-18T03:39:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.144+0000 7f2662fff700 1 -- 172.21.15.49:0/213503042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f265c0d1050 msgr2=0x7f265c0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.144+0000 7f2662fff700 1 --2- 172.21.15.49:0/213503042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f265c0d1050 0x7f265c0b25d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f265c0b5b00 tx=0x7f265400b330 comp rx=0 tx=0).stop 2023-07-18T03:39:24.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.144+0000 7f2662fff700 1 -- 172.21.15.49:0/213503042 shutdown_connections 2023-07-18T03:39:24.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.144+0000 7f2662fff700 1 --2- 172.21.15.49:0/213503042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f265c0d1050 0x7f265c0b25d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.144+0000 7f2662fff700 1 --2- 172.21.15.49:0/213503042 >> v2:172.21.15.107:3300/0 conn(0x7f265c0b8a60 0x7f265c0b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.144+0000 7f2662fff700 1 --2- 172.21.15.49:0/213503042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f265c0b4530 0x7f265c0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.144+0000 7f2662fff700 1 -- 172.21.15.49:0/213503042 >> 172.21.15.49:0/213503042 conn(0x7f265c09c1b0 msgr2=0x7f265c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:24.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.144+0000 7f2662fff700 1 -- 172.21.15.49:0/213503042 shutdown_connections 2023-07-18T03:39:24.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.144+0000 7f2662fff700 1 -- 172.21.15.49:0/213503042 wait complete. 2023-07-18T03:39:24.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.146+0000 7f2662fff700 1 Processor -- start 2023-07-18T03:39:24.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.146+0000 7f2662fff700 1 -- start start 2023-07-18T03:39:24.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.147+0000 7f2662fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f265c0b4530 0x7f265c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.147+0000 7f2662fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f265c0b8a60 0x7f265c15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.147+0000 7f2662fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f265c0d1050 0x7f265c1514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.147+0000 7f2662fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0cb460 con 0x7f265c0b8a60 2023-07-18T03:39:24.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.147+0000 7f2662fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c15fe30 con 0x7f265c0b4530 2023-07-18T03:39:24.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.147+0000 7f2662fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0ca5f0 con 0x7f265c0d1050 2023-07-18T03:39:24.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.147+0000 7f265bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f265c0b8a60 0x7f265c15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.147+0000 7f265bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f265c0b8a60 0x7f265c15ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37876/0 (socket says 172.21.15.49:37876) 2023-07-18T03:39:24.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.147+0000 7f265bfff700 1 -- 172.21.15.49:0/2757318525 learned_addr learned my addr 172.21.15.49:0/2757318525 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:24.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.147+0000 7f266159c700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f265c0d1050 0x7f265c1514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.147+0000 7f2660d9b700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f265c0b4530 0x7f265c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.148+0000 7f265bfff700 1 -- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f265c0d1050 msgr2=0x7f265c1514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.148+0000 7f265bfff700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f265c0d1050 0x7f265c1514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.148+0000 7f265bfff700 1 -- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f265c0b4530 msgr2=0x7f265c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.148+0000 7f265bfff700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f265c0b4530 0x7f265c0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.148+0000 7f265bfff700 1 -- 172.21.15.49:0/2757318525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2654009b30 con 0x7f265c0b8a60 2023-07-18T03:39:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.148+0000 7f2660d9b700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f265c0b4530 0x7f265c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.148+0000 7f265bfff700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f265c0b8a60 0x7f265c15ad80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f264800c750 tx=0x7f264800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.148+0000 7f26597fa700 1 -- 172.21.15.49:0/2757318525 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2648011800 con 0x7f265c0b8a60 2023-07-18T03:39:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.148+0000 7f26597fa700 1 -- 172.21.15.49:0/2757318525 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2648011960 con 0x7f265c0b8a60 2023-07-18T03:39:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.148+0000 7f26597fa700 1 -- 172.21.15.49:0/2757318525 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2648011cd0 con 0x7f265c0b8a60 2023-07-18T03:39:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.148+0000 7f2662fff700 1 -- 172.21.15.49:0/2757318525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265c0ca8d0 con 0x7f265c0b8a60 2023-07-18T03:39:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.148+0000 7f2662fff700 1 -- 172.21.15.49:0/2757318525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265c0cae90 con 0x7f265c0b8a60 2023-07-18T03:39:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.149+0000 7f2662fff700 1 -- 172.21.15.49:0/2757318525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265c0cb5d0 con 0x7f265c0b8a60 2023-07-18T03:39:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.150+0000 7f26597fa700 1 -- 172.21.15.49:0/2757318525 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2648021350 con 0x7f265c0b8a60 2023-07-18T03:39:24.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.151+0000 7f26597fa700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f264006ace0 0x7f264006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.152+0000 7f26597fa700 1 -- 172.21.15.49:0/2757318525 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2648089350 con 0x7f265c0b8a60 2023-07-18T03:39:24.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.152+0000 7f2660d9b700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f264006ace0 0x7f264006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.153+0000 7f2660d9b700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f264006ace0 0x7f264006d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2650006fd0 tx=0x7f2650008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.156+0000 7f26597fa700 1 -- 172.21.15.49:0/2757318525 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2648057ee0 con 0x7f265c0b8a60 2023-07-18T03:39:24.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.363+0000 7f2662fff700 1 -- 172.21.15.49:0/2757318525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f265c074350 con 0x7f265c0b8a60 2023-07-18T03:39:24.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.363+0000 7f26597fa700 1 -- 172.21.15.49:0/2757318525 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2648057a70 con 0x7f265c0b8a60 2023-07-18T03:39:24.358 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:39:24.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.366+0000 7f2662fff700 1 -- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f264006ace0 msgr2=0x7f264006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.366+0000 7f2662fff700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f264006ace0 0x7f264006d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2650006fd0 tx=0x7f2650008040 comp rx=0 tx=0).stop 2023-07-18T03:39:24.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.366+0000 7f2662fff700 1 -- 172.21.15.49:0/2757318525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f265c0b8a60 msgr2=0x7f265c15ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.366+0000 7f2662fff700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f265c0b8a60 0x7f265c15ad80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f264800c750 tx=0x7f264800ca60 comp rx=0 tx=0).stop 2023-07-18T03:39:24.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.366+0000 7f2662fff700 1 -- 172.21.15.49:0/2757318525 shutdown_connections 2023-07-18T03:39:24.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.366+0000 7f2662fff700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f265c0d1050 0x7f265c1514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.366+0000 7f2662fff700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f265c0b4530 0x7f265c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.366+0000 7f2662fff700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f264006ace0 0x7f264006d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.366+0000 7f2662fff700 1 --2- 172.21.15.49:0/2757318525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f265c0b8a60 0x7f265c15ad80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.366+0000 7f2662fff700 1 -- 172.21.15.49:0/2757318525 >> 172.21.15.49:0/2757318525 conn(0x7f265c09c1b0 msgr2=0x7f265c15c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:24.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.366+0000 7f2662fff700 1 -- 172.21.15.49:0/2757318525 shutdown_connections 2023-07-18T03:39:24.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.366+0000 7f2662fff700 1 -- 172.21.15.49:0/2757318525 wait complete. 2023-07-18T03:39:24.379 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:39:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.595+0000 7f51dbbff700 1 Processor -- start 2023-07-18T03:39:24.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.595+0000 7f51dbbff700 1 -- start start 2023-07-18T03:39:24.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.597+0000 7f51dbbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 0x7f51d40b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.597+0000 7f51dbbff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f51d40b4530 0x7f51d40d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.597+0000 7f51dbbff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f51d40d1050 0x7f51d40b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.597+0000 7f51dbbff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d40d14d0 con 0x7f51d40d1050 2023-07-18T03:39:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.597+0000 7f51dbbff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d40b8e30 con 0x7f51d40b4530 2023-07-18T03:39:24.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.597+0000 7f51dbbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40afaa0 con 0x7f51d40b8a60 2023-07-18T03:39:24.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.597+0000 7f51d999b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 0x7f51d40b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.597+0000 7f51d999b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 0x7f51d40b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:35266/0 (socket says 172.21.15.49:35266) 2023-07-18T03:39:24.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.597+0000 7f51d999b700 1 -- 172.21.15.49:0/43819051 learned_addr learned my addr 172.21.15.49:0/43819051 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:24.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.598+0000 7f51d919a700 1 --2- 172.21.15.49:0/43819051 >> v2:172.21.15.107:3300/0 conn(0x7f51d40b4530 0x7f51d40d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.598+0000 7f51da19c700 1 --2- 172.21.15.49:0/43819051 >> v2:172.21.15.49:3300/0 conn(0x7f51d40d1050 0x7f51d40b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.598+0000 7f51d999b700 1 -- 172.21.15.49:0/43819051 >> v2:172.21.15.107:3300/0 conn(0x7f51d40b4530 msgr2=0x7f51d40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:24.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.598+0000 7f51d999b700 1 --2- 172.21.15.49:0/43819051 >> v2:172.21.15.107:3300/0 conn(0x7f51d40b4530 0x7f51d40d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.598+0000 7f51d999b700 1 -- 172.21.15.49:0/43819051 >> v2:172.21.15.49:3300/0 conn(0x7f51d40d1050 msgr2=0x7f51d40b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:24.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.598+0000 7f51d999b700 1 --2- 172.21.15.49:0/43819051 >> v2:172.21.15.49:3300/0 conn(0x7f51d40d1050 0x7f51d40b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.598+0000 7f51d999b700 1 -- 172.21.15.49:0/43819051 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d40afd80 con 0x7f51d40b8a60 2023-07-18T03:39:24.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.598+0000 7f51da19c700 1 --2- 172.21.15.49:0/43819051 >> v2:172.21.15.49:3300/0 conn(0x7f51d40d1050 0x7f51d40b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:24.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.598+0000 7f51d999b700 1 --2- 172.21.15.49:0/43819051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 0x7f51d40b3ff0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51d40b5b30 tx=0x7f51d0009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dfe9e33f68fbe0b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:24.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.599+0000 7f51d8999700 1 -- 172.21.15.49:0/43819051 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0011830 con 0x7f51d40b8a60 2023-07-18T03:39:24.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.599+0000 7f51d8999700 1 -- 172.21.15.49:0/43819051 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d0011990 con 0x7f51d40b8a60 2023-07-18T03:39:24.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.599+0000 7f51d8999700 1 -- 172.21.15.49:0/43819051 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0011d20 con 0x7f51d40b8a60 2023-07-18T03:39:24.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.599+0000 7f51dbbff700 1 -- 172.21.15.49:0/43819051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 msgr2=0x7f51d40b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.599+0000 7f51dbbff700 1 --2- 172.21.15.49:0/43819051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 0x7f51d40b3ff0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51d40b5b30 tx=0x7f51d0009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.600+0000 7f51dbbff700 1 -- 172.21.15.49:0/43819051 shutdown_connections 2023-07-18T03:39:24.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.600+0000 7f51dbbff700 1 --2- 172.21.15.49:0/43819051 >> v2:172.21.15.49:3300/0 conn(0x7f51d40d1050 0x7f51d40b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.600+0000 7f51dbbff700 1 --2- 172.21.15.49:0/43819051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 0x7f51d40b3ff0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.600+0000 7f51dbbff700 1 --2- 172.21.15.49:0/43819051 >> v2:172.21.15.107:3300/0 conn(0x7f51d40b4530 0x7f51d40d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.600+0000 7f51dbbff700 1 -- 172.21.15.49:0/43819051 >> 172.21.15.49:0/43819051 conn(0x7f51d409c1b0 msgr2=0x7f51d40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:24.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.600+0000 7f51dbbff700 1 -- 172.21.15.49:0/43819051 shutdown_connections 2023-07-18T03:39:24.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.600+0000 7f51dbbff700 1 -- 172.21.15.49:0/43819051 wait complete. 2023-07-18T03:39:24.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.602+0000 7f51dbbff700 1 Processor -- start 2023-07-18T03:39:24.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.602+0000 7f51dbbff700 1 -- start start 2023-07-18T03:39:24.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.603+0000 7f51dbbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51d40b4530 0x7f51d415f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.603+0000 7f51dbbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 0x7f51d415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.603+0000 7f51dbbff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d40d1050 0x7f51d415e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.603+0000 7f51dbbff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40ce460 con 0x7f51d40d1050 2023-07-18T03:39:24.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.603+0000 7f51dbbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40ce740 con 0x7f51d40b4530 2023-07-18T03:39:24.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.603+0000 7f51dbbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40cea20 con 0x7f51d40b8a60 2023-07-18T03:39:24.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.604+0000 7f51da19c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d40d1050 0x7f51d415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.604+0000 7f51da19c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d40d1050 0x7f51d415e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37888/0 (socket says 172.21.15.49:37888) 2023-07-18T03:39:24.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.604+0000 7f51da19c700 1 -- 172.21.15.49:0/796950472 learned_addr learned my addr 172.21.15.49:0/796950472 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:24.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.604+0000 7f51d999b700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51d40b4530 0x7f51d415f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.604+0000 7f51d919a700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 0x7f51d415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.604+0000 7f51da19c700 1 -- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 msgr2=0x7f51d415e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.604+0000 7f51da19c700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 0x7f51d415e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.604+0000 7f51da19c700 1 -- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51d40b4530 msgr2=0x7f51d415f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.604+0000 7f51da19c700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51d40b4530 0x7f51d415f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.604+0000 7f51da19c700 1 -- 172.21.15.49:0/796950472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d0009550 con 0x7f51d40d1050 2023-07-18T03:39:24.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.604+0000 7f51d919a700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 0x7f51d415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:24.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.604+0000 7f51da19c700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d40d1050 0x7f51d415e710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f51cc007e70 tx=0x7f51cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:24.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.605+0000 7f51ca7fc700 1 -- 172.21.15.49:0/796950472 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc00ddd0 con 0x7f51d40d1050 2023-07-18T03:39:24.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.605+0000 7f51ca7fc700 1 -- 172.21.15.49:0/796950472 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc00f040 con 0x7f51d40d1050 2023-07-18T03:39:24.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.605+0000 7f51dbbff700 1 -- 172.21.15.49:0/796950472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d40ced00 con 0x7f51d40d1050 2023-07-18T03:39:24.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.605+0000 7f51ca7fc700 1 -- 172.21.15.49:0/796950472 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc003c60 con 0x7f51d40d1050 2023-07-18T03:39:24.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.605+0000 7f51dbbff700 1 -- 172.21.15.49:0/796950472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d40cdf80 con 0x7f51d40d1050 2023-07-18T03:39:24.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.606+0000 7f51dbbff700 1 -- 172.21.15.49:0/796950472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d4074350 con 0x7f51d40d1050 2023-07-18T03:39:24.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.607+0000 7f51ca7fc700 1 -- 172.21.15.49:0/796950472 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f51cc004030 con 0x7f51d40d1050 2023-07-18T03:39:24.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.608+0000 7f51ca7fc700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f51b806ace0 0x7f51b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.608+0000 7f51ca7fc700 1 -- 172.21.15.49:0/796950472 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f51cc089610 con 0x7f51d40d1050 2023-07-18T03:39:24.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.608+0000 7f51d999b700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f51b806ace0 0x7f51b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.609+0000 7f51d999b700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f51b806ace0 0x7f51b806d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009c10 tx=0x7f51d0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:24.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.612+0000 7f51ca7fc700 1 -- 172.21.15.49:0/796950472 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f51cc0545e0 con 0x7f51d40d1050 2023-07-18T03:39:24.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.819+0000 7f51dbbff700 1 -- 172.21.15.49:0/796950472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51d40ceef0 con 0x7f51d40d1050 2023-07-18T03:39:24.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.819+0000 7f51ca7fc700 1 -- 172.21.15.49:0/796950472 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51cc057c00 con 0x7f51d40d1050 2023-07-18T03:39:24.814 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:39:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.821+0000 7f51dbbff700 1 -- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f51b806ace0 msgr2=0x7f51b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.821+0000 7f51dbbff700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f51b806ace0 0x7f51b806d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009c10 tx=0x7f51d0008030 comp rx=0 tx=0).stop 2023-07-18T03:39:24.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.821+0000 7f51dbbff700 1 -- 172.21.15.49:0/796950472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d40d1050 msgr2=0x7f51d415e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.821+0000 7f51dbbff700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d40d1050 0x7f51d415e710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f51cc007e70 tx=0x7f51cc00d3b0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.822+0000 7f51dbbff700 1 -- 172.21.15.49:0/796950472 shutdown_connections 2023-07-18T03:39:24.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.822+0000 7f51dbbff700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f51b806ace0 0x7f51b806d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.822+0000 7f51dbbff700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51d40d1050 0x7f51d415e710 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.822+0000 7f51dbbff700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51d40b4530 0x7f51d415f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.822+0000 7f51dbbff700 1 --2- 172.21.15.49:0/796950472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d40b8a60 0x7f51d415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.822+0000 7f51dbbff700 1 -- 172.21.15.49:0/796950472 >> 172.21.15.49:0/796950472 conn(0x7f51d409c1b0 msgr2=0x7f51d415c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.822+0000 7f51dbbff700 1 -- 172.21.15.49:0/796950472 shutdown_connections 2023-07-18T03:39:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:24.822+0000 7f51dbbff700 1 -- 172.21.15.49:0/796950472 wait complete. 2023-07-18T03:39:24.835 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:39:25.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.051+0000 7f79715ff700 1 Processor -- start 2023-07-18T03:39:25.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.051+0000 7f79715ff700 1 -- start start 2023-07-18T03:39:25.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.052+0000 7f79715ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f796c0b4860 0x7f796c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:25.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.052+0000 7f79715ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796c0d02a0 con 0x7f796c0d12c0 2023-07-18T03:39:25.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.052+0000 7f79715ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796c0cfce0 con 0x7f796c0b8660 2023-07-18T03:39:25.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.052+0000 7f79715ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796c0b5240 con 0x7f796c0b4860 2023-07-18T03:39:25.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.053+0000 7f796a7fc700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f796c0b4860 0x7f796c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:25.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.053+0000 7f796b7fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f796c0d12c0 0x7f796c0d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55478/0 (socket says 172.21.15.49:55478) 2023-07-18T03:39:25.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.053+0000 7f796b7fe700 1 -- 172.21.15.49:0/2947607724 learned_addr learned my addr 172.21.15.49:0/2947607724 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:25.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.053+0000 7f796a7fc700 1 -- 172.21.15.49:0/2947607724 >> v1:172.21.15.107:6789/0 conn(0x7f796c0b8660 legacy=0x7f796c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:25.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.053+0000 7f796a7fc700 1 -- 172.21.15.49:0/2947607724 >> v1:172.21.15.49:6789/0 conn(0x7f796c0d12c0 legacy=0x7f796c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:25.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.053+0000 7f796a7fc700 1 -- 172.21.15.49:0/2947607724 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c0cffc0 con 0x7f796c0b4860 2023-07-18T03:39:25.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.053+0000 7f7969ffb700 1 -- 172.21.15.49:0/2947607724 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591864896 0 0) 0x7f796c0d02a0 con 0x7f796c0d12c0 2023-07-18T03:39:25.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.053+0000 7f796a7fc700 1 --2- 172.21.15.49:0/2947607724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f796c0b4860 0x7f796c0b4d00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7960009840 tx=0x7f7960009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a86d3514930bb4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:25.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.054+0000 7f7969ffb700 1 -- 172.21.15.49:0/2947607724 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960004230 con 0x7f796c0b4860 2023-07-18T03:39:25.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.054+0000 7f7969ffb700 1 -- 172.21.15.49:0/2947607724 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7960004390 con 0x7f796c0b4860 2023-07-18T03:39:25.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.054+0000 7f7969ffb700 1 -- 172.21.15.49:0/2947607724 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79600036a0 con 0x7f796c0b4860 2023-07-18T03:39:25.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.054+0000 7f79715ff700 1 -- 172.21.15.49:0/2947607724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f796c0b4860 msgr2=0x7f796c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:25.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.054+0000 7f79715ff700 1 --2- 172.21.15.49:0/2947607724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f796c0b4860 0x7f796c0b4d00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7960009840 tx=0x7f7960009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:25.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.054+0000 7f79715ff700 1 -- 172.21.15.49:0/2947607724 shutdown_connections 2023-07-18T03:39:25.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.054+0000 7f79715ff700 1 --2- 172.21.15.49:0/2947607724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f796c0b4860 0x7f796c0b4d00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.054+0000 7f79715ff700 1 -- 172.21.15.49:0/2947607724 >> 172.21.15.49:0/2947607724 conn(0x7f796c066860 msgr2=0x7f796c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:25.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.054+0000 7f79715ff700 1 -- 172.21.15.49:0/2947607724 shutdown_connections 2023-07-18T03:39:25.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.054+0000 7f79715ff700 1 -- 172.21.15.49:0/2947607724 wait complete. 2023-07-18T03:39:25.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.056+0000 7f79715ff700 1 Processor -- start 2023-07-18T03:39:25.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.056+0000 7f79715ff700 1 -- start start 2023-07-18T03:39:25.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.057+0000 7f79715ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f796c0b4860 0x7f796c1e18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:25.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.057+0000 7f79715ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f796c0b8660 0x7f796c14add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:25.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.057+0000 7f79715ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f796c0d12c0 0x7f796c14b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:25.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.057+0000 7f79715ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c1e1e10 con 0x7f796c0b4860 2023-07-18T03:39:25.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.057+0000 7f79715ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0ca820 con 0x7f796c0b8660 2023-07-18T03:39:25.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.057+0000 7f79715ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0cab00 con 0x7f796c0d12c0 2023-07-18T03:39:25.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f796c0b4860 0x7f796c1e18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:25.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f796c0b4860 0x7f796c1e18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37902/0 (socket says 172.21.15.49:37902) 2023-07-18T03:39:25.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796affd700 1 -- 172.21.15.49:0/1424874180 learned_addr learned my addr 172.21.15.49:0/1424874180 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:25.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796a7fc700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f796c0b8660 0x7f796c14add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796b7fe700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f796c0d12c0 0x7f796c14b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796affd700 1 -- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f796c0d12c0 msgr2=0x7f796c14b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796affd700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f796c0d12c0 0x7f796c14b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796affd700 1 -- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f796c0b8660 msgr2=0x7f796c14add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796affd700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f796c0b8660 0x7f796c14add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796affd700 1 -- 172.21.15.49:0/1424874180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79600094a0 con 0x7f796c0b4860 2023-07-18T03:39:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796b7fe700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f796c0d12c0 0x7f796c14b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796a7fc700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f796c0b8660 0x7f796c14add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.058+0000 7f796affd700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f796c0b4860 0x7f796c1e18d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f795c00c4d0 tx=0x7f795c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.059+0000 7f79537fe700 1 -- 172.21.15.49:0/1424874180 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c011790 con 0x7f796c0b4860 2023-07-18T03:39:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.059+0000 7f79537fe700 1 -- 172.21.15.49:0/1424874180 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c0118f0 con 0x7f796c0b4860 2023-07-18T03:39:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.059+0000 7f79537fe700 1 -- 172.21.15.49:0/1424874180 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c011ca0 con 0x7f796c0b4860 2023-07-18T03:39:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.059+0000 7f79715ff700 1 -- 172.21.15.49:0/1424874180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c0cade0 con 0x7f796c0b4860 2023-07-18T03:39:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.059+0000 7f79715ff700 1 -- 172.21.15.49:0/1424874180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c1e2000 con 0x7f796c0b4860 2023-07-18T03:39:25.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.060+0000 7f79715ff700 1 -- 172.21.15.49:0/1424874180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796c0cb300 con 0x7f796c0b4860 2023-07-18T03:39:25.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.061+0000 7f79537fe700 1 -- 172.21.15.49:0/1424874180 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f795c01c460 con 0x7f796c0b4860 2023-07-18T03:39:25.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.062+0000 7f79537fe700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f795806ace0 0x7f795806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:25.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.062+0000 7f79537fe700 1 -- 172.21.15.49:0/1424874180 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f795c089270 con 0x7f796c0b4860 2023-07-18T03:39:25.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.062+0000 7f796a7fc700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f795806ace0 0x7f795806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:25.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.063+0000 7f796a7fc700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f795806ace0 0x7f795806d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7960000c00 tx=0x7f796001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:25.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.067+0000 7f79537fe700 1 -- 172.21.15.49:0/1424874180 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f796c0cb300 con 0x7f796c0b4860 2023-07-18T03:39:25.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.275+0000 7f79715ff700 1 -- 172.21.15.49:0/1424874180 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f796c074350 con 0x7f796c0b4860 2023-07-18T03:39:25.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.275+0000 7f79537fe700 1 -- 172.21.15.49:0/1424874180 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f795c054370 con 0x7f796c0b4860 2023-07-18T03:39:25.270 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:39:25.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.278+0000 7f79715ff700 1 -- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f795806ace0 msgr2=0x7f795806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.278+0000 7f79715ff700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f795806ace0 0x7f795806d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7960000c00 tx=0x7f796001c040 comp rx=0 tx=0).stop 2023-07-18T03:39:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.278+0000 7f79715ff700 1 -- 172.21.15.49:0/1424874180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f796c0b4860 msgr2=0x7f796c1e18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.278+0000 7f79715ff700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f796c0b4860 0x7f796c1e18d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f795c00c4d0 tx=0x7f795c00c890 comp rx=0 tx=0).stop 2023-07-18T03:39:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.278+0000 7f79715ff700 1 -- 172.21.15.49:0/1424874180 shutdown_connections 2023-07-18T03:39:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.278+0000 7f79715ff700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f796c0d12c0 0x7f796c14b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.278+0000 7f79715ff700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f795806ace0 0x7f795806d190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.278+0000 7f79715ff700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f796c0b4860 0x7f796c1e18d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.278+0000 7f79715ff700 1 --2- 172.21.15.49:0/1424874180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f796c0b8660 0x7f796c14add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.278+0000 7f79715ff700 1 -- 172.21.15.49:0/1424874180 >> 172.21.15.49:0/1424874180 conn(0x7f796c066860 msgr2=0x7f796c0b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:25.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.278+0000 7f79715ff700 1 -- 172.21.15.49:0/1424874180 shutdown_connections 2023-07-18T03:39:25.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.278+0000 7f79715ff700 1 -- 172.21.15.49:0/1424874180 wait complete. 2023-07-18T03:39:25.291 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-18T03:39:25.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.503+0000 7facb75ff700 1 Processor -- start 2023-07-18T03:39:25.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.503+0000 7facb75ff700 1 -- start start 2023-07-18T03:39:25.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.504+0000 7facb75ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7facb00b8a60 0x7facb00b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:25.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.504+0000 7facb75ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7facb00b4530 0x7facb00d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.504+0000 7facb75ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb00d1490 con 0x7facb00d1050 2023-07-18T03:39:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.504+0000 7facb75ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb00b8e30 con 0x7facb00b4530 2023-07-18T03:39:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.504+0000 7facb75ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb00b1e30 con 0x7facb00b8a60 2023-07-18T03:39:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.504+0000 7facb4b9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7facb00b4530 0x7facb00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.504+0000 7facb4b9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7facb00b4530 0x7facb00d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37910/0 (socket says 172.21.15.49:37910) 2023-07-18T03:39:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.504+0000 7facb4b9a700 1 -- 172.21.15.49:0/420839906 learned_addr learned my addr 172.21.15.49:0/420839906 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:25.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.505+0000 7facb539b700 1 --2- 172.21.15.49:0/420839906 >> v2:172.21.15.107:3300/0 conn(0x7facb00b8a60 0x7facb00b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:25.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.505+0000 7facb4b9a700 1 -- 172.21.15.49:0/420839906 >> v2:172.21.15.107:3300/0 conn(0x7facb00b8a60 msgr2=0x7facb00b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:25.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.505+0000 7facb4b9a700 1 --2- 172.21.15.49:0/420839906 >> v2:172.21.15.107:3300/0 conn(0x7facb00b8a60 0x7facb00b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.505+0000 7facb4b9a700 1 -- 172.21.15.49:0/420839906 >> v1:172.21.15.107:6789/0 conn(0x7facb00d1050 legacy=0x7facb00b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:25.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.505+0000 7facb4b9a700 1 -- 172.21.15.49:0/420839906 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb00afaa0 con 0x7facb00b4530 2023-07-18T03:39:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.505+0000 7facb4b9a700 1 --2- 172.21.15.49:0/420839906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb00b4530 0x7facb00d0b10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7facb00b5b00 tx=0x7faca0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbb9966cbbdd4d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.505+0000 7faca7fff700 1 -- 172.21.15.49:0/420839906 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca000b870 con 0x7facb00b4530 2023-07-18T03:39:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.505+0000 7faca7fff700 1 -- 172.21.15.49:0/420839906 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca000b9d0 con 0x7facb00b4530 2023-07-18T03:39:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.506+0000 7faca7fff700 1 -- 172.21.15.49:0/420839906 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca000bd60 con 0x7facb00b4530 2023-07-18T03:39:25.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.506+0000 7facb75ff700 1 -- 172.21.15.49:0/420839906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb00b4530 msgr2=0x7facb00d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.506+0000 7facb75ff700 1 --2- 172.21.15.49:0/420839906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb00b4530 0x7facb00d0b10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7facb00b5b00 tx=0x7faca0009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.506+0000 7facb75ff700 1 -- 172.21.15.49:0/420839906 shutdown_connections 2023-07-18T03:39:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.506+0000 7facb75ff700 1 --2- 172.21.15.49:0/420839906 >> v2:172.21.15.107:3300/0 conn(0x7facb00b8a60 0x7facb00b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.506+0000 7facb75ff700 1 --2- 172.21.15.49:0/420839906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb00b4530 0x7facb00d0b10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.506+0000 7facb75ff700 1 -- 172.21.15.49:0/420839906 >> 172.21.15.49:0/420839906 conn(0x7facb009c1b0 msgr2=0x7facb00d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.506+0000 7facb75ff700 1 -- 172.21.15.49:0/420839906 shutdown_connections 2023-07-18T03:39:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.506+0000 7facb75ff700 1 -- 172.21.15.49:0/420839906 wait complete. 2023-07-18T03:39:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.508+0000 7facb75ff700 1 Processor -- start 2023-07-18T03:39:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.508+0000 7facb75ff700 1 -- start start 2023-07-18T03:39:25.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.509+0000 7facb75ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facb00b4530 0x7facb00cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.509+0000 7facb75ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb00b8a60 0x7facb01f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.509+0000 7facb75ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facb00d1050 0x7facb00cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.509+0000 7facb75ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb015dea0 con 0x7facb00b8a60 2023-07-18T03:39:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.509+0000 7facb75ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb015e110 con 0x7facb00b4530 2023-07-18T03:39:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.509+0000 7facb75ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb015e3f0 con 0x7facb00d1050 2023-07-18T03:39:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.509+0000 7facb4b9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb00b8a60 0x7facb01f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.509+0000 7facb4b9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb00b8a60 0x7facb01f55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37926/0 (socket says 172.21.15.49:37926) 2023-07-18T03:39:25.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.509+0000 7facb4b9a700 1 -- 172.21.15.49:0/2547866716 learned_addr learned my addr 172.21.15.49:0/2547866716 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:25.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7facb539b700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facb00b4530 0x7facb00cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:25.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7facb5b9c700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facb00d1050 0x7facb00cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:25.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7facb4b9a700 1 -- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facb00d1050 msgr2=0x7facb00cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:25.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7facb4b9a700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facb00d1050 0x7facb00cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7facb4b9a700 1 -- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facb00b4530 msgr2=0x7facb00cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:25.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7facb4b9a700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facb00b4530 0x7facb00cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7facb4b9a700 1 -- 172.21.15.49:0/2547866716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca0009550 con 0x7facb00b8a60 2023-07-18T03:39:25.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7facb5b9c700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facb00d1050 0x7facb00cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:25.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7facb4b9a700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb00b8a60 0x7facb01f55b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faca0000c00 tx=0x7faca0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:25.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7faca5ffb700 1 -- 172.21.15.49:0/2547866716 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0004540 con 0x7facb00b8a60 2023-07-18T03:39:25.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7faca5ffb700 1 -- 172.21.15.49:0/2547866716 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca00036a0 con 0x7facb00b8a60 2023-07-18T03:39:25.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7faca5ffb700 1 -- 172.21.15.49:0/2547866716 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0003a10 con 0x7facb00b8a60 2023-07-18T03:39:25.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.510+0000 7facb75ff700 1 -- 172.21.15.49:0/2547866716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb015e6d0 con 0x7facb00b8a60 2023-07-18T03:39:25.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.511+0000 7facb75ff700 1 -- 172.21.15.49:0/2547866716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb00cd1a0 con 0x7facb00b8a60 2023-07-18T03:39:25.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.512+0000 7facb75ff700 1 -- 172.21.15.49:0/2547866716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb0074350 con 0x7facb00b8a60 2023-07-18T03:39:25.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.513+0000 7faca5ffb700 1 -- 172.21.15.49:0/2547866716 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faca0021070 con 0x7facb00b8a60 2023-07-18T03:39:25.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.513+0000 7faca5ffb700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fac9406ace0 0x7fac9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:25.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.514+0000 7faca5ffb700 1 -- 172.21.15.49:0/2547866716 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7faca008ac40 con 0x7facb00b8a60 2023-07-18T03:39:25.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.514+0000 7facb539b700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fac9406ace0 0x7fac9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:25.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.515+0000 7facb539b700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fac9406ace0 0x7fac9406d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7facac0099d0 tx=0x7facac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:25.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.518+0000 7faca5ffb700 1 -- 172.21.15.49:0/2547866716 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7faca0055c10 con 0x7facb00b8a60 2023-07-18T03:39:25.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.743+0000 7facb75ff700 1 -- 172.21.15.49:0/2547866716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7facb00b3f90 con 0x7facb00b8a60 2023-07-18T03:39:25.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.743+0000 7faca5ffb700 1 -- 172.21.15.49:0/2547866716 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7faca0015080 con 0x7facb00b8a60 2023-07-18T03:39:25.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.745+0000 7facb75ff700 1 -- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fac9406ace0 msgr2=0x7fac9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:25.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.745+0000 7facb75ff700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fac9406ace0 0x7fac9406d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7facac0099d0 tx=0x7facac008040 comp rx=0 tx=0).stop 2023-07-18T03:39:25.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.745+0000 7facb75ff700 1 -- 172.21.15.49:0/2547866716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb00b8a60 msgr2=0x7facb01f55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:25.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.745+0000 7facb75ff700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb00b8a60 0x7facb01f55b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faca0000c00 tx=0x7faca0004750 comp rx=0 tx=0).stop 2023-07-18T03:39:25.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.745+0000 7facb75ff700 1 -- 172.21.15.49:0/2547866716 shutdown_connections 2023-07-18T03:39:25.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.745+0000 7facb75ff700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facb00d1050 0x7facb00cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.745+0000 7facb75ff700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facb00b4530 0x7facb00cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.745+0000 7facb75ff700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fac9406ace0 0x7fac9406d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.745+0000 7facb75ff700 1 --2- 172.21.15.49:0/2547866716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facb00b8a60 0x7facb01f55b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:25.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.745+0000 7facb75ff700 1 -- 172.21.15.49:0/2547866716 >> 172.21.15.49:0/2547866716 conn(0x7facb009c1b0 msgr2=0x7facb015c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:25.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.746+0000 7facb75ff700 1 -- 172.21.15.49:0/2547866716 shutdown_connections 2023-07-18T03:39:25.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:25.746+0000 7facb75ff700 1 -- 172.21.15.49:0/2547866716 wait complete. 2023-07-18T03:39:25.742 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-07-18T03:39:25.759 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-07-18T03:39:25.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:25.760 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-07-18T03:39:25.857 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:39:25.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:25.858 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-18T03:39:25.912 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:39:25.913 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:25.958 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:39:25.958 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:26.041 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:39:26.042 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:26.126 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:26.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.375+0000 7f13665ff700 1 Processor -- start 2023-07-18T03:39:26.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.376+0000 7f13665ff700 1 -- start start 2023-07-18T03:39:26.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.377+0000 7f13665ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f136015c950 0x7f13600b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.377+0000 7f13665ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f13600b1e00 0x7f13600d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.377+0000 7f13665ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13600d2ce0 con 0x7f13600b2350 2023-07-18T03:39:26.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.377+0000 7f13665ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13600b3350 con 0x7f13600b1e00 2023-07-18T03:39:26.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.377+0000 7f13665ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13600b3630 con 0x7f136015c950 2023-07-18T03:39:26.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.377+0000 7f135f7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f13600b1e00 0x7f13600d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.377+0000 7f1364b9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f13600b2350 0x7f13600afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55482/0 (socket says 172.21.15.49:55482) 2023-07-18T03:39:26.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.377+0000 7f135f7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f13600b1e00 0x7f13600d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37930/0 (socket says 172.21.15.49:37930) 2023-07-18T03:39:26.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.377+0000 7f1364b9c700 1 -- 172.21.15.49:0/2965594369 learned_addr learned my addr 172.21.15.49:0/2965594369 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:26.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.377+0000 7f135ffff700 1 --2- 172.21.15.49:0/2965594369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f136015c950 0x7f13600b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.378+0000 7f135f7fe700 1 -- 172.21.15.49:0/2965594369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f136015c950 msgr2=0x7f13600b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:26.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.378+0000 7f135f7fe700 1 --2- 172.21.15.49:0/2965594369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f136015c950 0x7f13600b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.378+0000 7f135f7fe700 1 -- 172.21.15.49:0/2965594369 >> v1:172.21.15.49:6789/0 conn(0x7f13600b2350 legacy=0x7f13600afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:26.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.378+0000 7f135f7fe700 1 -- 172.21.15.49:0/2965594369 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13600b3910 con 0x7f13600b1e00 2023-07-18T03:39:26.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.378+0000 7f135f7fe700 1 --2- 172.21.15.49:0/2965594369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13600b1e00 0x7f13600d0b10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1354009840 tx=0x7f1354009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c574a679c5081da server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:26.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.378+0000 7f135effd700 1 -- 172.21.15.49:0/2965594369 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354011830 con 0x7f13600b1e00 2023-07-18T03:39:26.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.378+0000 7f135effd700 1 -- 172.21.15.49:0/2965594369 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1354011990 con 0x7f13600b1e00 2023-07-18T03:39:26.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.378+0000 7f135effd700 1 -- 172.21.15.49:0/2965594369 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354011d20 con 0x7f13600b1e00 2023-07-18T03:39:26.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.379+0000 7f13665ff700 1 -- 172.21.15.49:0/2965594369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13600b1e00 msgr2=0x7f13600d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:26.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.379+0000 7f13665ff700 1 --2- 172.21.15.49:0/2965594369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13600b1e00 0x7f13600d0b10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1354009840 tx=0x7f1354009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:26.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.379+0000 7f13665ff700 1 -- 172.21.15.49:0/2965594369 shutdown_connections 2023-07-18T03:39:26.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.379+0000 7f13665ff700 1 --2- 172.21.15.49:0/2965594369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f136015c950 0x7f13600b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.379+0000 7f13665ff700 1 --2- 172.21.15.49:0/2965594369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f13600b1e00 0x7f13600d0b10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.379+0000 7f13665ff700 1 -- 172.21.15.49:0/2965594369 >> 172.21.15.49:0/2965594369 conn(0x7f136009c1d0 msgr2=0x7f13600d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:26.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.379+0000 7f13665ff700 1 -- 172.21.15.49:0/2965594369 shutdown_connections 2023-07-18T03:39:26.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.379+0000 7f13665ff700 1 -- 172.21.15.49:0/2965594369 wait complete. 2023-07-18T03:39:26.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.381+0000 7f13665ff700 1 Processor -- start 2023-07-18T03:39:26.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.381+0000 7f13665ff700 1 -- start start 2023-07-18T03:39:26.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f13665ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13600b1e00 0x7f1360151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f13665ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13600b2350 0x7f13600cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f13665ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f136015c950 0x7f13600ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f13665ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1360152410 con 0x7f136015c950 2023-07-18T03:39:26.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f13665ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13600cf6e0 con 0x7f13600b1e00 2023-07-18T03:39:26.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f13665ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13600c8bc0 con 0x7f13600b2350 2023-07-18T03:39:26.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f1364b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f136015c950 0x7f13600ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f1364b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f136015c950 0x7f13600ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37938/0 (socket says 172.21.15.49:37938) 2023-07-18T03:39:26.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f1364b9c700 1 -- 172.21.15.49:0/2382733506 learned_addr learned my addr 172.21.15.49:0/2382733506 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:26.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f135ffff700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13600b1e00 0x7f1360151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f135f7fe700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13600b2350 0x7f13600cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f1364b9c700 1 -- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13600b2350 msgr2=0x7f13600cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f1364b9c700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13600b2350 0x7f13600cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f1364b9c700 1 -- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13600b1e00 msgr2=0x7f1360151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f1364b9c700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13600b1e00 0x7f1360151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.382+0000 7f1364b9c700 1 -- 172.21.15.49:0/2382733506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13540094a0 con 0x7f136015c950 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.383+0000 7f1364b9c700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f136015c950 0x7f13600ce470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f135800cbc0 tx=0x7f135800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.383+0000 7f135cff9700 1 -- 172.21.15.49:0/2382733506 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135800a9e0 con 0x7f136015c950 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.383+0000 7f135cff9700 1 -- 172.21.15.49:0/2382733506 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135800ab40 con 0x7f136015c950 2023-07-18T03:39:26.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.383+0000 7f135cff9700 1 -- 172.21.15.49:0/2382733506 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358012400 con 0x7f136015c950 2023-07-18T03:39:26.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.383+0000 7f13665ff700 1 -- 172.21.15.49:0/2382733506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13600cead0 con 0x7f136015c950 2023-07-18T03:39:26.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.383+0000 7f13665ff700 1 -- 172.21.15.49:0/2382733506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13600cf090 con 0x7f136015c950 2023-07-18T03:39:26.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.384+0000 7f13665ff700 1 -- 172.21.15.49:0/2382733506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136004e280 con 0x7f136015c950 2023-07-18T03:39:26.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.385+0000 7f135cff9700 1 -- 172.21.15.49:0/2382733506 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f135801b3b0 con 0x7f136015c950 2023-07-18T03:39:26.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.386+0000 7f135cff9700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f134c06ace0 0x7f134c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.386+0000 7f135cff9700 1 -- 172.21.15.49:0/2382733506 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1358089070 con 0x7f136015c950 2023-07-18T03:39:26.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.387+0000 7f135ffff700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f134c06ace0 0x7f134c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.387+0000 7f135ffff700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f134c06ace0 0x7f134c06d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1350009b20 tx=0x7f13500091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:26.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.391+0000 7f135cff9700 1 -- 172.21.15.49:0/2382733506 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f13580bb330 con 0x7f136015c950 2023-07-18T03:39:26.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.607+0000 7f13665ff700 1 -- 172.21.15.49:0/2382733506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13600cf820 con 0x7f136015c950 2023-07-18T03:39:26.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.608+0000 7f135cff9700 1 -- 172.21.15.49:0/2382733506 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f13580540f0 con 0x7f136015c950 2023-07-18T03:39:26.603 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:26.603 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:23.937174+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:26.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.611+0000 7f13665ff700 1 -- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f134c06ace0 msgr2=0x7f134c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.611+0000 7f13665ff700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f134c06ace0 0x7f134c06d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1350009b20 tx=0x7f13500091c0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.611+0000 7f13665ff700 1 -- 172.21.15.49:0/2382733506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f136015c950 msgr2=0x7f13600ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.611+0000 7f13665ff700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f136015c950 0x7f13600ce470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f135800cbc0 tx=0x7f135800ced0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.611+0000 7f13665ff700 1 -- 172.21.15.49:0/2382733506 shutdown_connections 2023-07-18T03:39:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.611+0000 7f13665ff700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f134c06ace0 0x7f134c06d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.611+0000 7f13665ff700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f136015c950 0x7f13600ce470 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.611+0000 7f13665ff700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13600b1e00 0x7f1360151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.611+0000 7f13665ff700 1 --2- 172.21.15.49:0/2382733506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13600b2350 0x7f13600cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.611+0000 7f13665ff700 1 -- 172.21.15.49:0/2382733506 >> 172.21.15.49:0/2382733506 conn(0x7f136009c1d0 msgr2=0x7f13600d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:26.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.611+0000 7f13665ff700 1 -- 172.21.15.49:0/2382733506 shutdown_connections 2023-07-18T03:39:26.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.611+0000 7f13665ff700 1 -- 172.21.15.49:0/2382733506 wait complete. 2023-07-18T03:39:26.624 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:38:33.773236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:38:35.023692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:39:26.625 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:39:26.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.837+0000 7f87a8fff700 1 Processor -- start 2023-07-18T03:39:26.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.837+0000 7f87a8fff700 1 -- start start 2023-07-18T03:39:26.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.838+0000 7f87a8fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87a40b8a60 0x7f87a40b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a8fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f87a40b4530 0x7f87a40d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a8fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87a40d1050 0x7f87a40b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a8fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a40d14d0 con 0x7f87a40b8a60 2023-07-18T03:39:26.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a8fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a40b8e30 con 0x7f87a40b4530 2023-07-18T03:39:26.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a8fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40afaa0 con 0x7f87a40d1050 2023-07-18T03:39:26.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a259c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87a40b8a60 0x7f87a40b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a259c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87a40b8a60 0x7f87a40b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37944/0 (socket says 172.21.15.49:37944) 2023-07-18T03:39:26.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a259c700 1 -- 172.21.15.49:0/4244695592 learned_addr learned my addr 172.21.15.49:0/4244695592 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:26.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a2d9d700 1 --2- 172.21.15.49:0/4244695592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87a40d1050 0x7f87a40b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f879bfff700 1 --2- 172.21.15.49:0/4244695592 >> v2:172.21.15.107:3300/0 conn(0x7f87a40b4530 0x7f87a40d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a259c700 1 -- 172.21.15.49:0/4244695592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87a40d1050 msgr2=0x7f87a40b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:26.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a259c700 1 --2- 172.21.15.49:0/4244695592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87a40d1050 0x7f87a40b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a259c700 1 -- 172.21.15.49:0/4244695592 >> v2:172.21.15.107:3300/0 conn(0x7f87a40b4530 msgr2=0x7f87a40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:26.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a259c700 1 --2- 172.21.15.49:0/4244695592 >> v2:172.21.15.107:3300/0 conn(0x7f87a40b4530 0x7f87a40d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.839+0000 7f87a259c700 1 -- 172.21.15.49:0/4244695592 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a40afd80 con 0x7f87a40b8a60 2023-07-18T03:39:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.840+0000 7f87a259c700 1 --2- 172.21.15.49:0/4244695592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a40b8a60 0x7f87a40b3ff0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f87a409c8e0 tx=0x7f8794009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f50522c876c55a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.840+0000 7f87a1d9b700 1 -- 172.21.15.49:0/4244695592 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794011830 con 0x7f87a40b8a60 2023-07-18T03:39:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.840+0000 7f87a1d9b700 1 -- 172.21.15.49:0/4244695592 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8794011990 con 0x7f87a40b8a60 2023-07-18T03:39:26.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.840+0000 7f87a1d9b700 1 -- 172.21.15.49:0/4244695592 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794011d20 con 0x7f87a40b8a60 2023-07-18T03:39:26.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.840+0000 7f87a8fff700 1 -- 172.21.15.49:0/4244695592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a40b8a60 msgr2=0x7f87a40b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:26.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.840+0000 7f87a8fff700 1 --2- 172.21.15.49:0/4244695592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a40b8a60 0x7f87a40b3ff0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f87a409c8e0 tx=0x7f8794009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.841+0000 7f87a8fff700 1 -- 172.21.15.49:0/4244695592 shutdown_connections 2023-07-18T03:39:26.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.841+0000 7f87a8fff700 1 --2- 172.21.15.49:0/4244695592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87a40d1050 0x7f87a40b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.841+0000 7f87a8fff700 1 --2- 172.21.15.49:0/4244695592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a40b8a60 0x7f87a40b3ff0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.841+0000 7f87a8fff700 1 --2- 172.21.15.49:0/4244695592 >> v2:172.21.15.107:3300/0 conn(0x7f87a40b4530 0x7f87a40d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.841+0000 7f87a8fff700 1 -- 172.21.15.49:0/4244695592 >> 172.21.15.49:0/4244695592 conn(0x7f87a409c1b0 msgr2=0x7f87a40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.841+0000 7f87a8fff700 1 -- 172.21.15.49:0/4244695592 shutdown_connections 2023-07-18T03:39:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.841+0000 7f87a8fff700 1 -- 172.21.15.49:0/4244695592 wait complete. 2023-07-18T03:39:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.842+0000 7f87a8fff700 1 Processor -- start 2023-07-18T03:39:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.843+0000 7f87a8fff700 1 -- start start 2023-07-18T03:39:26.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.843+0000 7f87a8fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87a40b4530 0x7f87a41f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a8fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87a40b8a60 0x7f87a406b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a8fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a40d1050 0x7f87a40ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a8fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a41f31d0 con 0x7f87a40d1050 2023-07-18T03:39:26.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a8fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a41f66e0 con 0x7f87a40b4530 2023-07-18T03:39:26.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a8fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a41f2c10 con 0x7f87a40b8a60 2023-07-18T03:39:26.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a40d1050 0x7f87a40ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a40d1050 0x7f87a40ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37950/0 (socket says 172.21.15.49:37950) 2023-07-18T03:39:26.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a2d9d700 1 -- 172.21.15.49:0/1634312219 learned_addr learned my addr 172.21.15.49:0/1634312219 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:26.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a259c700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87a40b4530 0x7f87a41f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a2d9d700 1 -- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87a40b8a60 msgr2=0x7f87a406b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:26.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f879bfff700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87a40b8a60 0x7f87a406b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a2d9d700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87a40b8a60 0x7f87a406b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a2d9d700 1 -- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87a40b4530 msgr2=0x7f87a41f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:26.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a2d9d700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87a40b4530 0x7f87a41f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:26.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a2d9d700 1 -- 172.21.15.49:0/1634312219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8794009550 con 0x7f87a40d1050 2023-07-18T03:39:26.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f87a259c700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87a40b4530 0x7f87a41f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:26.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.844+0000 7f879bfff700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87a40b8a60 0x7f87a406b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:26.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.845+0000 7f87a2d9d700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a40d1050 0x7f87a40ca480 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f879c00efc0 tx=0x7f879c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:26.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.845+0000 7f879affd700 1 -- 172.21.15.49:0/1634312219 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c009ba0 con 0x7f87a40d1050 2023-07-18T03:39:26.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.845+0000 7f879affd700 1 -- 172.21.15.49:0/1634312219 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c00f040 con 0x7f87a40d1050 2023-07-18T03:39:26.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.845+0000 7f87a8fff700 1 -- 172.21.15.49:0/1634312219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a41f2ef0 con 0x7f87a40d1050 2023-07-18T03:39:26.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.845+0000 7f879affd700 1 -- 172.21.15.49:0/1634312219 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c014480 con 0x7f87a40d1050 2023-07-18T03:39:26.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.845+0000 7f87a8fff700 1 -- 172.21.15.49:0/1634312219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a41f5c80 con 0x7f87a40d1050 2023-07-18T03:39:26.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.846+0000 7f87a8fff700 1 -- 172.21.15.49:0/1634312219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a4074350 con 0x7f87a40d1050 2023-07-18T03:39:26.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.847+0000 7f879affd700 1 -- 172.21.15.49:0/1634312219 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f879c0145e0 con 0x7f87a40d1050 2023-07-18T03:39:26.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.848+0000 7f879affd700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f878406ace0 0x7f878406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:26.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.848+0000 7f879affd700 1 -- 172.21.15.49:0/1634312219 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f879c089d50 con 0x7f87a40d1050 2023-07-18T03:39:26.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.849+0000 7f87a259c700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f878406ace0 0x7f878406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:26.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.849+0000 7f87a259c700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f878406ace0 0x7f878406d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f87a406bcf0 tx=0x7f8794008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:26.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:26.852+0000 7f879affd700 1 -- 172.21.15.49:0/1634312219 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f879c054db0 con 0x7f87a40d1050 2023-07-18T03:39:27.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.060+0000 7f87a8fff700 1 -- 172.21.15.49:0/1634312219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f87a40b2dd0 con 0x7f87a40d1050 2023-07-18T03:39:27.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.060+0000 7f879affd700 1 -- 172.21.15.49:0/1634312219 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f879c0180a0 con 0x7f87a40d1050 2023-07-18T03:39:27.055 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:39:27.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.062+0000 7f87a8fff700 1 -- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f878406ace0 msgr2=0x7f878406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.062+0000 7f87a8fff700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f878406ace0 0x7f878406d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f87a406bcf0 tx=0x7f8794008030 comp rx=0 tx=0).stop 2023-07-18T03:39:27.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.062+0000 7f87a8fff700 1 -- 172.21.15.49:0/1634312219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a40d1050 msgr2=0x7f87a40ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.062+0000 7f87a8fff700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a40d1050 0x7f87a40ca480 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f879c00efc0 tx=0x7f879c00c3a0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.063+0000 7f87a8fff700 1 -- 172.21.15.49:0/1634312219 shutdown_connections 2023-07-18T03:39:27.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.063+0000 7f87a8fff700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f878406ace0 0x7f878406d190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.063+0000 7f87a8fff700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87a40d1050 0x7f87a40ca480 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.063+0000 7f87a8fff700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87a40b4530 0x7f87a41f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.063+0000 7f87a8fff700 1 --2- 172.21.15.49:0/1634312219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87a40b8a60 0x7f87a406b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.063+0000 7f87a8fff700 1 -- 172.21.15.49:0/1634312219 >> 172.21.15.49:0/1634312219 conn(0x7f87a409c1b0 msgr2=0x7f87a406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:27.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.063+0000 7f87a8fff700 1 -- 172.21.15.49:0/1634312219 shutdown_connections 2023-07-18T03:39:27.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.063+0000 7f87a8fff700 1 -- 172.21.15.49:0/1634312219 wait complete. 2023-07-18T03:39:27.075 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:39:27.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.288+0000 7f28353ff700 1 Processor -- start 2023-07-18T03:39:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.288+0000 7f28353ff700 1 -- start start 2023-07-18T03:39:27.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.289+0000 7f28353ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f28300d0f90 0x7f28300b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:27.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.289+0000 7f28353ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28300b8e30 con 0x7f28300b8a60 2023-07-18T03:39:27.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.289+0000 7f28353ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28300afaa0 con 0x7f28300d0b10 2023-07-18T03:39:27.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.289+0000 7f28353ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28300afd80 con 0x7f28300d0f90 2023-07-18T03:39:27.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.289+0000 7f282effd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f28300b8a60 0x7f28300b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55492/0 (socket says 172.21.15.49:55492) 2023-07-18T03:39:27.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.289+0000 7f282effd700 1 -- 172.21.15.49:0/3271393625 learned_addr learned my addr 172.21.15.49:0/3271393625 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:27.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.290+0000 7f282f7fe700 1 --2- 172.21.15.49:0/3271393625 >> v2:172.21.15.49:3300/0 conn(0x7f28300d0f90 0x7f28300b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:27.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.290+0000 7f282dffb700 1 -- 172.21.15.49:0/3271393625 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454952161 0 0) 0x7f28300b8e30 con 0x7f28300b8a60 2023-07-18T03:39:27.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.290+0000 7f282dffb700 1 -- 172.21.15.49:0/3271393625 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28180035b0 con 0x7f28300b8a60 2023-07-18T03:39:27.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.290+0000 7f282f7fe700 1 -- 172.21.15.49:0/3271393625 >> v1:172.21.15.107:6789/0 conn(0x7f28300d0b10 legacy=0x7f28300b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.290+0000 7f282f7fe700 1 -- 172.21.15.49:0/3271393625 >> v1:172.21.15.49:6789/0 conn(0x7f28300b8a60 legacy=0x7f28300b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.290+0000 7f282f7fe700 1 -- 172.21.15.49:0/3271393625 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28300b0060 con 0x7f28300d0f90 2023-07-18T03:39:27.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.290+0000 7f282f7fe700 1 --2- 172.21.15.49:0/3271393625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28300d0f90 0x7f28300b1530 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2828009d90 tx=0x7f282800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fc2a6dce9e0b38 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:27.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.291+0000 7f282dffb700 1 -- 172.21.15.49:0/3271393625 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282800e070 con 0x7f28300d0f90 2023-07-18T03:39:27.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.291+0000 7f282dffb700 1 -- 172.21.15.49:0/3271393625 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282800bb20 con 0x7f28300d0f90 2023-07-18T03:39:27.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.291+0000 7f282dffb700 1 -- 172.21.15.49:0/3271393625 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828012430 con 0x7f28300d0f90 2023-07-18T03:39:27.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.291+0000 7f28353ff700 1 -- 172.21.15.49:0/3271393625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28300d0f90 msgr2=0x7f28300b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.291+0000 7f28353ff700 1 --2- 172.21.15.49:0/3271393625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28300d0f90 0x7f28300b1530 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2828009d90 tx=0x7f282800b330 comp rx=0 tx=0).stop 2023-07-18T03:39:27.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.291+0000 7f28353ff700 1 -- 172.21.15.49:0/3271393625 shutdown_connections 2023-07-18T03:39:27.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.291+0000 7f28353ff700 1 --2- 172.21.15.49:0/3271393625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28300d0f90 0x7f28300b1530 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.291+0000 7f28353ff700 1 -- 172.21.15.49:0/3271393625 >> 172.21.15.49:0/3271393625 conn(0x7f283009c1b0 msgr2=0x7f28300d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:27.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.291+0000 7f28353ff700 1 -- 172.21.15.49:0/3271393625 shutdown_connections 2023-07-18T03:39:27.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.291+0000 7f28353ff700 1 -- 172.21.15.49:0/3271393625 wait complete. 2023-07-18T03:39:27.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.293+0000 7f28353ff700 1 Processor -- start 2023-07-18T03:39:27.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.293+0000 7f28353ff700 1 -- start start 2023-07-18T03:39:27.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.294+0000 7f28353ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28300b8a60 0x7f28301f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:27.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.294+0000 7f28353ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28300d0b10 0x7f28300c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:27.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.294+0000 7f28353ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28300d0f90 0x7f283015f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:27.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.294+0000 7f28353ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28300c8830 con 0x7f28300d0f90 2023-07-18T03:39:27.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.294+0000 7f28353ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28300c8b10 con 0x7f28300d0b10 2023-07-18T03:39:27.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.294+0000 7f28353ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28300c8df0 con 0x7f28300b8a60 2023-07-18T03:39:27.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.294+0000 7f282f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28300d0f90 0x7f283015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:27.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.294+0000 7f282f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28300d0f90 0x7f283015f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37974/0 (socket says 172.21.15.49:37974) 2023-07-18T03:39:27.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.294+0000 7f282f7fe700 1 -- 172.21.15.49:0/627868068 learned_addr learned my addr 172.21.15.49:0/627868068 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:27.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.295+0000 7f282effd700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28300b8a60 0x7f28301f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:27.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.295+0000 7f282e7fc700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28300d0b10 0x7f28300c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:27.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.295+0000 7f282f7fe700 1 -- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28300b8a60 msgr2=0x7f28301f4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.295+0000 7f282f7fe700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28300b8a60 0x7f28301f4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.295+0000 7f282f7fe700 1 -- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28300d0b10 msgr2=0x7f28300c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.295+0000 7f282f7fe700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28300d0b10 0x7f28300c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.295+0000 7f282f7fe700 1 -- 172.21.15.49:0/627868068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2828009a70 con 0x7f28300d0f90 2023-07-18T03:39:27.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.295+0000 7f282f7fe700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28300d0f90 0x7f283015f200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f28300681d0 tx=0x7f2828004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:27.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.295+0000 7f28177fe700 1 -- 172.21.15.49:0/627868068 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828012430 con 0x7f28300d0f90 2023-07-18T03:39:27.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.295+0000 7f28177fe700 1 -- 172.21.15.49:0/627868068 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2828020070 con 0x7f28300d0f90 2023-07-18T03:39:27.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.295+0000 7f28177fe700 1 -- 172.21.15.49:0/627868068 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828004280 con 0x7f28300d0f90 2023-07-18T03:39:27.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.295+0000 7f28353ff700 1 -- 172.21.15.49:0/627868068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28300c90d0 con 0x7f28300d0f90 2023-07-18T03:39:27.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.296+0000 7f28353ff700 1 -- 172.21.15.49:0/627868068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28300cd7a0 con 0x7f28300d0f90 2023-07-18T03:39:27.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.296+0000 7f28353ff700 1 -- 172.21.15.49:0/627868068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28300c95f0 con 0x7f28300d0f90 2023-07-18T03:39:27.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.297+0000 7f28177fe700 1 -- 172.21.15.49:0/627868068 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f28280043e0 con 0x7f28300d0f90 2023-07-18T03:39:27.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.298+0000 7f28177fe700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f281c06ace0 0x7f281c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:27.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.298+0000 7f28177fe700 1 -- 172.21.15.49:0/627868068 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2828022070 con 0x7f28300d0f90 2023-07-18T03:39:27.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.299+0000 7f282effd700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f281c06ace0 0x7f281c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:27.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.300+0000 7f282effd700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f281c06ace0 0x7f281c06d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f28200011e0 tx=0x7f2820009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:27.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.303+0000 7f28177fe700 1 -- 172.21.15.49:0/627868068 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f28280bd330 con 0x7f28300d0f90 2023-07-18T03:39:27.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.508+0000 7f28353ff700 1 -- 172.21.15.49:0/627868068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2830074350 con 0x7f28300d0f90 2023-07-18T03:39:27.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.509+0000 7f28177fe700 1 -- 172.21.15.49:0/627868068 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f282802c020 con 0x7f28300d0f90 2023-07-18T03:39:27.504 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:39:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.511+0000 7f28353ff700 1 -- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f281c06ace0 msgr2=0x7f281c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.511+0000 7f28353ff700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f281c06ace0 0x7f281c06d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f28200011e0 tx=0x7f2820009040 comp rx=0 tx=0).stop 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.511+0000 7f28353ff700 1 -- 172.21.15.49:0/627868068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28300d0f90 msgr2=0x7f283015f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.511+0000 7f28353ff700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28300d0f90 0x7f283015f200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f28300681d0 tx=0x7f2828004ad0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.511+0000 7f28353ff700 1 -- 172.21.15.49:0/627868068 shutdown_connections 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.511+0000 7f28353ff700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f281c06ace0 0x7f281c06d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.511+0000 7f28353ff700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28300d0f90 0x7f283015f200 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.511+0000 7f28353ff700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28300b8a60 0x7f28301f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.511+0000 7f28353ff700 1 --2- 172.21.15.49:0/627868068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28300d0b10 0x7f28300c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.511+0000 7f28353ff700 1 -- 172.21.15.49:0/627868068 >> 172.21.15.49:0/627868068 conn(0x7f283009c1b0 msgr2=0x7f283006b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.511+0000 7f28353ff700 1 -- 172.21.15.49:0/627868068 shutdown_connections 2023-07-18T03:39:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.511+0000 7f28353ff700 1 -- 172.21.15.49:0/627868068 wait complete. 2023-07-18T03:39:27.524 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:39:27.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.738+0000 7f34c27ff700 1 Processor -- start 2023-07-18T03:39:27.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.738+0000 7f34c27ff700 1 -- start start 2023-07-18T03:39:27.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.739+0000 7f34c27ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f34bc0b4860 0x7f34bc0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:27.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.739+0000 7f34c27ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34bc0d12c0 0x7f34bc0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:27.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.739+0000 7f34c27ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34bc0d02a0 con 0x7f34bc0b8660 2023-07-18T03:39:27.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.739+0000 7f34c27ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34bc0b5240 con 0x7f34bc0b4860 2023-07-18T03:39:27.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.739+0000 7f34c27ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc0cfce0 con 0x7f34bc0d12c0 2023-07-18T03:39:27.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.740+0000 7f34c0d9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34bc0d12c0 0x7f34bc0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:27.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.740+0000 7f34bb7fe700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f34bc0b4860 0x7f34bc0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:27.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.740+0000 7f34bbfff700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f34bc0b8660 0x7f34bc0cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:38898/0 (socket says 172.21.15.49:38898) 2023-07-18T03:39:27.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.740+0000 7f34bbfff700 1 -- 172.21.15.49:0/3192583394 learned_addr learned my addr 172.21.15.49:0/3192583394 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:27.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.740+0000 7f34c0d9c700 1 -- 172.21.15.49:0/3192583394 >> v2:172.21.15.107:3300/0 conn(0x7f34bc0b4860 msgr2=0x7f34bc0b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:27.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.740+0000 7f34c0d9c700 1 --2- 172.21.15.49:0/3192583394 >> v2:172.21.15.107:3300/0 conn(0x7f34bc0b4860 0x7f34bc0b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.740+0000 7f34c0d9c700 1 -- 172.21.15.49:0/3192583394 >> v1:172.21.15.107:6789/0 conn(0x7f34bc0b8660 legacy=0x7f34bc0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.740+0000 7f34c0d9c700 1 -- 172.21.15.49:0/3192583394 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc0cffc0 con 0x7f34bc0d12c0 2023-07-18T03:39:27.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.740+0000 7f34bb7fe700 1 --2- 172.21.15.49:0/3192583394 >> v2:172.21.15.107:3300/0 conn(0x7f34bc0b4860 0x7f34bc0b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:27.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.741+0000 7f34c0d9c700 1 --2- 172.21.15.49:0/3192583394 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34bc0d12c0 0x7f34bc0d16e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009d90 tx=0x7f34b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ef57ba5313be029 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:27.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.741+0000 7f34baffd700 1 -- 172.21.15.49:0/3192583394 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b400e070 con 0x7f34bc0d12c0 2023-07-18T03:39:27.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.741+0000 7f34baffd700 1 -- 172.21.15.49:0/3192583394 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b400bb20 con 0x7f34bc0d12c0 2023-07-18T03:39:27.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.741+0000 7f34baffd700 1 -- 172.21.15.49:0/3192583394 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4012430 con 0x7f34bc0d12c0 2023-07-18T03:39:27.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.741+0000 7f34c27ff700 1 -- 172.21.15.49:0/3192583394 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34bc0d12c0 msgr2=0x7f34bc0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.741+0000 7f34c27ff700 1 --2- 172.21.15.49:0/3192583394 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34bc0d12c0 0x7f34bc0d16e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009d90 tx=0x7f34b400b330 comp rx=0 tx=0).stop 2023-07-18T03:39:27.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.742+0000 7f34c27ff700 1 -- 172.21.15.49:0/3192583394 shutdown_connections 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.742+0000 7f34c27ff700 1 --2- 172.21.15.49:0/3192583394 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34bc0d12c0 0x7f34bc0d16e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.742+0000 7f34c27ff700 1 --2- 172.21.15.49:0/3192583394 >> v2:172.21.15.107:3300/0 conn(0x7f34bc0b4860 0x7f34bc0b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.742+0000 7f34c27ff700 1 -- 172.21.15.49:0/3192583394 >> 172.21.15.49:0/3192583394 conn(0x7f34bc066860 msgr2=0x7f34bc06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.742+0000 7f34c27ff700 1 -- 172.21.15.49:0/3192583394 shutdown_connections 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.742+0000 7f34c27ff700 1 -- 172.21.15.49:0/3192583394 wait complete. 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.743+0000 7f34c27ff700 1 Processor -- start 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.744+0000 7f34c27ff700 1 -- start start 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.744+0000 7f34c27ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34bc0b4860 0x7f34bc14a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:27.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.744+0000 7f34c27ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34bc0b8660 0x7f34bc158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:27.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.744+0000 7f34c27ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34bc0d12c0 0x7f34bc0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:27.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34c27ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc0ca330 con 0x7f34bc0d12c0 2023-07-18T03:39:27.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34c27ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc14a730 con 0x7f34bc0b8660 2023-07-18T03:39:27.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34c27ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1590b0 con 0x7f34bc0b4860 2023-07-18T03:39:27.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34c0d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34bc0d12c0 0x7f34bc0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:27.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34c0d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34bc0d12c0 0x7f34bc0cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37980/0 (socket says 172.21.15.49:37980) 2023-07-18T03:39:27.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34c0d9c700 1 -- 172.21.15.49:0/3798137203 learned_addr learned my addr 172.21.15.49:0/3798137203 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:27.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34bb7fe700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34bc0b8660 0x7f34bc158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:27.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34bbfff700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34bc0b4860 0x7f34bc14a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:27.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34c0d9c700 1 -- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34bc0b4860 msgr2=0x7f34bc14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34c0d9c700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34bc0b4860 0x7f34bc14a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34c0d9c700 1 -- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34bc0b8660 msgr2=0x7f34bc158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34c0d9c700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34bc0b8660 0x7f34bc158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34c0d9c700 1 -- 172.21.15.49:0/3798137203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b00094a0 con 0x7f34bc0d12c0 2023-07-18T03:39:27.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.745+0000 7f34bb7fe700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34bc0b8660 0x7f34bc158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:27.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.746+0000 7f34c0d9c700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34bc0d12c0 0x7f34bc0cbfc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f34b4014040 tx=0x7f34b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:27.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.746+0000 7f34b8ff9700 1 -- 172.21.15.49:0/3798137203 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4012430 con 0x7f34bc0d12c0 2023-07-18T03:39:27.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.746+0000 7f34b8ff9700 1 -- 172.21.15.49:0/3798137203 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b4020070 con 0x7f34bc0d12c0 2023-07-18T03:39:27.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.746+0000 7f34c27ff700 1 -- 172.21.15.49:0/3798137203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b4009a70 con 0x7f34bc0d12c0 2023-07-18T03:39:27.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.746+0000 7f34b8ff9700 1 -- 172.21.15.49:0/3798137203 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4003ea0 con 0x7f34bc0d12c0 2023-07-18T03:39:27.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.746+0000 7f34c27ff700 1 -- 172.21.15.49:0/3798137203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34bc159670 con 0x7f34bc0d12c0 2023-07-18T03:39:27.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.747+0000 7f34c27ff700 1 -- 172.21.15.49:0/3798137203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34bc0ca470 con 0x7f34bc0d12c0 2023-07-18T03:39:27.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.748+0000 7f34b8ff9700 1 -- 172.21.15.49:0/3798137203 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f34b4004280 con 0x7f34bc0d12c0 2023-07-18T03:39:27.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.749+0000 7f34b8ff9700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f34a806ace0 0x7f34a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:27.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.749+0000 7f34b8ff9700 1 -- 172.21.15.49:0/3798137203 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f34b4022070 con 0x7f34bc0d12c0 2023-07-18T03:39:27.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.749+0000 7f34bbfff700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f34a806ace0 0x7f34a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:27.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.750+0000 7f34bbfff700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f34a806ace0 0x7f34a806d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f34ac005910 tx=0x7f34ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:27.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.753+0000 7f34b8ff9700 1 -- 172.21.15.49:0/3798137203 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f34bc0ca470 con 0x7f34bc0d12c0 2023-07-18T03:39:27.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.960+0000 7f34c27ff700 1 -- 172.21.15.49:0/3798137203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34bc0ca470 con 0x7f34bc0d12c0 2023-07-18T03:39:27.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.961+0000 7f34b8ff9700 1 -- 172.21.15.49:0/3798137203 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f34bc0ca470 con 0x7f34bc0d12c0 2023-07-18T03:39:27.956 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:39:27.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.963+0000 7f34c27ff700 1 -- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f34a806ace0 msgr2=0x7f34a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.963+0000 7f34c27ff700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f34a806ace0 0x7f34a806d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f34ac005910 tx=0x7f34ac009040 comp rx=0 tx=0).stop 2023-07-18T03:39:27.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.963+0000 7f34c27ff700 1 -- 172.21.15.49:0/3798137203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34bc0d12c0 msgr2=0x7f34bc0cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.963+0000 7f34c27ff700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34bc0d12c0 0x7f34bc0cbfc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f34b4014040 tx=0x7f34b4004ad0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.963+0000 7f34c27ff700 1 -- 172.21.15.49:0/3798137203 shutdown_connections 2023-07-18T03:39:27.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.963+0000 7f34c27ff700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f34a806ace0 0x7f34a806d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.963+0000 7f34c27ff700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34bc0d12c0 0x7f34bc0cbfc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.963+0000 7f34c27ff700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34bc0b4860 0x7f34bc14a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.963+0000 7f34c27ff700 1 --2- 172.21.15.49:0/3798137203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34bc0b8660 0x7f34bc158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.963+0000 7f34c27ff700 1 -- 172.21.15.49:0/3798137203 >> 172.21.15.49:0/3798137203 conn(0x7f34bc066860 msgr2=0x7f34bc0c8920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:27.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.963+0000 7f34c27ff700 1 -- 172.21.15.49:0/3798137203 shutdown_connections 2023-07-18T03:39:27.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:27.963+0000 7f34c27ff700 1 -- 172.21.15.49:0/3798137203 wait complete. 2023-07-18T03:39:27.976 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:39:28.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.188+0000 7fa6fabff700 1 Processor -- start 2023-07-18T03:39:28.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.188+0000 7fa6fabff700 1 -- start start 2023-07-18T03:39:28.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.190+0000 7fa6fabff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6f40d0b10 0x7fa6f40d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:28.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.190+0000 7fa6fabff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa6f40b0e20 0x7fa6f40b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:28.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.190+0000 7fa6fabff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f40d14a0 con 0x7fa6f40b8a60 2023-07-18T03:39:28.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.190+0000 7fa6fabff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f40b8e30 con 0x7fa6f40b0e20 2023-07-18T03:39:28.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.190+0000 7fa6fabff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40afaa0 con 0x7fa6f40d0b10 2023-07-18T03:39:28.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.190+0000 7fa6f899b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa6f40b8a60 0x7fa6f40b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55502/0 (socket says 172.21.15.49:55502) 2023-07-18T03:39:28.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.190+0000 7fa6f899b700 1 -- 172.21.15.49:0/2044053640 learned_addr learned my addr 172.21.15.49:0/2044053640 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:28.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.190+0000 7fa6f919c700 1 --2- 172.21.15.49:0/2044053640 >> v2:172.21.15.49:3300/0 conn(0x7fa6f40b0e20 0x7fa6f40b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:28.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.190+0000 7fa6f3fff700 1 --2- 172.21.15.49:0/2044053640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6f40d0b10 0x7fa6f40d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:28.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.190+0000 7fa6f37fe700 1 -- 172.21.15.49:0/2044053640 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141888387 0 0) 0x7fa6e8001110 con 0x7fa6f40b8a60 2023-07-18T03:39:28.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.190+0000 7fa6f37fe700 1 -- 172.21.15.49:0/2044053640 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e40035b0 con 0x7fa6f40b8a60 2023-07-18T03:39:28.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.190+0000 7fa6f919c700 1 -- 172.21.15.49:0/2044053640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6f40d0b10 msgr2=0x7fa6f40d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:28.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.191+0000 7fa6f919c700 1 --2- 172.21.15.49:0/2044053640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6f40d0b10 0x7fa6f40d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.191+0000 7fa6f919c700 1 -- 172.21.15.49:0/2044053640 >> v1:172.21.15.49:6789/0 conn(0x7fa6f40b8a60 legacy=0x7fa6f40b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:28.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.191+0000 7fa6f919c700 1 -- 172.21.15.49:0/2044053640 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f40afd80 con 0x7fa6f40b0e20 2023-07-18T03:39:28.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.191+0000 7fa6f3fff700 1 --2- 172.21.15.49:0/2044053640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6f40d0b10 0x7fa6f40d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:28.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.191+0000 7fa6f919c700 1 --2- 172.21.15.49:0/2044053640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6f40b0e20 0x7fa6f40b12a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009d90 tx=0x7fa6ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af9b861958ad63f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:28.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.191+0000 7fa6f37fe700 1 -- 172.21.15.49:0/2044053640 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00e070 con 0x7fa6f40b0e20 2023-07-18T03:39:28.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.191+0000 7fa6f37fe700 1 -- 172.21.15.49:0/2044053640 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec00bb20 con 0x7fa6f40b0e20 2023-07-18T03:39:28.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.191+0000 7fa6f37fe700 1 -- 172.21.15.49:0/2044053640 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00be90 con 0x7fa6f40b0e20 2023-07-18T03:39:28.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.192+0000 7fa6fabff700 1 -- 172.21.15.49:0/2044053640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6f40b0e20 msgr2=0x7fa6f40b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:28.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.192+0000 7fa6fabff700 1 --2- 172.21.15.49:0/2044053640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6f40b0e20 0x7fa6f40b12a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009d90 tx=0x7fa6ec00b330 comp rx=0 tx=0).stop 2023-07-18T03:39:28.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.192+0000 7fa6fabff700 1 -- 172.21.15.49:0/2044053640 shutdown_connections 2023-07-18T03:39:28.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.192+0000 7fa6fabff700 1 --2- 172.21.15.49:0/2044053640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6f40b0e20 0x7fa6f40b12a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.192+0000 7fa6fabff700 1 --2- 172.21.15.49:0/2044053640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6f40d0b10 0x7fa6f40d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.192+0000 7fa6fabff700 1 -- 172.21.15.49:0/2044053640 >> 172.21.15.49:0/2044053640 conn(0x7fa6f409c1b0 msgr2=0x7fa6f40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:28.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.192+0000 7fa6fabff700 1 -- 172.21.15.49:0/2044053640 shutdown_connections 2023-07-18T03:39:28.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.192+0000 7fa6fabff700 1 -- 172.21.15.49:0/2044053640 wait complete. 2023-07-18T03:39:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.194+0000 7fa6fabff700 1 Processor -- start 2023-07-18T03:39:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.194+0000 7fa6fabff700 1 -- start start 2023-07-18T03:39:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.195+0000 7fa6fabff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6f40b0e20 0x7fa6f415f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.195+0000 7fa6fabff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6f40b8a60 0x7fa6f40cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.195+0000 7fa6fabff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6f40d0b10 0x7fa6f41f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.195+0000 7fa6fabff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f4145560 con 0x7fa6f40b0e20 2023-07-18T03:39:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.195+0000 7fa6fabff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40d04b0 con 0x7fa6f40d0b10 2023-07-18T03:39:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.195+0000 7fa6fabff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f4159910 con 0x7fa6f40b8a60 2023-07-18T03:39:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.195+0000 7fa6f899b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6f40b0e20 0x7fa6f415f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.195+0000 7fa6f899b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6f40b0e20 0x7fa6f415f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37996/0 (socket says 172.21.15.49:37996) 2023-07-18T03:39:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.195+0000 7fa6f899b700 1 -- 172.21.15.49:0/3656087756 learned_addr learned my addr 172.21.15.49:0/3656087756 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.195+0000 7fa6f919c700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6f40d0b10 0x7fa6f41f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.195+0000 7fa6f3fff700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6f40b8a60 0x7fa6f40cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.196+0000 7fa6f899b700 1 -- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6f40b8a60 msgr2=0x7fa6f40cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.196+0000 7fa6f899b700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6f40b8a60 0x7fa6f40cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.196+0000 7fa6f899b700 1 -- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6f40d0b10 msgr2=0x7fa6f41f5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.196+0000 7fa6f899b700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6f40d0b10 0x7fa6f41f5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.196+0000 7fa6f899b700 1 -- 172.21.15.49:0/3656087756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec009a70 con 0x7fa6f40b0e20 2023-07-18T03:39:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.196+0000 7fa6f3fff700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6f40b8a60 0x7fa6f40cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.196+0000 7fa6f899b700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6f40b0e20 0x7fa6f415f7f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c500 tx=0x7fa6e800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.196+0000 7fa6f17fa700 1 -- 172.21.15.49:0/3656087756 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8005d70 con 0x7fa6f40b0e20 2023-07-18T03:39:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.196+0000 7fa6f17fa700 1 -- 172.21.15.49:0/3656087756 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e80070d0 con 0x7fa6f40b0e20 2023-07-18T03:39:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.196+0000 7fa6fabff700 1 -- 172.21.15.49:0/3656087756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f40cfef0 con 0x7fa6f40b0e20 2023-07-18T03:39:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.196+0000 7fa6f17fa700 1 -- 172.21.15.49:0/3656087756 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8007480 con 0x7fa6f40b0e20 2023-07-18T03:39:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.196+0000 7fa6fabff700 1 -- 172.21.15.49:0/3656087756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f40ca5a0 con 0x7fa6f40b0e20 2023-07-18T03:39:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.197+0000 7fa6fabff700 1 -- 172.21.15.49:0/3656087756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f40d2440 con 0x7fa6f40b0e20 2023-07-18T03:39:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.198+0000 7fa6f17fa700 1 -- 172.21.15.49:0/3656087756 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa6e80075e0 con 0x7fa6f40b0e20 2023-07-18T03:39:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.200+0000 7fa6f17fa700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa6d806ace0 0x7fa6d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.200+0000 7fa6f17fa700 1 -- 172.21.15.49:0/3656087756 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa6e8088820 con 0x7fa6f40b0e20 2023-07-18T03:39:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.200+0000 7fa6f3fff700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa6d806ace0 0x7fa6d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:28.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.201+0000 7fa6f3fff700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa6d806ace0 0x7fa6d806d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa6f414bc70 tx=0x7fa6e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:28.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.204+0000 7fa6f17fa700 1 -- 172.21.15.49:0/3656087756 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa6e8053890 con 0x7fa6f40b0e20 2023-07-18T03:39:28.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.411+0000 7fa6fabff700 1 -- 172.21.15.49:0/3656087756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6f40d14a0 con 0x7fa6f40b0e20 2023-07-18T03:39:28.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.412+0000 7fa6f17fa700 1 -- 172.21.15.49:0/3656087756 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa6e8017020 con 0x7fa6f40b0e20 2023-07-18T03:39:28.407 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:39:28.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.414+0000 7fa6fabff700 1 -- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa6d806ace0 msgr2=0x7fa6d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:28.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.414+0000 7fa6fabff700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa6d806ace0 0x7fa6d806d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa6f414bc70 tx=0x7fa6e0008040 comp rx=0 tx=0).stop 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.414+0000 7fa6fabff700 1 -- 172.21.15.49:0/3656087756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6f40b0e20 msgr2=0x7fa6f415f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.414+0000 7fa6fabff700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6f40b0e20 0x7fa6f415f7f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c500 tx=0x7fa6e800c810 comp rx=0 tx=0).stop 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.414+0000 7fa6fabff700 1 -- 172.21.15.49:0/3656087756 shutdown_connections 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.414+0000 7fa6fabff700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6f40d0b10 0x7fa6f41f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.414+0000 7fa6fabff700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa6d806ace0 0x7fa6d806d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.414+0000 7fa6fabff700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6f40b0e20 0x7fa6f415f7f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.414+0000 7fa6fabff700 1 --2- 172.21.15.49:0/3656087756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6f40b8a60 0x7fa6f40cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.414+0000 7fa6fabff700 1 -- 172.21.15.49:0/3656087756 >> 172.21.15.49:0/3656087756 conn(0x7fa6f409c1b0 msgr2=0x7fa6f40b7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:28.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.414+0000 7fa6fabff700 1 -- 172.21.15.49:0/3656087756 shutdown_connections 2023-07-18T03:39:28.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.414+0000 7fa6fabff700 1 -- 172.21.15.49:0/3656087756 wait complete. 2023-07-18T03:39:28.427 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-18T03:39:28.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.637+0000 7f3d7cdff700 1 Processor -- start 2023-07-18T03:39:28.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.638+0000 7f3d7cdff700 1 -- start start 2023-07-18T03:39:28.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.639+0000 7f3d7cdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3d780b8a60 0x7f3d780b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:28.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.639+0000 7f3d7cdff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3d780d1220 0x7f3d780b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:28.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.639+0000 7f3d7cdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d780b8e30 con 0x7f3d780b4530 2023-07-18T03:39:28.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.639+0000 7f3d7cdff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d780afaa0 con 0x7f3d780d1220 2023-07-18T03:39:28.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.639+0000 7f3d7cdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d780afd80 con 0x7f3d780b8a60 2023-07-18T03:39:28.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.639+0000 7f3d76d9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3d780d1220 0x7f3d780b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:28.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.639+0000 7f3d76d9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3d780d1220 0x7f3d780b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38002/0 (socket says 172.21.15.49:38002) 2023-07-18T03:39:28.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.639+0000 7f3d76d9d700 1 -- 172.21.15.49:0/2905192082 learned_addr learned my addr 172.21.15.49:0/2905192082 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.639+0000 7f3d7659c700 1 --2- 172.21.15.49:0/2905192082 >> v2:172.21.15.107:3300/0 conn(0x7f3d780b8a60 0x7f3d780b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.640+0000 7f3d76d9d700 1 -- 172.21.15.49:0/2905192082 >> v2:172.21.15.107:3300/0 conn(0x7f3d780b8a60 msgr2=0x7f3d780b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.640+0000 7f3d76d9d700 1 --2- 172.21.15.49:0/2905192082 >> v2:172.21.15.107:3300/0 conn(0x7f3d780b8a60 0x7f3d780b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.640+0000 7f3d76d9d700 1 -- 172.21.15.49:0/2905192082 >> v1:172.21.15.107:6789/0 conn(0x7f3d780b4530 legacy=0x7f3d780d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.640+0000 7f3d76d9d700 1 -- 172.21.15.49:0/2905192082 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d780b0060 con 0x7f3d780d1220 2023-07-18T03:39:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.640+0000 7f3d76d9d700 1 --2- 172.21.15.49:0/2905192082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d780d1220 0x7f3d780b0e20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3d780b5b00 tx=0x7f3d7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d971ec7ba6f53f61 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.640+0000 7f3d7559a700 1 -- 172.21.15.49:0/2905192082 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000e070 con 0x7f3d780d1220 2023-07-18T03:39:28.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.640+0000 7f3d7559a700 1 -- 172.21.15.49:0/2905192082 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7000b9c0 con 0x7f3d780d1220 2023-07-18T03:39:28.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.640+0000 7f3d7559a700 1 -- 172.21.15.49:0/2905192082 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000bd70 con 0x7f3d780d1220 2023-07-18T03:39:28.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.641+0000 7f3d7cdff700 1 -- 172.21.15.49:0/2905192082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d780d1220 msgr2=0x7f3d780b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:28.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.641+0000 7f3d7cdff700 1 --2- 172.21.15.49:0/2905192082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d780d1220 0x7f3d780b0e20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3d780b5b00 tx=0x7f3d7000b330 comp rx=0 tx=0).stop 2023-07-18T03:39:28.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.641+0000 7f3d7cdff700 1 -- 172.21.15.49:0/2905192082 shutdown_connections 2023-07-18T03:39:28.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.641+0000 7f3d7cdff700 1 --2- 172.21.15.49:0/2905192082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d780d1220 0x7f3d780b0e20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.641+0000 7f3d7cdff700 1 --2- 172.21.15.49:0/2905192082 >> v2:172.21.15.107:3300/0 conn(0x7f3d780b8a60 0x7f3d780b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.641+0000 7f3d7cdff700 1 -- 172.21.15.49:0/2905192082 >> 172.21.15.49:0/2905192082 conn(0x7f3d7809c1b0 msgr2=0x7f3d780d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:28.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.641+0000 7f3d7cdff700 1 -- 172.21.15.49:0/2905192082 shutdown_connections 2023-07-18T03:39:28.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.641+0000 7f3d7cdff700 1 -- 172.21.15.49:0/2905192082 wait complete. 2023-07-18T03:39:28.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.643+0000 7f3d7cdff700 1 Processor -- start 2023-07-18T03:39:28.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.643+0000 7f3d7cdff700 1 -- start start 2023-07-18T03:39:28.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.644+0000 7f3d7cdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d780b4530 0x7f3d7809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:28.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.644+0000 7f3d7cdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d780b8a60 0x7f3d780c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:28.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.644+0000 7f3d7cdff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d780d1220 0x7f3d7814b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:28.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.644+0000 7f3d7cdff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7809d320 con 0x7f3d780d1220 2023-07-18T03:39:28.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.644+0000 7f3d7cdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7809f6e0 con 0x7f3d780b4530 2023-07-18T03:39:28.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.644+0000 7f3d7cdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7809f9c0 con 0x7f3d780b8a60 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.644+0000 7f3d76d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d780d1220 0x7f3d7814b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.644+0000 7f3d76d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d780d1220 0x7f3d7814b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38004/0 (socket says 172.21.15.49:38004) 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.644+0000 7f3d76d9d700 1 -- 172.21.15.49:0/754144336 learned_addr learned my addr 172.21.15.49:0/754144336 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.644+0000 7f3d7659c700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d780b4530 0x7f3d7809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.644+0000 7f3d75d9b700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d780b8a60 0x7f3d780c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.645+0000 7f3d76d9d700 1 -- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d780b8a60 msgr2=0x7f3d780c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.645+0000 7f3d76d9d700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d780b8a60 0x7f3d780c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.645+0000 7f3d76d9d700 1 -- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d780b4530 msgr2=0x7f3d7809cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.645+0000 7f3d76d9d700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d780b4530 0x7f3d7809cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.645+0000 7f3d76d9d700 1 -- 172.21.15.49:0/754144336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d70009b30 con 0x7f3d780d1220 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.645+0000 7f3d75d9b700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d780b8a60 0x7f3d780c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.645+0000 7f3d76d9d700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d780d1220 0x7f3d7814b2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3d70012000 tx=0x7f3d70004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.645+0000 7f3d66ffd700 1 -- 172.21.15.49:0/754144336 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70004930 con 0x7f3d780d1220 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.645+0000 7f3d66ffd700 1 -- 172.21.15.49:0/754144336 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d700099b0 con 0x7f3d780d1220 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.645+0000 7f3d7cdff700 1 -- 172.21.15.49:0/754144336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7809fca0 con 0x7f3d780d1220 2023-07-18T03:39:28.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.645+0000 7f3d66ffd700 1 -- 172.21.15.49:0/754144336 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7001b670 con 0x7f3d780d1220 2023-07-18T03:39:28.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.645+0000 7f3d7cdff700 1 -- 172.21.15.49:0/754144336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7814b830 con 0x7f3d780d1220 2023-07-18T03:39:28.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.646+0000 7f3d7cdff700 1 -- 172.21.15.49:0/754144336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d780a0160 con 0x7f3d780d1220 2023-07-18T03:39:28.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.647+0000 7f3d66ffd700 1 -- 172.21.15.49:0/754144336 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3d7001b7d0 con 0x7f3d780d1220 2023-07-18T03:39:28.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.648+0000 7f3d66ffd700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d5806ace0 0x7f3d5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:28.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.649+0000 7f3d66ffd700 1 -- 172.21.15.49:0/754144336 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3d7008a750 con 0x7f3d780d1220 2023-07-18T03:39:28.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.649+0000 7f3d7659c700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d5806ace0 0x7f3d5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:28.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.650+0000 7f3d7659c700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d5806ace0 0x7f3d5806d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3d7809e3e0 tx=0x7f3d68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:28.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.653+0000 7f3d66ffd700 1 -- 172.21.15.49:0/754144336 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3d70055740 con 0x7f3d780d1220 2023-07-18T03:39:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.875+0000 7f3d7cdff700 1 -- 172.21.15.49:0/754144336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3d78074350 con 0x7f3d780d1220 2023-07-18T03:39:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.876+0000 7f3d66ffd700 1 -- 172.21.15.49:0/754144336 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3d70058d60 con 0x7f3d780d1220 2023-07-18T03:39:28.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.878+0000 7f3d7cdff700 1 -- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d5806ace0 msgr2=0x7f3d5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:28.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.878+0000 7f3d7cdff700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d5806ace0 0x7f3d5806d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3d7809e3e0 tx=0x7f3d68008040 comp rx=0 tx=0).stop 2023-07-18T03:39:28.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.878+0000 7f3d7cdff700 1 -- 172.21.15.49:0/754144336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d780d1220 msgr2=0x7f3d7814b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:28.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.878+0000 7f3d7cdff700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d780d1220 0x7f3d7814b2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3d70012000 tx=0x7f3d70004ad0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.878+0000 7f3d7cdff700 1 -- 172.21.15.49:0/754144336 shutdown_connections 2023-07-18T03:39:28.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.878+0000 7f3d7cdff700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d5806ace0 0x7f3d5806d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.878+0000 7f3d7cdff700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d780d1220 0x7f3d7814b2f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.878+0000 7f3d7cdff700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d780b4530 0x7f3d7809cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.878+0000 7f3d7cdff700 1 --2- 172.21.15.49:0/754144336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d780b8a60 0x7f3d780c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:28.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.878+0000 7f3d7cdff700 1 -- 172.21.15.49:0/754144336 >> 172.21.15.49:0/754144336 conn(0x7f3d7809c1b0 msgr2=0x7f3d7806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:28.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.878+0000 7f3d7cdff700 1 -- 172.21.15.49:0/754144336 shutdown_connections 2023-07-18T03:39:28.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:28.878+0000 7f3d7cdff700 1 -- 172.21.15.49:0/754144336 wait complete. 2023-07-18T03:39:28.875 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-07-18T03:39:28.891 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-07-18T03:39:28.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:28.892 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-07-18T03:39:28.981 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:39:28.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:28.982 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-18T03:39:29.027 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:39:29.028 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:29.078 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:39:29.079 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:29.163 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:39:29.163 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:39:29.247 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:29.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.459+0000 7f0cc11ff700 1 Processor -- start 2023-07-18T03:39:29.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.459+0000 7f0cc11ff700 1 -- start start 2023-07-18T03:39:29.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.460+0000 7f0cc11ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0cbc15c710 0x7f0cbc0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:29.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cc11ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cbc069f30 0x7f0cbc06a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:29.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cc11ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cbc0cd320 con 0x7f0cbc06d420 2023-07-18T03:39:29.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cc11ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cbc0ce750 con 0x7f0cbc15c710 2023-07-18T03:39:29.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cc11ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc0cea30 con 0x7f0cbc069f30 2023-07-18T03:39:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cbad9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0cbc15c710 0x7f0cbc0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cbb59e700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0cbc06d420 0x7f0cbc070950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57690/0 (socket says 172.21.15.49:57690) 2023-07-18T03:39:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cbb59e700 1 -- 172.21.15.49:0/4275556946 learned_addr learned my addr 172.21.15.49:0/4275556946 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cbad9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0cbc15c710 0x7f0cbc0cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53518/0 (socket says 172.21.15.49:53518) 2023-07-18T03:39:29.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cb3fff700 1 --2- 172.21.15.49:0/4275556946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cbc069f30 0x7f0cbc06a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:29.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cbad9d700 1 -- 172.21.15.49:0/4275556946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cbc069f30 msgr2=0x7f0cbc06a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cbad9d700 1 --2- 172.21.15.49:0/4275556946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cbc069f30 0x7f0cbc06a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cbad9d700 1 -- 172.21.15.49:0/4275556946 >> v1:172.21.15.49:6789/0 conn(0x7f0cbc06d420 legacy=0x7f0cbc070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:29.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cbad9d700 1 -- 172.21.15.49:0/4275556946 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc0ced10 con 0x7f0cbc15c710 2023-07-18T03:39:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.461+0000 7f0cba59c700 1 -- 172.21.15.49:0/4275556946 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026615548 0 0) 0x7f0cbc0cd320 con 0x7f0cbc06d420 2023-07-18T03:39:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.462+0000 7f0cbad9d700 1 --2- 172.21.15.49:0/4275556946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc15c710 0x7f0cbc0cf510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009840 tx=0x7f0cac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691f4fafe8d5ccf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.462+0000 7f0cba59c700 1 -- 172.21.15.49:0/4275556946 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac011830 con 0x7f0cbc15c710 2023-07-18T03:39:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.462+0000 7f0cba59c700 1 -- 172.21.15.49:0/4275556946 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac011990 con 0x7f0cbc15c710 2023-07-18T03:39:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.462+0000 7f0cba59c700 1 -- 172.21.15.49:0/4275556946 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac011d20 con 0x7f0cbc15c710 2023-07-18T03:39:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.462+0000 7f0cc11ff700 1 -- 172.21.15.49:0/4275556946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc15c710 msgr2=0x7f0cbc0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:29.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.462+0000 7f0cc11ff700 1 --2- 172.21.15.49:0/4275556946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc15c710 0x7f0cbc0cf510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009840 tx=0x7f0cac009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:29.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.463+0000 7f0cc11ff700 1 -- 172.21.15.49:0/4275556946 shutdown_connections 2023-07-18T03:39:29.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.463+0000 7f0cc11ff700 1 --2- 172.21.15.49:0/4275556946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc15c710 0x7f0cbc0cf510 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.463+0000 7f0cc11ff700 1 --2- 172.21.15.49:0/4275556946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cbc069f30 0x7f0cbc06a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.463+0000 7f0cc11ff700 1 -- 172.21.15.49:0/4275556946 >> 172.21.15.49:0/4275556946 conn(0x7f0cbc0b16c0 msgr2=0x7f0cbc0b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:29.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.463+0000 7f0cc11ff700 1 -- 172.21.15.49:0/4275556946 shutdown_connections 2023-07-18T03:39:29.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.463+0000 7f0cc11ff700 1 -- 172.21.15.49:0/4275556946 wait complete. 2023-07-18T03:39:29.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.464+0000 7f0cc11ff700 1 Processor -- start 2023-07-18T03:39:29.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.465+0000 7f0cc11ff700 1 -- start start 2023-07-18T03:39:29.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cc11ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cbc069f30 0x7f0cbc149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:29.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cc11ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc06d420 0x7f0cbc1f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cc11ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cbc0b57b0 0x7f0cbc1f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cc11ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc149dd0 con 0x7f0cbc06d420 2023-07-18T03:39:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cc11ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc1f2ca0 con 0x7f0cbc0b57b0 2023-07-18T03:39:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cc11ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc1f26e0 con 0x7f0cbc069f30 2023-07-18T03:39:29.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cb3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc06d420 0x7f0cbc1f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:29.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cb3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc06d420 0x7f0cbc1f5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53526/0 (socket says 172.21.15.49:53526) 2023-07-18T03:39:29.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cb3fff700 1 -- 172.21.15.49:0/931758244 learned_addr learned my addr 172.21.15.49:0/931758244 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:29.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cbad9d700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cbc069f30 0x7f0cbc149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:29.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cbb59e700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cbc0b57b0 0x7f0cbc1f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:29.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cb3fff700 1 -- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cbc069f30 msgr2=0x7f0cbc149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:29.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cb3fff700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cbc069f30 0x7f0cbc149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cb3fff700 1 -- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cbc0b57b0 msgr2=0x7f0cbc1f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:29.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cb3fff700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cbc0b57b0 0x7f0cbc1f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cb3fff700 1 -- 172.21.15.49:0/931758244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac0094a0 con 0x7f0cbc06d420 2023-07-18T03:39:29.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cbb59e700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cbc0b57b0 0x7f0cbc1f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:29.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.466+0000 7f0cbad9d700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cbc069f30 0x7f0cbc149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:29.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.467+0000 7f0cb3fff700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc06d420 0x7f0cbc1f5c00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400e810 tx=0x7f0ca400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:29.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.467+0000 7f0cb37fe700 1 -- 172.21.15.49:0/931758244 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4013610 con 0x7f0cbc06d420 2023-07-18T03:39:29.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.467+0000 7f0cb37fe700 1 -- 172.21.15.49:0/931758244 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca4013770 con 0x7f0cbc06d420 2023-07-18T03:39:29.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.467+0000 7f0cb37fe700 1 -- 172.21.15.49:0/931758244 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4013b20 con 0x7f0cbc06d420 2023-07-18T03:39:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.467+0000 7f0cc11ff700 1 -- 172.21.15.49:0/931758244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cbc06ac20 con 0x7f0cbc06d420 2023-07-18T03:39:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.467+0000 7f0cc11ff700 1 -- 172.21.15.49:0/931758244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cbc1f5400 con 0x7f0cbc06d420 2023-07-18T03:39:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.468+0000 7f0cc11ff700 1 -- 172.21.15.49:0/931758244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cbc067840 con 0x7f0cbc06d420 2023-07-18T03:39:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.469+0000 7f0cb37fe700 1 -- 172.21.15.49:0/931758244 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ca4013c80 con 0x7f0cbc06d420 2023-07-18T03:39:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.470+0000 7f0cb37fe700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0c9c06ace0 0x7f0c9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.470+0000 7f0cb37fe700 1 -- 172.21.15.49:0/931758244 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0ca4089230 con 0x7f0cbc06d420 2023-07-18T03:39:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.470+0000 7f0cbad9d700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0c9c06ace0 0x7f0c9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.471+0000 7f0cbad9d700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0c9c06ace0 0x7f0c9c06d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0090b0 tx=0x7f0cac01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.475+0000 7f0cb37fe700 1 -- 172.21.15.49:0/931758244 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0ca4054210 con 0x7f0cbc06d420 2023-07-18T03:39:29.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.684+0000 7f0cc11ff700 1 -- 172.21.15.49:0/931758244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cbc04e280 con 0x7f0cbc06d420 2023-07-18T03:39:29.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.685+0000 7f0cb37fe700 1 -- 172.21.15.49:0/931758244 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f0ca4057830 con 0x7f0cbc06d420 2023-07-18T03:39:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:29.680 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:23.937174+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:29.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.687+0000 7f0cc11ff700 1 -- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0c9c06ace0 msgr2=0x7f0c9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:29.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.687+0000 7f0cc11ff700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0c9c06ace0 0x7f0c9c06d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0090b0 tx=0x7f0cac01c040 comp rx=0 tx=0).stop 2023-07-18T03:39:29.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.687+0000 7f0cc11ff700 1 -- 172.21.15.49:0/931758244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc06d420 msgr2=0x7f0cbc1f5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:29.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.687+0000 7f0cc11ff700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc06d420 0x7f0cbc1f5c00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400e810 tx=0x7f0ca400ebd0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.688+0000 7f0cc11ff700 1 -- 172.21.15.49:0/931758244 shutdown_connections 2023-07-18T03:39:29.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.688+0000 7f0cc11ff700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cbc0b57b0 0x7f0cbc1f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.688+0000 7f0cc11ff700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cbc069f30 0x7f0cbc149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.688+0000 7f0cc11ff700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0c9c06ace0 0x7f0c9c06d190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.688+0000 7f0cc11ff700 1 --2- 172.21.15.49:0/931758244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cbc06d420 0x7f0cbc1f5c00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.688+0000 7f0cc11ff700 1 -- 172.21.15.49:0/931758244 >> 172.21.15.49:0/931758244 conn(0x7f0cbc0b16c0 msgr2=0x7f0cbc0b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:29.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.688+0000 7f0cc11ff700 1 -- 172.21.15.49:0/931758244 shutdown_connections 2023-07-18T03:39:29.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.688+0000 7f0cc11ff700 1 -- 172.21.15.49:0/931758244 wait complete. 2023-07-18T03:39:29.701 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:38:33.773236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:38:35.023692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:39:29.702 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:39:29.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.914+0000 7f54b47ff700 1 Processor -- start 2023-07-18T03:39:29.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.914+0000 7f54b47ff700 1 -- start start 2023-07-18T03:39:29.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.915+0000 7f54b47ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f54ac0b4600 0x7f54ac0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:29.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.915+0000 7f54b47ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54ac0cfe40 0x7f54ac0d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:29.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.915+0000 7f54b47ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ac15cd60 con 0x7f54ac0cd540 2023-07-18T03:39:29.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.915+0000 7f54b47ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ac0d1550 con 0x7f54ac0b4600 2023-07-18T03:39:29.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.915+0000 7f54b47ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0d2d90 con 0x7f54ac0cfe40 2023-07-18T03:39:29.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.916+0000 7f54b259b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f54ac0b4600 0x7f54ac0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.916+0000 7f54b2d9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f54ac0cd540 0x7f54ac0cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57692/0 (socket says 172.21.15.49:57692) 2023-07-18T03:39:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.916+0000 7f54b2d9c700 1 -- 172.21.15.49:0/2252191254 learned_addr learned my addr 172.21.15.49:0/2252191254 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.916+0000 7f54b259b700 1 --2- 172.21.15.49:0/2252191254 >> v2:172.21.15.49:3300/0 conn(0x7f54ac0b4600 0x7f54ac0cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53542/0 (socket says 172.21.15.49:53542) 2023-07-18T03:39:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.916+0000 7f54b1d9a700 1 --2- 172.21.15.49:0/2252191254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54ac0cfe40 0x7f54ac0d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.916+0000 7f54b259b700 1 -- 172.21.15.49:0/2252191254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54ac0cfe40 msgr2=0x7f54ac0d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:29.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.916+0000 7f54b259b700 1 --2- 172.21.15.49:0/2252191254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54ac0cfe40 0x7f54ac0d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.916+0000 7f54b259b700 1 -- 172.21.15.49:0/2252191254 >> v1:172.21.15.49:6789/0 conn(0x7f54ac0cd540 legacy=0x7f54ac0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:29.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.916+0000 7f54b259b700 1 -- 172.21.15.49:0/2252191254 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac0cf340 con 0x7f54ac0b4600 2023-07-18T03:39:29.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.916+0000 7f54b1599700 1 -- 172.21.15.49:0/2252191254 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011476174 0 0) 0x7f54ac15cd60 con 0x7f54ac0cd540 2023-07-18T03:39:29.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.917+0000 7f54b259b700 1 --2- 172.21.15.49:0/2252191254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54ac0b4600 0x7f54ac0cf900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009870 tx=0x7f54a8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd2880806793191 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:29.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.917+0000 7f54b1599700 1 -- 172.21.15.49:0/2252191254 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8011830 con 0x7f54ac0b4600 2023-07-18T03:39:29.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.917+0000 7f54b1599700 1 -- 172.21.15.49:0/2252191254 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a8011990 con 0x7f54ac0b4600 2023-07-18T03:39:29.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.917+0000 7f54b1599700 1 -- 172.21.15.49:0/2252191254 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8011d20 con 0x7f54ac0b4600 2023-07-18T03:39:29.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.917+0000 7f54b47ff700 1 -- 172.21.15.49:0/2252191254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54ac0b4600 msgr2=0x7f54ac0cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:29.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.917+0000 7f54b47ff700 1 --2- 172.21.15.49:0/2252191254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54ac0b4600 0x7f54ac0cf900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009870 tx=0x7f54a8009b80 comp rx=0 tx=0).stop 2023-07-18T03:39:29.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.917+0000 7f54b47ff700 1 -- 172.21.15.49:0/2252191254 shutdown_connections 2023-07-18T03:39:29.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.917+0000 7f54b47ff700 1 --2- 172.21.15.49:0/2252191254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54ac0b4600 0x7f54ac0cf900 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.917+0000 7f54b47ff700 1 --2- 172.21.15.49:0/2252191254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54ac0cfe40 0x7f54ac0d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.917+0000 7f54b47ff700 1 -- 172.21.15.49:0/2252191254 >> 172.21.15.49:0/2252191254 conn(0x7f54ac09c200 msgr2=0x7f54ac0d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:29.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.917+0000 7f54b47ff700 1 -- 172.21.15.49:0/2252191254 shutdown_connections 2023-07-18T03:39:29.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.918+0000 7f54b47ff700 1 -- 172.21.15.49:0/2252191254 wait complete. 2023-07-18T03:39:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.919+0000 7f54b47ff700 1 Processor -- start 2023-07-18T03:39:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.919+0000 7f54b47ff700 1 -- start start 2023-07-18T03:39:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.920+0000 7f54b47ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54ac0cd540 0x7f54ac0caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.920+0000 7f54b47ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54ac0cfe40 0x7f54ac158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.920+0000 7f54b47ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54ac1f21a0 0x7f54ac0b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:29.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.920+0000 7f54b47ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0c9fc0 con 0x7f54ac1f21a0 2023-07-18T03:39:29.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.920+0000 7f54b47ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac1f2bf0 con 0x7f54ac0cd540 2023-07-18T03:39:29.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.920+0000 7f54b47ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac1f2ed0 con 0x7f54ac0cfe40 2023-07-18T03:39:29.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.921+0000 7f54b2d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54ac1f21a0 0x7f54ac0b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:29.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.921+0000 7f54b2d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54ac1f21a0 0x7f54ac0b71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53554/0 (socket says 172.21.15.49:53554) 2023-07-18T03:39:29.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.921+0000 7f54b2d9c700 1 -- 172.21.15.49:0/407499502 learned_addr learned my addr 172.21.15.49:0/407499502 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:29.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.921+0000 7f54b1d9a700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54ac0cfe40 0x7f54ac158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.921+0000 7f54b259b700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54ac0cd540 0x7f54ac0caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.921+0000 7f54b2d9c700 1 -- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54ac0cfe40 msgr2=0x7f54ac158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.921+0000 7f54b2d9c700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54ac0cfe40 0x7f54ac158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.921+0000 7f54b2d9c700 1 -- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54ac0cd540 msgr2=0x7f54ac0caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.921+0000 7f54b2d9c700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54ac0cd540 0x7f54ac0caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.921+0000 7f54b2d9c700 1 -- 172.21.15.49:0/407499502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a80094a0 con 0x7f54ac1f21a0 2023-07-18T03:39:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.921+0000 7f54b2d9c700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54ac1f21a0 0x7f54ac0b71c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f54a4002230 tx=0x7f54a400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.922+0000 7f54a2ffd700 1 -- 172.21.15.49:0/407499502 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a400aac0 con 0x7f54ac1f21a0 2023-07-18T03:39:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.922+0000 7f54a2ffd700 1 -- 172.21.15.49:0/407499502 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a400ac40 con 0x7f54ac1f21a0 2023-07-18T03:39:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.922+0000 7f54a2ffd700 1 -- 172.21.15.49:0/407499502 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a40070d0 con 0x7f54ac1f21a0 2023-07-18T03:39:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.922+0000 7f54b47ff700 1 -- 172.21.15.49:0/407499502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac0ca210 con 0x7f54ac1f21a0 2023-07-18T03:39:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.922+0000 7f54b47ff700 1 -- 172.21.15.49:0/407499502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac159f60 con 0x7f54ac1f21a0 2023-07-18T03:39:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.923+0000 7f54b47ff700 1 -- 172.21.15.49:0/407499502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ac074350 con 0x7f54ac1f21a0 2023-07-18T03:39:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.924+0000 7f54a2ffd700 1 -- 172.21.15.49:0/407499502 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f54a4007250 con 0x7f54ac1f21a0 2023-07-18T03:39:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.925+0000 7f54a2ffd700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f549006ace0 0x7f549006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:29.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.925+0000 7f54a2ffd700 1 -- 172.21.15.49:0/407499502 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f54a4088f30 con 0x7f54ac1f21a0 2023-07-18T03:39:29.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.925+0000 7f54b259b700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f549006ace0 0x7f549006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:29.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.926+0000 7f54b259b700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f549006ace0 0x7f549006d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f54a80097c0 tx=0x7f54a801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:29.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:29.930+0000 7f54a2ffd700 1 -- 172.21.15.49:0/407499502 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f54a4053f90 con 0x7f54ac1f21a0 2023-07-18T03:39:30.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.140+0000 7f54b47ff700 1 -- 172.21.15.49:0/407499502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f54ac0d0e60 con 0x7f54ac1f21a0 2023-07-18T03:39:30.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.140+0000 7f54a2ffd700 1 -- 172.21.15.49:0/407499502 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f54a40180a0 con 0x7f54ac1f21a0 2023-07-18T03:39:30.136 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:39:30.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.142+0000 7f54b47ff700 1 -- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f549006ace0 msgr2=0x7f549006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.142+0000 7f54b47ff700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f549006ace0 0x7f549006d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f54a80097c0 tx=0x7f54a801c040 comp rx=0 tx=0).stop 2023-07-18T03:39:30.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.142+0000 7f54b47ff700 1 -- 172.21.15.49:0/407499502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54ac1f21a0 msgr2=0x7f54ac0b71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.142+0000 7f54b47ff700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54ac1f21a0 0x7f54ac0b71c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f54a4002230 tx=0x7f54a400cf50 comp rx=0 tx=0).stop 2023-07-18T03:39:30.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.142+0000 7f54b47ff700 1 -- 172.21.15.49:0/407499502 shutdown_connections 2023-07-18T03:39:30.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.142+0000 7f54b47ff700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f549006ace0 0x7f549006d190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.142+0000 7f54b47ff700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54ac1f21a0 0x7f54ac0b71c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.142+0000 7f54b47ff700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54ac0cd540 0x7f54ac0caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.142+0000 7f54b47ff700 1 --2- 172.21.15.49:0/407499502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54ac0cfe40 0x7f54ac158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.142+0000 7f54b47ff700 1 -- 172.21.15.49:0/407499502 >> 172.21.15.49:0/407499502 conn(0x7f54ac09c200 msgr2=0x7f54ac0cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:30.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.143+0000 7f54b47ff700 1 -- 172.21.15.49:0/407499502 shutdown_connections 2023-07-18T03:39:30.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.143+0000 7f54b47ff700 1 -- 172.21.15.49:0/407499502 wait complete. 2023-07-18T03:39:30.155 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:39:30.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.369+0000 7fb7fdbff700 1 Processor -- start 2023-07-18T03:39:30.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.369+0000 7fb7fdbff700 1 -- start start 2023-07-18T03:39:30.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.370+0000 7fb7fdbff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb7f80d1e40 0x7fb7f80cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.370+0000 7fb7fdbff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f80cd5a0 con 0x7fb7f80c71b0 2023-07-18T03:39:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.370+0000 7fb7fdbff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f80cd7f0 con 0x7fb7f815c860 2023-07-18T03:39:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.370+0000 7fb7fdbff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f8146740 con 0x7fb7f80d1e40 2023-07-18T03:39:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.371+0000 7fb7f7fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb7f80c71b0 0x7fb7f80b49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57704/0 (socket says 172.21.15.49:57704) 2023-07-18T03:39:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.371+0000 7fb7f7fff700 1 -- 172.21.15.49:0/2126396276 learned_addr learned my addr 172.21.15.49:0/2126396276 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:30.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.371+0000 7fb7f6ffd700 1 --2- 172.21.15.49:0/2126396276 >> v2:172.21.15.107:3300/0 conn(0x7fb7f80d1e40 0x7fb7f80cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.371+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613448753 0 0) 0x7fb7f80cd5a0 con 0x7fb7f80c71b0 2023-07-18T03:39:30.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.371+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7e00035b0 con 0x7fb7f80c71b0 2023-07-18T03:39:30.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.371+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2476914453 0 0) 0x7fb7e00035b0 con 0x7fb7f80c71b0 2023-07-18T03:39:30.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.371+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 >> v2:172.21.15.107:3300/0 conn(0x7fb7f80d1e40 msgr2=0x7fb7f80cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:30.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.371+0000 7fb7f67fc700 1 --2- 172.21.15.49:0/2126396276 >> v2:172.21.15.107:3300/0 conn(0x7fb7f80d1e40 0x7fb7f80cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.371+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 >> v1:172.21.15.107:6789/0 conn(0x7fb7f815c860 legacy=0x7fb7f80c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.371+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f80cdad0 con 0x7fb7f80c71b0 2023-07-18T03:39:30.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.371+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7fb7f00019f0 con 0x7fb7f80c71b0 2023-07-18T03:39:30.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.371+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 >> v1:172.21.15.49:6789/0 conn(0x7fb7f80c71b0 legacy=0x7fb7f80b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f67fc700 1 --2- 172.21.15.49:0/2126396276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7e0003a00 0x7fb7e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f67fc700 1 --2- 172.21.15.49:0/2126396276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7e0004850 0x7fb7e0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f67fc700 1 --2- 172.21.15.49:0/2126396276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7e0005200 0x7fb7e000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e000c050 con 0x7fb7e0005200 2023-07-18T03:39:30.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e000c1c0 con 0x7fb7e0004850 2023-07-18T03:39:30.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e000c380 con 0x7fb7e0003a00 2023-07-18T03:39:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f7fff700 1 --2- 172.21.15.49:0/2126396276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7e0005200 0x7fb7e000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f77fe700 1 --2- 172.21.15.49:0/2126396276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7e0003a00 0x7fb7e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f6ffd700 1 --2- 172.21.15.49:0/2126396276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7e0004850 0x7fb7e0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f7fff700 1 -- 172.21.15.49:0/2126396276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7e0003a00 msgr2=0x7fb7e0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f7fff700 1 --2- 172.21.15.49:0/2126396276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7e0003a00 0x7fb7e0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f7fff700 1 -- 172.21.15.49:0/2126396276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7e0004850 msgr2=0x7fb7e0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f7fff700 1 --2- 172.21.15.49:0/2126396276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7e0004850 0x7fb7e0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f7fff700 1 -- 172.21.15.49:0/2126396276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e00035b0 con 0x7fb7e0005200 2023-07-18T03:39:30.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.372+0000 7fb7f7fff700 1 -- 172.21.15.49:0/2126396276 reap_dead start 2023-07-18T03:39:30.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.373+0000 7fb7f7fff700 1 --2- 172.21.15.49:0/2126396276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7e0005200 0x7fb7e000ba10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000a830 tx=0x7fb7e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b93dd5e8306acf0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:30.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.373+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f000a2f0 con 0x7fb7e0005200 2023-07-18T03:39:30.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.373+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f0012620 con 0x7fb7e0005200 2023-07-18T03:39:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.373+0000 7fb7f67fc700 1 -- 172.21.15.49:0/2126396276 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f00129d0 con 0x7fb7e0005200 2023-07-18T03:39:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.373+0000 7fb7fdbff700 1 -- 172.21.15.49:0/2126396276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7e0005200 msgr2=0x7fb7e000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.373+0000 7fb7fdbff700 1 --2- 172.21.15.49:0/2126396276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7e0005200 0x7fb7e000ba10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000a830 tx=0x7fb7e0005880 comp rx=0 tx=0).stop 2023-07-18T03:39:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.373+0000 7fb7fdbff700 1 -- 172.21.15.49:0/2126396276 shutdown_connections 2023-07-18T03:39:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.373+0000 7fb7fdbff700 1 --2- 172.21.15.49:0/2126396276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7e0005200 0x7fb7e000ba10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.373+0000 7fb7fdbff700 1 -- 172.21.15.49:0/2126396276 >> 172.21.15.49:0/2126396276 conn(0x7fb7f8065bd0 msgr2=0x7fb7f8065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:30.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.374+0000 7fb7fdbff700 1 -- 172.21.15.49:0/2126396276 shutdown_connections 2023-07-18T03:39:30.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.374+0000 7fb7fdbff700 1 -- 172.21.15.49:0/2126396276 wait complete. 2023-07-18T03:39:30.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.375+0000 7fb7fdbff700 1 Processor -- start 2023-07-18T03:39:30.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.376+0000 7fb7fdbff700 1 -- start start 2023-07-18T03:39:30.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.376+0000 7fb7fdbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7e0005200 0x7fb7f8158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.376+0000 7fb7fdbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f815ce10 0x7fb7f80cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.376+0000 7fb7fdbff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f815d280 0x7fb7f80cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.376+0000 7fb7fdbff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f815da20 con 0x7fb7f815d280 2023-07-18T03:39:30.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.376+0000 7fb7fdbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f815dd00 con 0x7fb7f815ce10 2023-07-18T03:39:30.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.376+0000 7fb7fdbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f815dfe0 con 0x7fb7e0005200 2023-07-18T03:39:30.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.377+0000 7fb7f7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f815d280 0x7fb7f80cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.377+0000 7fb7f7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f815d280 0x7fb7f80cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53580/0 (socket says 172.21.15.49:53580) 2023-07-18T03:39:30.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.377+0000 7fb7f7fff700 1 -- 172.21.15.49:0/1423404336 learned_addr learned my addr 172.21.15.49:0/1423404336 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:30.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.377+0000 7fb7f77fe700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7e0005200 0x7fb7f8158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.377+0000 7fb7f6ffd700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f815ce10 0x7fb7f80cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.377+0000 7fb7f7fff700 1 -- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7e0005200 msgr2=0x7fb7f8158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.377+0000 7fb7f7fff700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7e0005200 0x7fb7f8158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.377+0000 7fb7f7fff700 1 -- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f815ce10 msgr2=0x7fb7f80cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.377+0000 7fb7f7fff700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f815ce10 0x7fb7f80cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.377+0000 7fb7f7fff700 1 -- 172.21.15.49:0/1423404336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f815e2c0 con 0x7fb7f815d280 2023-07-18T03:39:30.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.377+0000 7fb7f6ffd700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f815ce10 0x7fb7f80cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:30.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.377+0000 7fb7f7fff700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f815d280 0x7fb7f80cc730 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb7f81602d0 tx=0x7fb7f0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:30.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.378+0000 7fb7dffff700 1 -- 172.21.15.49:0/1423404336 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0024070 con 0x7fb7f815d280 2023-07-18T03:39:30.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.378+0000 7fb7dffff700 1 -- 172.21.15.49:0/1423404336 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f00129a0 con 0x7fb7f815d280 2023-07-18T03:39:30.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.378+0000 7fb7dffff700 1 -- 172.21.15.49:0/1423404336 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0012d50 con 0x7fb7f815d280 2023-07-18T03:39:30.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.378+0000 7fb7fdbff700 1 -- 172.21.15.49:0/1423404336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f815e5a0 con 0x7fb7f815d280 2023-07-18T03:39:30.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.378+0000 7fb7fdbff700 1 -- 172.21.15.49:0/1423404336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f80c76d0 con 0x7fb7f815d280 2023-07-18T03:39:30.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.379+0000 7fb7fdbff700 1 -- 172.21.15.49:0/1423404336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f8074350 con 0x7fb7f815d280 2023-07-18T03:39:30.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.380+0000 7fb7dffff700 1 -- 172.21.15.49:0/1423404336 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7f001d420 con 0x7fb7f815d280 2023-07-18T03:39:30.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.381+0000 7fb7dffff700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb7e406ace0 0x7fb7e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.381+0000 7fb7dffff700 1 -- 172.21.15.49:0/1423404336 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb7f008afc0 con 0x7fb7f815d280 2023-07-18T03:39:30.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.381+0000 7fb7f77fe700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb7e406ace0 0x7fb7e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.382+0000 7fb7f77fe700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb7e406ace0 0x7fb7e406d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8000c00 tx=0x7fb7e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:30.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.385+0000 7fb7dffff700 1 -- 172.21.15.49:0/1423404336 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb7f0055fa0 con 0x7fb7f815d280 2023-07-18T03:39:30.407 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:39:30.423 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:39:30.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.593+0000 7fb7fdbff700 1 -- 172.21.15.49:0/1423404336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7f80cdd10 con 0x7fb7f815d280 2023-07-18T03:39:30.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.594+0000 7fb7dffff700 1 -- 172.21.15.49:0/1423404336 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7f00595c0 con 0x7fb7f815d280 2023-07-18T03:39:30.589 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:39:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.596+0000 7fb7fdbff700 1 -- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb7e406ace0 msgr2=0x7fb7e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.596+0000 7fb7fdbff700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb7e406ace0 0x7fb7e406d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8000c00 tx=0x7fb7e8009040 comp rx=0 tx=0).stop 2023-07-18T03:39:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.596+0000 7fb7fdbff700 1 -- 172.21.15.49:0/1423404336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f815d280 msgr2=0x7fb7f80cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.596+0000 7fb7fdbff700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f815d280 0x7fb7f80cc730 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb7f81602d0 tx=0x7fb7f0001d80 comp rx=0 tx=0).stop 2023-07-18T03:39:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.596+0000 7fb7fdbff700 1 -- 172.21.15.49:0/1423404336 shutdown_connections 2023-07-18T03:39:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.596+0000 7fb7fdbff700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb7e406ace0 0x7fb7e406d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.597+0000 7fb7fdbff700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7f815d280 0x7fb7f80cc730 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.597+0000 7fb7fdbff700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7e0005200 0x7fb7f8158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.597+0000 7fb7fdbff700 1 --2- 172.21.15.49:0/1423404336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f815ce10 0x7fb7f80cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.597+0000 7fb7fdbff700 1 -- 172.21.15.49:0/1423404336 >> 172.21.15.49:0/1423404336 conn(0x7fb7f8065bd0 msgr2=0x7fb7f8159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.597+0000 7fb7fdbff700 1 -- 172.21.15.49:0/1423404336 shutdown_connections 2023-07-18T03:39:30.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.597+0000 7fb7fdbff700 1 -- 172.21.15.49:0/1423404336 wait complete. 2023-07-18T03:39:30.610 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:39:30.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.826+0000 7f5b16bff700 1 Processor -- start 2023-07-18T03:39:30.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.827+0000 7f5b16bff700 1 -- start start 2023-07-18T03:39:30.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.828+0000 7f5b16bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5b100d1e40 0x7f5b100cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.828+0000 7f5b16bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b100c71b0 0x7f5b100b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.828+0000 7f5b16bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b100cd5a0 con 0x7f5b1015c860 2023-07-18T03:39:30.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.828+0000 7f5b16bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b10146740 con 0x7f5b100d1e40 2023-07-18T03:39:30.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.828+0000 7f5b16bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b100cd7f0 con 0x7f5b100c71b0 2023-07-18T03:39:30.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.828+0000 7f5b0ffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5b100d1e40 0x7f5b100cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.828+0000 7f5b1499b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5b1015c860 0x7f5b100c6aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57720/0 (socket says 172.21.15.49:57720) 2023-07-18T03:39:30.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.828+0000 7f5b1499b700 1 -- 172.21.15.49:0/253589935 learned_addr learned my addr 172.21.15.49:0/253589935 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:30.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.828+0000 7f5b0ffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5b100d1e40 0x7f5b100cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53584/0 (socket says 172.21.15.49:53584) 2023-07-18T03:39:30.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.828+0000 7f5b1519c700 1 --2- 172.21.15.49:0/253589935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b100c71b0 0x7f5b100b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.829+0000 7f5b0ffff700 1 -- 172.21.15.49:0/253589935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b100c71b0 msgr2=0x7f5b100b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:30.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.829+0000 7f5b0ffff700 1 --2- 172.21.15.49:0/253589935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b100c71b0 0x7f5b100b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.829+0000 7f5b0ffff700 1 -- 172.21.15.49:0/253589935 >> v1:172.21.15.49:6789/0 conn(0x7f5b1015c860 legacy=0x7f5b100c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.829+0000 7f5b0ffff700 1 -- 172.21.15.49:0/253589935 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b100cdad0 con 0x7f5b100d1e40 2023-07-18T03:39:30.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.829+0000 7f5b0f7fe700 1 -- 172.21.15.49:0/253589935 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449356507 0 0) 0x7f5b100cd5a0 con 0x7f5b1015c860 2023-07-18T03:39:30.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.829+0000 7f5b0ffff700 1 --2- 172.21.15.49:0/253589935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b100d1e40 0x7f5b100cf2a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009840 tx=0x7f5afc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94967be9f518576 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:30.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.829+0000 7f5b0f7fe700 1 -- 172.21.15.49:0/253589935 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc011830 con 0x7f5b100d1e40 2023-07-18T03:39:30.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.829+0000 7f5b0f7fe700 1 -- 172.21.15.49:0/253589935 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc011990 con 0x7f5b100d1e40 2023-07-18T03:39:30.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.829+0000 7f5b0f7fe700 1 -- 172.21.15.49:0/253589935 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc011d20 con 0x7f5b100d1e40 2023-07-18T03:39:30.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.830+0000 7f5b16bff700 1 -- 172.21.15.49:0/253589935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b100d1e40 msgr2=0x7f5b100cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.830+0000 7f5b16bff700 1 --2- 172.21.15.49:0/253589935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b100d1e40 0x7f5b100cf2a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009840 tx=0x7f5afc009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:30.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.830+0000 7f5b16bff700 1 -- 172.21.15.49:0/253589935 shutdown_connections 2023-07-18T03:39:30.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.830+0000 7f5b16bff700 1 --2- 172.21.15.49:0/253589935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b100c71b0 0x7f5b100b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.830+0000 7f5b16bff700 1 --2- 172.21.15.49:0/253589935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b100d1e40 0x7f5b100cf2a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.830+0000 7f5b16bff700 1 -- 172.21.15.49:0/253589935 >> 172.21.15.49:0/253589935 conn(0x7f5b10065bd0 msgr2=0x7f5b10065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:30.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.830+0000 7f5b16bff700 1 -- 172.21.15.49:0/253589935 shutdown_connections 2023-07-18T03:39:30.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.830+0000 7f5b16bff700 1 -- 172.21.15.49:0/253589935 wait complete. 2023-07-18T03:39:30.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.832+0000 7f5b16bff700 1 Processor -- start 2023-07-18T03:39:30.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.832+0000 7f5b16bff700 1 -- start start 2023-07-18T03:39:30.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b16bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b100c71b0 0x7f5b10154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b16bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b100d1e40 0x7f5b10158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b16bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b1015c860 0x7f5b10158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b16bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b10154de0 con 0x7f5b100c71b0 2023-07-18T03:39:30.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b16bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b100b82c0 con 0x7f5b1015c860 2023-07-18T03:39:30.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b16bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b10155030 con 0x7f5b100d1e40 2023-07-18T03:39:30.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b1499b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b100c71b0 0x7f5b10154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b1499b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b100c71b0 0x7f5b10154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53596/0 (socket says 172.21.15.49:53596) 2023-07-18T03:39:30.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b1499b700 1 -- 172.21.15.49:0/64170878 learned_addr learned my addr 172.21.15.49:0/64170878 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:30.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b0ffff700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b100d1e40 0x7f5b10158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b1519c700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b1015c860 0x7f5b10158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b1499b700 1 -- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b100d1e40 msgr2=0x7f5b10158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b1499b700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b100d1e40 0x7f5b10158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b1499b700 1 -- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b1015c860 msgr2=0x7f5b10158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:30.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b1499b700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b1015c860 0x7f5b10158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:30.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b1499b700 1 -- 172.21.15.49:0/64170878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc0094a0 con 0x7f5b100c71b0 2023-07-18T03:39:30.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.833+0000 7f5b0ffff700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b100d1e40 0x7f5b10158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:30.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.834+0000 7f5b1499b700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b100c71b0 0x7f5b10154610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c550 tx=0x7f5b0400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:30.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.834+0000 7f5b0d7fa700 1 -- 172.21.15.49:0/64170878 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04004c80 con 0x7f5b100c71b0 2023-07-18T03:39:30.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.834+0000 7f5b0d7fa700 1 -- 172.21.15.49:0/64170878 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b04004e00 con 0x7f5b100c71b0 2023-07-18T03:39:30.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.834+0000 7f5b16bff700 1 -- 172.21.15.49:0/64170878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b100b77e0 con 0x7f5b100c71b0 2023-07-18T03:39:30.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.834+0000 7f5b0d7fa700 1 -- 172.21.15.49:0/64170878 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04007310 con 0x7f5b100c71b0 2023-07-18T03:39:30.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.834+0000 7f5b16bff700 1 -- 172.21.15.49:0/64170878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b100b7da0 con 0x7f5b100c71b0 2023-07-18T03:39:30.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.835+0000 7f5b16bff700 1 -- 172.21.15.49:0/64170878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b10074350 con 0x7f5b100c71b0 2023-07-18T03:39:30.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.836+0000 7f5b0d7fa700 1 -- 172.21.15.49:0/64170878 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5b04007470 con 0x7f5b100c71b0 2023-07-18T03:39:30.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.837+0000 7f5b0d7fa700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5af406ace0 0x7f5af406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:30.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.837+0000 7f5b0d7fa700 1 -- 172.21.15.49:0/64170878 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5b040892c0 con 0x7f5b100c71b0 2023-07-18T03:39:30.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.838+0000 7f5b0ffff700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5af406ace0 0x7f5af406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:30.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.839+0000 7f5b0ffff700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5af406ace0 0x7f5af406d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0090b0 tx=0x7f5afc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:30.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:30.841+0000 7f5b0d7fa700 1 -- 172.21.15.49:0/64170878 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5b04054290 con 0x7f5b100c71b0 2023-07-18T03:39:31.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.049+0000 7f5b16bff700 1 -- 172.21.15.49:0/64170878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b100c6d20 con 0x7f5b100c71b0 2023-07-18T03:39:31.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.050+0000 7f5b0d7fa700 1 -- 172.21.15.49:0/64170878 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b040578b0 con 0x7f5b100c71b0 2023-07-18T03:39:31.046 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:39:31.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.052+0000 7f5b16bff700 1 -- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5af406ace0 msgr2=0x7f5af406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.052+0000 7f5b16bff700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5af406ace0 0x7f5af406d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0090b0 tx=0x7f5afc008030 comp rx=0 tx=0).stop 2023-07-18T03:39:31.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.052+0000 7f5b16bff700 1 -- 172.21.15.49:0/64170878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b100c71b0 msgr2=0x7f5b10154610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.052+0000 7f5b16bff700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b100c71b0 0x7f5b10154610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c550 tx=0x7f5b0400c910 comp rx=0 tx=0).stop 2023-07-18T03:39:31.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.052+0000 7f5b16bff700 1 -- 172.21.15.49:0/64170878 shutdown_connections 2023-07-18T03:39:31.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.052+0000 7f5b16bff700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b1015c860 0x7f5b10158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.052+0000 7f5b16bff700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5af406ace0 0x7f5af406d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.052+0000 7f5b16bff700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b100c71b0 0x7f5b10154610 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.052+0000 7f5b16bff700 1 --2- 172.21.15.49:0/64170878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b100d1e40 0x7f5b10158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.052+0000 7f5b16bff700 1 -- 172.21.15.49:0/64170878 >> 172.21.15.49:0/64170878 conn(0x7f5b10065bd0 msgr2=0x7f5b100cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:31.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.052+0000 7f5b16bff700 1 -- 172.21.15.49:0/64170878 shutdown_connections 2023-07-18T03:39:31.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.052+0000 7f5b16bff700 1 -- 172.21.15.49:0/64170878 wait complete. 2023-07-18T03:39:31.066 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:39:31.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.280+0000 7f7b6bbff700 1 Processor -- start 2023-07-18T03:39:31.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.280+0000 7f7b6bbff700 1 -- start start 2023-07-18T03:39:31.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.282+0000 7f7b6bbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640b12f0 0x7f7b640ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.282+0000 7f7b6bbff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6415ce70 con 0x7f7b640c92d0 2023-07-18T03:39:31.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.282+0000 7f7b6bbff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6415d0c0 con 0x7f7b640adea0 2023-07-18T03:39:31.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.282+0000 7f7b6bbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6415da60 con 0x7f7b640b12f0 2023-07-18T03:39:31.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.282+0000 7f7b6999b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7b640c92d0 0x7f7b640c96a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57732/0 (socket says 172.21.15.49:57732) 2023-07-18T03:39:31.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.282+0000 7f7b6999b700 1 -- 172.21.15.49:0/3194084100 learned_addr learned my addr 172.21.15.49:0/3194084100 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:31.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.282+0000 7f7b6919a700 1 --2- 172.21.15.49:0/3194084100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640b12f0 0x7f7b640ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.282+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241623604 0 0) 0x7f7b6415ce70 con 0x7f7b640c92d0 2023-07-18T03:39:31.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.282+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b500035b0 con 0x7f7b640c92d0 2023-07-18T03:39:31.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 464000662 0 0) 0x7f7b500035b0 con 0x7f7b640c92d0 2023-07-18T03:39:31.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640b12f0 msgr2=0x7f7b640ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 --2- 172.21.15.49:0/3194084100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640b12f0 0x7f7b640ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 >> v1:172.21.15.107:6789/0 conn(0x7f7b640adea0 legacy=0x7f7b640cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6415d3a0 con 0x7f7b640c92d0 2023-07-18T03:39:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f7b60001670 con 0x7f7b640c92d0 2023-07-18T03:39:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 >> v1:172.21.15.49:6789/0 conn(0x7f7b640c92d0 legacy=0x7f7b640c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 --2- 172.21.15.49:0/3194084100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b50003a30 0x7f7b50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 --2- 172.21.15.49:0/3194084100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640b12f0 0x7f7b50004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 --2- 172.21.15.49:0/3194084100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004d90 0x7f7b50005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6415da60 con 0x7f7b50004d90 2023-07-18T03:39:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5000baf0 con 0x7f7b50003a30 2023-07-18T03:39:31.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5000bcb0 con 0x7f7b640b12f0 2023-07-18T03:39:31.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b6a19c700 1 --2- 172.21.15.49:0/3194084100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004d90 0x7f7b50005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b6919a700 1 --2- 172.21.15.49:0/3194084100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640b12f0 0x7f7b50004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.283+0000 7f7b6999b700 1 --2- 172.21.15.49:0/3194084100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b50003a30 0x7f7b50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.284+0000 7f7b6a19c700 1 -- 172.21.15.49:0/3194084100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640b12f0 msgr2=0x7f7b50004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:31.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.284+0000 7f7b6a19c700 1 --2- 172.21.15.49:0/3194084100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640b12f0 0x7f7b50004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.284+0000 7f7b6a19c700 1 -- 172.21.15.49:0/3194084100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b50003a30 msgr2=0x7f7b50003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:31.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.284+0000 7f7b6a19c700 1 --2- 172.21.15.49:0/3194084100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b50003a30 0x7f7b50003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.284+0000 7f7b6a19c700 1 -- 172.21.15.49:0/3194084100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b500035b0 con 0x7f7b50004d90 2023-07-18T03:39:31.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.284+0000 7f7b6a19c700 1 --2- 172.21.15.49:0/3194084100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004d90 0x7f7b50005200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c001f90 tx=0x7f7b50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df015d6ca9373dbf server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:31.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.284+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c00cd80 con 0x7f7b50004d90 2023-07-18T03:39:31.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.284+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5c011470 con 0x7f7b50004d90 2023-07-18T03:39:31.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.284+0000 7f7b68999700 1 -- 172.21.15.49:0/3194084100 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c011820 con 0x7f7b50004d90 2023-07-18T03:39:31.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.285+0000 7f7b6bbff700 1 -- 172.21.15.49:0/3194084100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004d90 msgr2=0x7f7b50005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.285+0000 7f7b6bbff700 1 --2- 172.21.15.49:0/3194084100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004d90 0x7f7b50005200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c001f90 tx=0x7f7b50005880 comp rx=0 tx=0).stop 2023-07-18T03:39:31.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.285+0000 7f7b6a19c700 1 -- 172.21.15.49:0/3194084100 reap_dead start 2023-07-18T03:39:31.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.285+0000 7f7b6bbff700 1 -- 172.21.15.49:0/3194084100 shutdown_connections 2023-07-18T03:39:31.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.285+0000 7f7b6bbff700 1 -- 172.21.15.49:0/3194084100 >> 172.21.15.49:0/3194084100 conn(0x7f7b640b05d0 msgr2=0x7f7b640b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:31.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.285+0000 7f7b6bbff700 1 -- 172.21.15.49:0/3194084100 shutdown_connections 2023-07-18T03:39:31.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.285+0000 7f7b6bbff700 1 -- 172.21.15.49:0/3194084100 wait complete. 2023-07-18T03:39:31.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.287+0000 7f7b6bbff700 1 Processor -- start 2023-07-18T03:39:31.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.287+0000 7f7b6bbff700 1 -- start start 2023-07-18T03:39:31.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6bbff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004d90 0x7f7b64147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6bbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640c7e10 0x7f7b640c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6bbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b64157b90 0x7f7b64158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6bbff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6414ae10 con 0x7f7b50004d90 2023-07-18T03:39:31.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6bbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64158540 con 0x7f7b64157b90 2023-07-18T03:39:31.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6bbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6414bc30 con 0x7f7b640c7e10 2023-07-18T03:39:31.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6999b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004d90 0x7f7b64147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6999b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004d90 0x7f7b64147190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53622/0 (socket says 172.21.15.49:53622) 2023-07-18T03:39:31.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6999b700 1 -- 172.21.15.49:0/4267741696 learned_addr learned my addr 172.21.15.49:0/4267741696 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:31.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6919a700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640c7e10 0x7f7b640c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6a19c700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b64157b90 0x7f7b64158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6999b700 1 -- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640c7e10 msgr2=0x7f7b640c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6999b700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640c7e10 0x7f7b640c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6999b700 1 -- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b64157b90 msgr2=0x7f7b64158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6999b700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b64157b90 0x7f7b64158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.288+0000 7f7b6999b700 1 -- 172.21.15.49:0/4267741696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b641476d0 con 0x7f7b50004d90 2023-07-18T03:39:31.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.289+0000 7f7b6999b700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004d90 0x7f7b64147190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7b600073a0 tx=0x7f7b600076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:31.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.289+0000 7f7b5a7fc700 1 -- 172.21.15.49:0/4267741696 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000f6c0 con 0x7f7b50004d90 2023-07-18T03:39:31.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.289+0000 7f7b5a7fc700 1 -- 172.21.15.49:0/4267741696 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6000f820 con 0x7f7b50004d90 2023-07-18T03:39:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.289+0000 7f7b5a7fc700 1 -- 172.21.15.49:0/4267741696 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000fbd0 con 0x7f7b50004d90 2023-07-18T03:39:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.289+0000 7f7b6bbff700 1 -- 172.21.15.49:0/4267741696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6414b020 con 0x7f7b50004d90 2023-07-18T03:39:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.289+0000 7f7b6bbff700 1 -- 172.21.15.49:0/4267741696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6414b5e0 con 0x7f7b50004d90 2023-07-18T03:39:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.290+0000 7f7b6bbff700 1 -- 172.21.15.49:0/4267741696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b64074350 con 0x7f7b50004d90 2023-07-18T03:39:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.291+0000 7f7b5a7fc700 1 -- 172.21.15.49:0/4267741696 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b6000fd30 con 0x7f7b50004d90 2023-07-18T03:39:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.292+0000 7f7b5a7fc700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7b4806ace0 0x7f7b4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.292+0000 7f7b5a7fc700 1 -- 172.21.15.49:0/4267741696 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7b60088ac0 con 0x7f7b50004d90 2023-07-18T03:39:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.292+0000 7f7b6919a700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7b4806ace0 0x7f7b4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.293+0000 7f7b6919a700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7b4806ace0 0x7f7b4806d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009c60 tx=0x7f7b54009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:31.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.296+0000 7f7b5a7fc700 1 -- 172.21.15.49:0/4267741696 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7b60053a90 con 0x7f7b50004d90 2023-07-18T03:39:31.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.500+0000 7f7b6bbff700 1 -- 172.21.15.49:0/4267741696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b640b4a20 con 0x7f7b50004d90 2023-07-18T03:39:31.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.500+0000 7f7b5a7fc700 1 -- 172.21.15.49:0/4267741696 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b6001a0a0 con 0x7f7b50004d90 2023-07-18T03:39:31.496 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:39:31.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.503+0000 7f7b6bbff700 1 -- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7b4806ace0 msgr2=0x7f7b4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.503+0000 7f7b6bbff700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7b4806ace0 0x7f7b4806d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009c60 tx=0x7f7b54009210 comp rx=0 tx=0).stop 2023-07-18T03:39:31.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.503+0000 7f7b6bbff700 1 -- 172.21.15.49:0/4267741696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004d90 msgr2=0x7f7b64147190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.503+0000 7f7b6bbff700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004d90 0x7f7b64147190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7b600073a0 tx=0x7f7b600076b0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.503+0000 7f7b6bbff700 1 -- 172.21.15.49:0/4267741696 shutdown_connections 2023-07-18T03:39:31.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.503+0000 7f7b6bbff700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b64157b90 0x7f7b64158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.503+0000 7f7b6bbff700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7b4806ace0 0x7f7b4806d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.503+0000 7f7b6bbff700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b50004d90 0x7f7b64147190 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.503+0000 7f7b6bbff700 1 --2- 172.21.15.49:0/4267741696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b640c7e10 0x7f7b640c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.503+0000 7f7b6bbff700 1 -- 172.21.15.49:0/4267741696 >> 172.21.15.49:0/4267741696 conn(0x7f7b640b05d0 msgr2=0x7f7b6406f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:31.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.503+0000 7f7b6bbff700 1 -- 172.21.15.49:0/4267741696 shutdown_connections 2023-07-18T03:39:31.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.503+0000 7f7b6bbff700 1 -- 172.21.15.49:0/4267741696 wait complete. 2023-07-18T03:39:31.517 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-18T03:39:31.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.733+0000 7f1cc05ff700 1 Processor -- start 2023-07-18T03:39:31.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.733+0000 7f1cc05ff700 1 -- start start 2023-07-18T03:39:31.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.734+0000 7f1cc05ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1cb80d0f90 0x7f1cb80b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.735+0000 7f1cc05ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb80b8e30 con 0x7f1cb80b8a60 2023-07-18T03:39:31.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.735+0000 7f1cc05ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb80afaa0 con 0x7f1cb80d0b10 2023-07-18T03:39:31.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.735+0000 7f1cc05ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb80afd80 con 0x7f1cb80d0f90 2023-07-18T03:39:31.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.735+0000 7f1cbe39b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1cb80b8a60 0x7f1cb80b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57744/0 (socket says 172.21.15.49:57744) 2023-07-18T03:39:31.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.735+0000 7f1cbe39b700 1 -- 172.21.15.49:0/1838586801 learned_addr learned my addr 172.21.15.49:0/1838586801 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:31.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.735+0000 7f1cbeb9c700 1 --2- 172.21.15.49:0/1838586801 >> v2:172.21.15.49:3300/0 conn(0x7f1cb80d0f90 0x7f1cb80b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.735+0000 7f1cbd399700 1 -- 172.21.15.49:0/1838586801 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448639000 0 0) 0x7f1cb80b8e30 con 0x7f1cb80b8a60 2023-07-18T03:39:31.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.735+0000 7f1cbd399700 1 -- 172.21.15.49:0/1838586801 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ca40035b0 con 0x7f1cb80b8a60 2023-07-18T03:39:31.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.735+0000 7f1cbeb9c700 1 -- 172.21.15.49:0/1838586801 >> v1:172.21.15.107:6789/0 conn(0x7f1cb80d0b10 legacy=0x7f1cb80b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.735+0000 7f1cbeb9c700 1 -- 172.21.15.49:0/1838586801 >> v1:172.21.15.49:6789/0 conn(0x7f1cb80b8a60 legacy=0x7f1cb80b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.735+0000 7f1cbeb9c700 1 -- 172.21.15.49:0/1838586801 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb80b0060 con 0x7f1cb80d0f90 2023-07-18T03:39:31.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.736+0000 7f1cbeb9c700 1 --2- 172.21.15.49:0/1838586801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cb80d0f90 0x7f1cb80b1530 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009d90 tx=0x7f1cb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65398069af8a14d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:31.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.736+0000 7f1cbd399700 1 -- 172.21.15.49:0/1838586801 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb000e070 con 0x7f1cb80d0f90 2023-07-18T03:39:31.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.736+0000 7f1cbd399700 1 -- 172.21.15.49:0/1838586801 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb000bb20 con 0x7f1cb80d0f90 2023-07-18T03:39:31.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.736+0000 7f1cbd399700 1 -- 172.21.15.49:0/1838586801 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0012430 con 0x7f1cb80d0f90 2023-07-18T03:39:31.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.736+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1838586801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cb80d0f90 msgr2=0x7f1cb80b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.736+0000 7f1cc05ff700 1 --2- 172.21.15.49:0/1838586801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cb80d0f90 0x7f1cb80b1530 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009d90 tx=0x7f1cb000b330 comp rx=0 tx=0).stop 2023-07-18T03:39:31.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.737+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1838586801 shutdown_connections 2023-07-18T03:39:31.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.737+0000 7f1cc05ff700 1 --2- 172.21.15.49:0/1838586801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cb80d0f90 0x7f1cb80b1530 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.737+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1838586801 >> 172.21.15.49:0/1838586801 conn(0x7f1cb809c1b0 msgr2=0x7f1cb80d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:31.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.737+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1838586801 shutdown_connections 2023-07-18T03:39:31.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.737+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1838586801 wait complete. 2023-07-18T03:39:31.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.739+0000 7f1cc05ff700 1 Processor -- start 2023-07-18T03:39:31.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.739+0000 7f1cc05ff700 1 -- start start 2023-07-18T03:39:31.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.740+0000 7f1cc05ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cb80b8a60 0x7f1cb81f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cc05ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cb80d0b10 0x7f1cb80c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cc05ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb80d0f90 0x7f1cb815f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cc05ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb80c8830 con 0x7f1cb80d0b10 2023-07-18T03:39:31.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cc05ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb80c8b10 con 0x7f1cb80d0f90 2023-07-18T03:39:31.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cc05ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb80c8df0 con 0x7f1cb80b8a60 2023-07-18T03:39:31.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cbdb9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cb80d0b10 0x7f1cb80c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cbdb9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cb80d0b10 0x7f1cb80c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53638/0 (socket says 172.21.15.49:53638) 2023-07-18T03:39:31.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cbdb9a700 1 -- 172.21.15.49:0/1807068622 learned_addr learned my addr 172.21.15.49:0/1807068622 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:31.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cbe39b700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cb80b8a60 0x7f1cb81f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cbeb9c700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb80d0f90 0x7f1cb815f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cbdb9a700 1 -- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cb80b8a60 msgr2=0x7f1cb81f4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cbdb9a700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cb80b8a60 0x7f1cb81f4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cbdb9a700 1 -- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb80d0f90 msgr2=0x7f1cb815f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cbdb9a700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb80d0f90 0x7f1cb815f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cbdb9a700 1 -- 172.21.15.49:0/1807068622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb0009a70 con 0x7f1cb80d0b10 2023-07-18T03:39:31.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.741+0000 7f1cbeb9c700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb80d0f90 0x7f1cb815f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:31.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.742+0000 7f1cbdb9a700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cb80d0b10 0x7f1cb80c7d30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800c700 tx=0x7f1ca800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:31.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.742+0000 7f1caeffd700 1 -- 172.21.15.49:0/1807068622 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca80052d0 con 0x7f1cb80d0b10 2023-07-18T03:39:31.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.742+0000 7f1caeffd700 1 -- 172.21.15.49:0/1807068622 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca8005430 con 0x7f1cb80d0b10 2023-07-18T03:39:31.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.742+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1807068622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb80c90d0 con 0x7f1cb80d0b10 2023-07-18T03:39:31.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.742+0000 7f1caeffd700 1 -- 172.21.15.49:0/1807068622 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8003880 con 0x7f1cb80d0b10 2023-07-18T03:39:31.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.742+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1807068622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb80cd7a0 con 0x7f1cb80d0b10 2023-07-18T03:39:31.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.743+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1807068622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb80c95f0 con 0x7f1cb80d0b10 2023-07-18T03:39:31.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.744+0000 7f1caeffd700 1 -- 172.21.15.49:0/1807068622 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1ca80039e0 con 0x7f1cb80d0b10 2023-07-18T03:39:31.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.745+0000 7f1caeffd700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1c9c06ace0 0x7f1c9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:31.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.745+0000 7f1caeffd700 1 -- 172.21.15.49:0/1807068622 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1ca80888c0 con 0x7f1cb80d0b10 2023-07-18T03:39:31.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.746+0000 7f1cbe39b700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1c9c06ace0 0x7f1c9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:31.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.746+0000 7f1cbe39b700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1c9c06ace0 0x7f1c9c06d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4007fd0 tx=0x7f1cb4007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:31.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.750+0000 7f1caeffd700 1 -- 172.21.15.49:0/1807068622 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1cb80c95f0 con 0x7f1cb80d0b10 2023-07-18T03:39:31.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.973+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1807068622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1cb80c95f0 con 0x7f1cb80d0b10 2023-07-18T03:39:31.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.973+0000 7f1caeffd700 1 -- 172.21.15.49:0/1807068622 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1cb80c95f0 con 0x7f1cb80d0b10 2023-07-18T03:39:31.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.975+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1c9c06ace0 msgr2=0x7f1c9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.975+0000 7f1cc05ff700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1c9c06ace0 0x7f1c9c06d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4007fd0 tx=0x7f1cb4007dc0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.975+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1807068622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cb80d0b10 msgr2=0x7f1cb80c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:31.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.975+0000 7f1cc05ff700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cb80d0b10 0x7f1cb80c7d30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800c700 tx=0x7f1ca800ca10 comp rx=0 tx=0).stop 2023-07-18T03:39:31.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.976+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1807068622 shutdown_connections 2023-07-18T03:39:31.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.976+0000 7f1cc05ff700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cb80d0f90 0x7f1cb815f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.976+0000 7f1cc05ff700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cb80b8a60 0x7f1cb81f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.976+0000 7f1cc05ff700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1c9c06ace0 0x7f1c9c06d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.976+0000 7f1cc05ff700 1 --2- 172.21.15.49:0/1807068622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cb80d0b10 0x7f1cb80c7d30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:31.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.976+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1807068622 >> 172.21.15.49:0/1807068622 conn(0x7f1cb809c1b0 msgr2=0x7f1cb806b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:31.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.976+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1807068622 shutdown_connections 2023-07-18T03:39:31.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:31.976+0000 7f1cc05ff700 1 -- 172.21.15.49:0/1807068622 wait complete. 2023-07-18T03:39:31.973 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-07-18T03:39:31.990 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-07-18T03:39:31.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:31.990 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-07-18T03:39:32.086 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:39:32.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:39:32.087 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-18T03:39:32.133 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-18T03:39:32.133 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:39:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.345+0000 7f71da1ff700 1 Processor -- start 2023-07-18T03:39:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.345+0000 7f71da1ff700 1 -- start start 2023-07-18T03:39:32.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.346+0000 7f71da1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f71d40b1e00 0x7f71d40b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:32.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.346+0000 7f71da1ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d40d2ce0 con 0x7f71d40d1220 2023-07-18T03:39:32.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.346+0000 7f71da1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d40b3350 con 0x7f71d415c950 2023-07-18T03:39:32.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.346+0000 7f71da1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d40b3630 con 0x7f71d40b1e00 2023-07-18T03:39:32.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.346+0000 7f71d2ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f71d40d1220 0x7f71d40afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57750/0 (socket says 172.21.15.49:57750) 2023-07-18T03:39:32.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.346+0000 7f71d2ffd700 1 -- 172.21.15.49:0/1881568504 learned_addr learned my addr 172.21.15.49:0/1881568504 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:32.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.346+0000 7f71d3fff700 1 --2- 172.21.15.49:0/1881568504 >> v2:172.21.15.107:3300/0 conn(0x7f71d40b1e00 0x7f71d40b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:32.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.347+0000 7f71d27fc700 1 -- 172.21.15.49:0/1881568504 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4093282146 0 0) 0x7f71d40d2ce0 con 0x7f71d40d1220 2023-07-18T03:39:32.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.347+0000 7f71d27fc700 1 -- 172.21.15.49:0/1881568504 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71bc0035b0 con 0x7f71d40d1220 2023-07-18T03:39:32.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.347+0000 7f71d27fc700 1 -- 172.21.15.49:0/1881568504 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488625942 0 0) 0x7f71d40b3350 con 0x7f71d415c950 2023-07-18T03:39:32.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.347+0000 7f71d27fc700 1 -- 172.21.15.49:0/1881568504 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71d40d2ce0 con 0x7f71d415c950 2023-07-18T03:39:32.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.347+0000 7f71d3fff700 1 -- 172.21.15.49:0/1881568504 >> v1:172.21.15.107:6789/0 conn(0x7f71d415c950 legacy=0x7f71d40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:32.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.347+0000 7f71d3fff700 1 -- 172.21.15.49:0/1881568504 >> v1:172.21.15.49:6789/0 conn(0x7f71d40d1220 legacy=0x7f71d40afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:32.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.347+0000 7f71d3fff700 1 -- 172.21.15.49:0/1881568504 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d40b3910 con 0x7f71d40b1e00 2023-07-18T03:39:32.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.347+0000 7f71d3fff700 1 --2- 172.21.15.49:0/1881568504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71d40b1e00 0x7f71d40b43f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009d90 tx=0x7f71cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d19049ab696b7586 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:32.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.348+0000 7f71d27fc700 1 -- 172.21.15.49:0/1881568504 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc00e070 con 0x7f71d40b1e00 2023-07-18T03:39:32.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.348+0000 7f71d27fc700 1 -- 172.21.15.49:0/1881568504 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc00bb20 con 0x7f71d40b1e00 2023-07-18T03:39:32.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.348+0000 7f71d27fc700 1 -- 172.21.15.49:0/1881568504 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc012430 con 0x7f71d40b1e00 2023-07-18T03:39:32.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.348+0000 7f71da1ff700 1 -- 172.21.15.49:0/1881568504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71d40b1e00 msgr2=0x7f71d40b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:32.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.348+0000 7f71da1ff700 1 --2- 172.21.15.49:0/1881568504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71d40b1e00 0x7f71d40b43f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009d90 tx=0x7f71cc00b330 comp rx=0 tx=0).stop 2023-07-18T03:39:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.348+0000 7f71da1ff700 1 -- 172.21.15.49:0/1881568504 shutdown_connections 2023-07-18T03:39:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.348+0000 7f71da1ff700 1 --2- 172.21.15.49:0/1881568504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71d40b1e00 0x7f71d40b43f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.348+0000 7f71da1ff700 1 -- 172.21.15.49:0/1881568504 >> 172.21.15.49:0/1881568504 conn(0x7f71d409c1d0 msgr2=0x7f71d40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.348+0000 7f71da1ff700 1 -- 172.21.15.49:0/1881568504 shutdown_connections 2023-07-18T03:39:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.348+0000 7f71da1ff700 1 -- 172.21.15.49:0/1881568504 wait complete. 2023-07-18T03:39:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.350+0000 7f71da1ff700 1 Processor -- start 2023-07-18T03:39:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.350+0000 7f71da1ff700 1 -- start start 2023-07-18T03:39:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.351+0000 7f71da1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71d40b1e00 0x7f71d40c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.351+0000 7f71da1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d40d1220 0x7f71d40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.351+0000 7f71da1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71d415c950 0x7f71d41f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.351+0000 7f71da1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40c84d0 con 0x7f71d40d1220 2023-07-18T03:39:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.351+0000 7f71da1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40b76f0 con 0x7f71d415c950 2023-07-18T03:39:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.351+0000 7f71da1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40c8700 con 0x7f71d40b1e00 2023-07-18T03:39:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.351+0000 7f71d2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d40d1220 0x7f71d40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:32.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71d2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d40d1220 0x7f71d40b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53654/0 (socket says 172.21.15.49:53654) 2023-07-18T03:39:32.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71d2ffd700 1 -- 172.21.15.49:0/302566372 learned_addr learned my addr 172.21.15.49:0/302566372 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:32.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71d3fff700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71d415c950 0x7f71d41f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:32.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71d37fe700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71d40b1e00 0x7f71d40c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:32.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71d2ffd700 1 -- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71d40b1e00 msgr2=0x7f71d40c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:32.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71d2ffd700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71d40b1e00 0x7f71d40c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:32.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71d2ffd700 1 -- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71d415c950 msgr2=0x7f71d41f1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:32.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71d2ffd700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71d415c950 0x7f71d41f1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71d2ffd700 1 -- 172.21.15.49:0/302566372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc009a70 con 0x7f71d40d1220 2023-07-18T03:39:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71d3fff700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71d415c950 0x7f71d41f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71d37fe700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71d40b1e00 0x7f71d40c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71d2ffd700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d40d1220 0x7f71d40b71b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f71c8005910 tx=0x7f71c800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.352+0000 7f71bbfff700 1 -- 172.21.15.49:0/302566372 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8005d70 con 0x7f71d40d1220 2023-07-18T03:39:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.353+0000 7f71bbfff700 1 -- 172.21.15.49:0/302566372 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c80070d0 con 0x7f71d40d1220 2023-07-18T03:39:32.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.353+0000 7f71bbfff700 1 -- 172.21.15.49:0/302566372 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8007480 con 0x7f71d40d1220 2023-07-18T03:39:32.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.353+0000 7f71da1ff700 1 -- 172.21.15.49:0/302566372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d40c89e0 con 0x7f71d40d1220 2023-07-18T03:39:32.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.353+0000 7f71da1ff700 1 -- 172.21.15.49:0/302566372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d40c8fa0 con 0x7f71d40d1220 2023-07-18T03:39:32.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.354+0000 7f71da1ff700 1 -- 172.21.15.49:0/302566372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d404e280 con 0x7f71d40d1220 2023-07-18T03:39:32.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.355+0000 7f71bbfff700 1 -- 172.21.15.49:0/302566372 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f71c80075e0 con 0x7f71d40d1220 2023-07-18T03:39:32.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.356+0000 7f71bbfff700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f71c006ace0 0x7f71c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:32.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.356+0000 7f71bbfff700 1 -- 172.21.15.49:0/302566372 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f71c8088b80 con 0x7f71d40d1220 2023-07-18T03:39:32.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.356+0000 7f71d37fe700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f71c006ace0 0x7f71c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:32.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.357+0000 7f71d37fe700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f71c006ace0 0x7f71c006d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f71c4001230 tx=0x7f71c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:32.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.360+0000 7f71bbfff700 1 -- 172.21.15.49:0/302566372 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f71c8053b50 con 0x7f71d40d1220 2023-07-18T03:39:32.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.624+0000 7f71da1ff700 1 -- 172.21.15.49:0/302566372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71d408c5a0 con 0x7f71d40d1220 2023-07-18T03:39:32.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.625+0000 7f71bbfff700 1 -- 172.21.15.49:0/302566372 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4709 (secure 0 0 0) 0x7f71c8057170 con 0x7f71d40d1220 2023-07-18T03:39:32.621 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:32.621 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1477016053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1477016053},{"type":"v1","addr":"172.21.15.107:6835","nonce":1477016053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2998451500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2998451500},{"type":"v1","addr":"172.21.15.49:6835","nonce":2998451500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4247281109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4247281109},{"type":"v1","addr":"172.21.15.49:6833","nonce":4247281109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:38:36.025442+0000","modified":"2023-07-18T03:38:39.535929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1127901907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1127901907},{"type":"v1","addr":"172.21.15.107:6837","nonce":1127901907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:39:32.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.627+0000 7f71da1ff700 1 -- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f71c006ace0 msgr2=0x7f71c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:32.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.627+0000 7f71da1ff700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f71c006ace0 0x7f71c006d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f71c4001230 tx=0x7f71c4009040 comp rx=0 tx=0).stop 2023-07-18T03:39:32.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.627+0000 7f71da1ff700 1 -- 172.21.15.49:0/302566372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d40d1220 msgr2=0x7f71d40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:32.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.627+0000 7f71da1ff700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d40d1220 0x7f71d40b71b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f71c8005910 tx=0x7f71c800ca90 comp rx=0 tx=0).stop 2023-07-18T03:39:32.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.627+0000 7f71da1ff700 1 -- 172.21.15.49:0/302566372 shutdown_connections 2023-07-18T03:39:32.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.627+0000 7f71da1ff700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71d415c950 0x7f71d41f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:32.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.627+0000 7f71da1ff700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71d40b1e00 0x7f71d40c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:32.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.627+0000 7f71da1ff700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f71c006ace0 0x7f71c006d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:32.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.627+0000 7f71da1ff700 1 --2- 172.21.15.49:0/302566372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d40d1220 0x7f71d40b71b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:32.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.627+0000 7f71da1ff700 1 -- 172.21.15.49:0/302566372 >> 172.21.15.49:0/302566372 conn(0x7f71d409c1d0 msgr2=0x7f71d40b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:32.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.628+0000 7f71da1ff700 1 -- 172.21.15.49:0/302566372 shutdown_connections 2023-07-18T03:39:32.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.628+0000 7f71da1ff700 1 -- 172.21.15.49:0/302566372 wait complete. 2023-07-18T03:39:32.625 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2023-07-18T03:39:32.642 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:32.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.851+0000 7faf991ff700 1 Processor -- start 2023-07-18T03:39:32.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.851+0000 7faf991ff700 1 -- start start 2023-07-18T03:39:32.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.853+0000 7faf991ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7faf940b3e90 0x7faf940cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:32.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.853+0000 7faf991ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faf9415a8c0 0x7faf9415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:32.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.853+0000 7faf991ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf940ce110 con 0x7faf9415e890 2023-07-18T03:39:32.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.853+0000 7faf991ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf940cf380 con 0x7faf9415a8c0 2023-07-18T03:39:32.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.853+0000 7faf991ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9415b660 con 0x7faf940b3e90 2023-07-18T03:39:32.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.853+0000 7faf9359e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faf9415a8c0 0x7faf9415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:32.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.853+0000 7faf9359e700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faf9415a8c0 0x7faf9415ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53666/0 (socket says 172.21.15.49:53666) 2023-07-18T03:39:32.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.853+0000 7faf9359e700 1 -- 172.21.15.49:0/4258241489 learned_addr learned my addr 172.21.15.49:0/4258241489 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:32.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.853+0000 7faf9259c700 1 --2- 172.21.15.49:0/4258241489 >> v2:172.21.15.107:3300/0 conn(0x7faf940b3e90 0x7faf940cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:32.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.854+0000 7faf9359e700 1 -- 172.21.15.49:0/4258241489 >> v2:172.21.15.107:3300/0 conn(0x7faf940b3e90 msgr2=0x7faf940cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:32.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.854+0000 7faf9359e700 1 --2- 172.21.15.49:0/4258241489 >> v2:172.21.15.107:3300/0 conn(0x7faf940b3e90 0x7faf940cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:32.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.854+0000 7faf9359e700 1 -- 172.21.15.49:0/4258241489 >> v1:172.21.15.107:6789/0 conn(0x7faf9415e890 legacy=0x7faf940b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:32.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.854+0000 7faf9359e700 1 -- 172.21.15.49:0/4258241489 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf940ce360 con 0x7faf9415a8c0 2023-07-18T03:39:32.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.854+0000 7faf9359e700 1 --2- 172.21.15.49:0/4258241489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf9415a8c0 0x7faf9415ad20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faf9406b730 tx=0x7faf8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e135e886af95c386 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:32.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.854+0000 7faf91d9b700 1 -- 172.21.15.49:0/4258241489 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00e070 con 0x7faf9415a8c0 2023-07-18T03:39:32.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.854+0000 7faf91d9b700 1 -- 172.21.15.49:0/4258241489 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c00b9c0 con 0x7faf9415a8c0 2023-07-18T03:39:32.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.854+0000 7faf91d9b700 1 -- 172.21.15.49:0/4258241489 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00bd70 con 0x7faf9415a8c0 2023-07-18T03:39:32.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.855+0000 7faf991ff700 1 -- 172.21.15.49:0/4258241489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf9415a8c0 msgr2=0x7faf9415ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:32.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.855+0000 7faf991ff700 1 --2- 172.21.15.49:0/4258241489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf9415a8c0 0x7faf9415ad20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faf9406b730 tx=0x7faf8c00b330 comp rx=0 tx=0).stop 2023-07-18T03:39:32.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.855+0000 7faf991ff700 1 -- 172.21.15.49:0/4258241489 shutdown_connections 2023-07-18T03:39:32.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.855+0000 7faf991ff700 1 --2- 172.21.15.49:0/4258241489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf9415a8c0 0x7faf9415ad20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:32.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.855+0000 7faf991ff700 1 --2- 172.21.15.49:0/4258241489 >> v2:172.21.15.107:3300/0 conn(0x7faf940b3e90 0x7faf940cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:32.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.855+0000 7faf991ff700 1 -- 172.21.15.49:0/4258241489 >> 172.21.15.49:0/4258241489 conn(0x7faf94069f30 msgr2=0x7faf9406a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:32.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.855+0000 7faf991ff700 1 -- 172.21.15.49:0/4258241489 shutdown_connections 2023-07-18T03:39:32.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.855+0000 7faf991ff700 1 -- 172.21.15.49:0/4258241489 wait complete. 2023-07-18T03:39:32.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.857+0000 7faf991ff700 1 Processor -- start 2023-07-18T03:39:32.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.857+0000 7faf991ff700 1 -- start start 2023-07-18T03:39:32.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf991ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faf940b3e90 0x7faf940afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:32.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf991ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf9415a8c0 0x7faf940b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:32.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf991ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf9415e890 0x7faf941518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:32.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf991ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf940b00a0 con 0x7faf9415e890 2023-07-18T03:39:32.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf991ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9414bd60 con 0x7faf9415a8c0 2023-07-18T03:39:32.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf991ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf940aeac0 con 0x7faf940b3e90 2023-07-18T03:39:32.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf9359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf9415e890 0x7faf941518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:32.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf9359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf9415e890 0x7faf941518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53668/0 (socket says 172.21.15.49:53668) 2023-07-18T03:39:32.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf9359e700 1 -- 172.21.15.49:0/2008337708 learned_addr learned my addr 172.21.15.49:0/2008337708 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:32.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf92d9d700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faf940b3e90 0x7faf940afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:32.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf9259c700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf9415a8c0 0x7faf940b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:32.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf9359e700 1 -- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faf940b3e90 msgr2=0x7faf940afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:32.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf9359e700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faf940b3e90 0x7faf940afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:32.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf9359e700 1 -- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf9415a8c0 msgr2=0x7faf940b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:32.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf9359e700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf9415a8c0 0x7faf940b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:32.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf9359e700 1 -- 172.21.15.49:0/2008337708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c009b30 con 0x7faf9415e890 2023-07-18T03:39:32.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.858+0000 7faf9259c700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf9415a8c0 0x7faf940b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:32.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.859+0000 7faf9359e700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf9415e890 0x7faf941518f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf9406ac20 tx=0x7faf8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:32.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.859+0000 7faf7b7fe700 1 -- 172.21.15.49:0/2008337708 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c0099b0 con 0x7faf9415e890 2023-07-18T03:39:32.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.859+0000 7faf7b7fe700 1 -- 172.21.15.49:0/2008337708 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c0092e0 con 0x7faf9415e890 2023-07-18T03:39:32.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.859+0000 7faf7b7fe700 1 -- 172.21.15.49:0/2008337708 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c01c640 con 0x7faf9415e890 2023-07-18T03:39:32.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.859+0000 7faf991ff700 1 -- 172.21.15.49:0/2008337708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf940aeda0 con 0x7faf9415e890 2023-07-18T03:39:32.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.859+0000 7faf991ff700 1 -- 172.21.15.49:0/2008337708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf940af360 con 0x7faf9415e890 2023-07-18T03:39:32.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.860+0000 7faf991ff700 1 -- 172.21.15.49:0/2008337708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9404e280 con 0x7faf9415e890 2023-07-18T03:39:32.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.861+0000 7faf7b7fe700 1 -- 172.21.15.49:0/2008337708 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faf8c01c7a0 con 0x7faf9415e890 2023-07-18T03:39:32.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.862+0000 7faf7b7fe700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7faf8006ace0 0x7faf8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:32.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.862+0000 7faf7b7fe700 1 -- 172.21.15.49:0/2008337708 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7faf8c08b230 con 0x7faf9415e890 2023-07-18T03:39:32.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.863+0000 7faf92d9d700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7faf8006ace0 0x7faf8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:32.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.863+0000 7faf92d9d700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7faf8006ace0 0x7faf8006d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf84000f60 tx=0x7faf84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:32.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:32.867+0000 7faf7b7fe700 1 -- 172.21.15.49:0/2008337708 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7faf8c056290 con 0x7faf9415e890 2023-07-18T03:39:33.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.078+0000 7faf991ff700 1 -- 172.21.15.49:0/2008337708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf9415a0e0 con 0x7faf9415e890 2023-07-18T03:39:33.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.079+0000 7faf7b7fe700 1 -- 172.21.15.49:0/2008337708 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18280 (secure 0 0 0) 0x7faf8c0598b0 con 0x7faf9415e890 2023-07-18T03:39:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:33.075 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:23.937174+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:33.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.082+0000 7faf991ff700 1 -- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7faf8006ace0 msgr2=0x7faf8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.082+0000 7faf991ff700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7faf8006ace0 0x7faf8006d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf84000f60 tx=0x7faf84009040 comp rx=0 tx=0).stop 2023-07-18T03:39:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.082+0000 7faf991ff700 1 -- 172.21.15.49:0/2008337708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf9415e890 msgr2=0x7faf941518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.082+0000 7faf991ff700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf9415e890 0x7faf941518f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf9406ac20 tx=0x7faf8c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.082+0000 7faf991ff700 1 -- 172.21.15.49:0/2008337708 shutdown_connections 2023-07-18T03:39:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.082+0000 7faf991ff700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7faf8006ace0 0x7faf8006d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.082+0000 7faf991ff700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf9415e890 0x7faf941518f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.082+0000 7faf991ff700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faf940b3e90 0x7faf940afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.082+0000 7faf991ff700 1 --2- 172.21.15.49:0/2008337708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf9415a8c0 0x7faf940b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.082+0000 7faf991ff700 1 -- 172.21.15.49:0/2008337708 >> 172.21.15.49:0/2008337708 conn(0x7faf94069f30 msgr2=0x7faf9415ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:33.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.082+0000 7faf991ff700 1 -- 172.21.15.49:0/2008337708 shutdown_connections 2023-07-18T03:39:33.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.082+0000 7faf991ff700 1 -- 172.21.15.49:0/2008337708 wait complete. 2023-07-18T03:39:33.096 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:38:33.773236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:38:35.023692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:39:33.097 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:39:33.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.309+0000 7f9bf8dff700 1 Processor -- start 2023-07-18T03:39:33.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.309+0000 7f9bf8dff700 1 -- start start 2023-07-18T03:39:33.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.310+0000 7f9bf8dff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9bf40b4d00 0x7f9bf40d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:33.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.310+0000 7f9bf8dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf40d02a0 con 0x7f9bf40b8660 2023-07-18T03:39:33.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.310+0000 7f9bf8dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf40cfce0 con 0x7f9bf40b4860 2023-07-18T03:39:33.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.310+0000 7f9bf8dff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf40cffc0 con 0x7f9bf40b4d00 2023-07-18T03:39:33.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.310+0000 7f9bf259c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9bf40b8660 0x7f9bf40cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57754/0 (socket says 172.21.15.49:57754) 2023-07-18T03:39:33.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.310+0000 7f9bf259c700 1 -- 172.21.15.49:0/571001042 learned_addr learned my addr 172.21.15.49:0/571001042 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:33.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.310+0000 7f9bf2d9d700 1 --2- 172.21.15.49:0/571001042 >> v2:172.21.15.49:3300/0 conn(0x7f9bf40b4d00 0x7f9bf40d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:33.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.311+0000 7f9bf2d9d700 1 -- 172.21.15.49:0/571001042 >> v1:172.21.15.107:6789/0 conn(0x7f9bf40b4860 legacy=0x7f9bf40d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:33.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.311+0000 7f9bf159a700 1 -- 172.21.15.49:0/571001042 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888449686 0 0) 0x7f9bf40d02a0 con 0x7f9bf40b8660 2023-07-18T03:39:33.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.311+0000 7f9bf2d9d700 1 -- 172.21.15.49:0/571001042 >> v1:172.21.15.49:6789/0 conn(0x7f9bf40b8660 legacy=0x7f9bf40cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.311+0000 7f9bf2d9d700 1 -- 172.21.15.49:0/571001042 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf40d0ac0 con 0x7f9bf40b4d00 2023-07-18T03:39:33.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.311+0000 7f9bf2d9d700 1 --2- 172.21.15.49:0/571001042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf40b4d00 0x7f9bf40d19d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009d90 tx=0x7f9bec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0a8ebdce1b9b02 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:33.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.312+0000 7f9bf159a700 1 -- 172.21.15.49:0/571001042 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00e070 con 0x7f9bf40b4d00 2023-07-18T03:39:33.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.312+0000 7f9bf159a700 1 -- 172.21.15.49:0/571001042 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec00bbd0 con 0x7f9bf40b4d00 2023-07-18T03:39:33.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.312+0000 7f9bf159a700 1 -- 172.21.15.49:0/571001042 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec012430 con 0x7f9bf40b4d00 2023-07-18T03:39:33.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.312+0000 7f9bf8dff700 1 -- 172.21.15.49:0/571001042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf40b4d00 msgr2=0x7f9bf40d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.312+0000 7f9bf8dff700 1 --2- 172.21.15.49:0/571001042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf40b4d00 0x7f9bf40d19d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009d90 tx=0x7f9bec00b330 comp rx=0 tx=0).stop 2023-07-18T03:39:33.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.312+0000 7f9bf8dff700 1 -- 172.21.15.49:0/571001042 shutdown_connections 2023-07-18T03:39:33.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.312+0000 7f9bf8dff700 1 --2- 172.21.15.49:0/571001042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf40b4d00 0x7f9bf40d19d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.312+0000 7f9bf8dff700 1 -- 172.21.15.49:0/571001042 >> 172.21.15.49:0/571001042 conn(0x7f9bf4066860 msgr2=0x7f9bf406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:33.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.312+0000 7f9bf8dff700 1 -- 172.21.15.49:0/571001042 shutdown_connections 2023-07-18T03:39:33.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.312+0000 7f9bf8dff700 1 -- 172.21.15.49:0/571001042 wait complete. 2023-07-18T03:39:33.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.314+0000 7f9bf8dff700 1 Processor -- start 2023-07-18T03:39:33.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.314+0000 7f9bf8dff700 1 -- start start 2023-07-18T03:39:33.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.315+0000 7f9bf8dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9bf40b4860 0x7f9bf41e6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:33.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.315+0000 7f9bf8dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf40b4d00 0x7f9bf41e7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:33.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.315+0000 7f9bf8dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf40b8660 0x7f9bf414ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:33.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.315+0000 7f9bf8dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf41e6410 con 0x7f9bf40b4d00 2023-07-18T03:39:33.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.315+0000 7f9bf8dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf41e5b00 con 0x7f9bf40b8660 2023-07-18T03:39:33.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.315+0000 7f9bf8dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf40b44f0 con 0x7f9bf40b4860 2023-07-18T03:39:33.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9bf1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf40b4d00 0x7f9bf41e7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:33.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9bf1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf40b4d00 0x7f9bf41e7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53686/0 (socket says 172.21.15.49:53686) 2023-07-18T03:39:33.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9bf1d9b700 1 -- 172.21.15.49:0/1724212598 learned_addr learned my addr 172.21.15.49:0/1724212598 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:33.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9bf259c700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9bf40b4860 0x7f9bf41e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:33.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9bf2d9d700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf40b8660 0x7f9bf414ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:33.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9bf1d9b700 1 -- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9bf40b4860 msgr2=0x7f9bf41e6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9bf1d9b700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9bf40b4860 0x7f9bf41e6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9bf1d9b700 1 -- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf40b8660 msgr2=0x7f9bf414ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9bf1d9b700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf40b8660 0x7f9bf414ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9bf1d9b700 1 -- 172.21.15.49:0/1724212598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec009a70 con 0x7f9bf40b4d00 2023-07-18T03:39:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9bf259c700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9bf40b4860 0x7f9bf41e6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9bf1d9b700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf40b4d00 0x7f9bf41e7500 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9be800c4e0 tx=0x7f9be800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.316+0000 7f9be2ffd700 1 -- 172.21.15.49:0/1724212598 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80052d0 con 0x7f9bf40b4d00 2023-07-18T03:39:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.317+0000 7f9be2ffd700 1 -- 172.21.15.49:0/1724212598 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8005430 con 0x7f9bf40b4d00 2023-07-18T03:39:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.317+0000 7f9be2ffd700 1 -- 172.21.15.49:0/1724212598 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80038c0 con 0x7f9bf40b4d00 2023-07-18T03:39:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.317+0000 7f9bf8dff700 1 -- 172.21.15.49:0/1724212598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf41e5d50 con 0x7f9bf40b4d00 2023-07-18T03:39:33.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.317+0000 7f9bf8dff700 1 -- 172.21.15.49:0/1724212598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf41e7a40 con 0x7f9bf40b4d00 2023-07-18T03:39:33.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.318+0000 7f9bf8dff700 1 -- 172.21.15.49:0/1724212598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf41e7bd0 con 0x7f9bf40b4d00 2023-07-18T03:39:33.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.319+0000 7f9be2ffd700 1 -- 172.21.15.49:0/1724212598 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9be8003a20 con 0x7f9bf40b4d00 2023-07-18T03:39:33.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.320+0000 7f9be2ffd700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9bd406ace0 0x7f9bd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:33.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.320+0000 7f9be2ffd700 1 -- 172.21.15.49:0/1724212598 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9be80888b0 con 0x7f9bf40b4d00 2023-07-18T03:39:33.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.321+0000 7f9bf259c700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9bd406ace0 0x7f9bd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:33.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.321+0000 7f9bf259c700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9bd406ace0 0x7f9bd406d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9be4000f60 tx=0x7f9be4007e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:33.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.325+0000 7f9be2ffd700 1 -- 172.21.15.49:0/1724212598 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9be8053910 con 0x7f9bf40b4d00 2023-07-18T03:39:33.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.531+0000 7f9bf8dff700 1 -- 172.21.15.49:0/1724212598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9bf4074350 con 0x7f9bf40b4d00 2023-07-18T03:39:33.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.531+0000 7f9be2ffd700 1 -- 172.21.15.49:0/1724212598 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9be8056f30 con 0x7f9bf40b4d00 2023-07-18T03:39:33.528 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:39:33.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.534+0000 7f9bf8dff700 1 -- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9bd406ace0 msgr2=0x7f9bd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.534+0000 7f9bf8dff700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9bd406ace0 0x7f9bd406d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9be4000f60 tx=0x7f9be4007e50 comp rx=0 tx=0).stop 2023-07-18T03:39:33.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.534+0000 7f9bf8dff700 1 -- 172.21.15.49:0/1724212598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf40b4d00 msgr2=0x7f9bf41e7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.534+0000 7f9bf8dff700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf40b4d00 0x7f9bf41e7500 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9be800c4e0 tx=0x7f9be800c7f0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.534+0000 7f9bf8dff700 1 -- 172.21.15.49:0/1724212598 shutdown_connections 2023-07-18T03:39:33.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.534+0000 7f9bf8dff700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf40b8660 0x7f9bf414ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.534+0000 7f9bf8dff700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9bf40b4860 0x7f9bf41e6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.534+0000 7f9bf8dff700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9bd406ace0 0x7f9bd406d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.534+0000 7f9bf8dff700 1 --2- 172.21.15.49:0/1724212598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf40b4d00 0x7f9bf41e7500 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.534+0000 7f9bf8dff700 1 -- 172.21.15.49:0/1724212598 >> 172.21.15.49:0/1724212598 conn(0x7f9bf4066860 msgr2=0x7f9bf40cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:33.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.534+0000 7f9bf8dff700 1 -- 172.21.15.49:0/1724212598 shutdown_connections 2023-07-18T03:39:33.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.534+0000 7f9bf8dff700 1 -- 172.21.15.49:0/1724212598 wait complete. 2023-07-18T03:39:33.548 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:39:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.761+0000 7f0e811ff700 1 Processor -- start 2023-07-18T03:39:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.761+0000 7f0e811ff700 1 -- start start 2023-07-18T03:39:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.762+0000 7f0e811ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e7c0b8a60 0x7f0e7c0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.762+0000 7f0e811ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c0b8e30 con 0x7f0e7c0b4530 2023-07-18T03:39:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.762+0000 7f0e811ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c0afaa0 con 0x7f0e7c0d1220 2023-07-18T03:39:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.762+0000 7f0e811ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7c0afd80 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.762+0000 7f0e7ad9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e7c0b8a60 0x7f0e7c0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.762+0000 7f0e7ad9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e7c0b8a60 0x7f0e7c0b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53696/0 (socket says 172.21.15.49:53696) 2023-07-18T03:39:33.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.762+0000 7f0e7ad9d700 1 -- 172.21.15.49:0/1797529111 learned_addr learned my addr 172.21.15.49:0/1797529111 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:33.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.762+0000 7f0e7a59c700 1 --1- 172.21.15.49:0/1797529111 >> v1:172.21.15.49:6789/0 conn(0x7f0e7c0b4530 0x7f0e7c0d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57758/0 (socket says 172.21.15.49:57758) 2023-07-18T03:39:33.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.763+0000 7f0e7ad9d700 1 -- 172.21.15.49:0/1797529111 >> v1:172.21.15.107:6789/0 conn(0x7f0e7c0d1220 legacy=0x7f0e7c0b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:33.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.763+0000 7f0e79d9b700 1 -- 172.21.15.49:0/1797529111 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003976347 0 0) 0x7f0e7c0b8e30 con 0x7f0e7c0b4530 2023-07-18T03:39:33.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.763+0000 7f0e7ad9d700 1 -- 172.21.15.49:0/1797529111 >> v1:172.21.15.49:6789/0 conn(0x7f0e7c0b4530 legacy=0x7f0e7c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.763+0000 7f0e7ad9d700 1 -- 172.21.15.49:0/1797529111 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c0b0060 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.763+0000 7f0e7ad9d700 1 --2- 172.21.15.49:0/1797529111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0b8a60 0x7f0e7c0b3ff0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0098f0 tx=0x7f0e6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108f2de80de14fb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:33.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.763+0000 7f0e79d9b700 1 -- 172.21.15.49:0/1797529111 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00b8a0 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.763+0000 7f0e79d9b700 1 -- 172.21.15.49:0/1797529111 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c00ba00 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.763+0000 7f0e79d9b700 1 -- 172.21.15.49:0/1797529111 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00bd90 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.764+0000 7f0e811ff700 1 -- 172.21.15.49:0/1797529111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0b8a60 msgr2=0x7f0e7c0b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.764+0000 7f0e811ff700 1 --2- 172.21.15.49:0/1797529111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0b8a60 0x7f0e7c0b3ff0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0098f0 tx=0x7f0e6c009c00 comp rx=0 tx=0).stop 2023-07-18T03:39:33.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.764+0000 7f0e811ff700 1 -- 172.21.15.49:0/1797529111 shutdown_connections 2023-07-18T03:39:33.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.764+0000 7f0e811ff700 1 --2- 172.21.15.49:0/1797529111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0b8a60 0x7f0e7c0b3ff0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.764+0000 7f0e811ff700 1 -- 172.21.15.49:0/1797529111 >> 172.21.15.49:0/1797529111 conn(0x7f0e7c09c1b0 msgr2=0x7f0e7c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:33.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.764+0000 7f0e811ff700 1 -- 172.21.15.49:0/1797529111 shutdown_connections 2023-07-18T03:39:33.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.764+0000 7f0e811ff700 1 -- 172.21.15.49:0/1797529111 wait complete. 2023-07-18T03:39:33.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.766+0000 7f0e811ff700 1 Processor -- start 2023-07-18T03:39:33.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.766+0000 7f0e811ff700 1 -- start start 2023-07-18T03:39:33.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e811ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e7c0b4530 0x7f0e7c149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:33.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e811ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0b8a60 0x7f0e7c15f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:33.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e811ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e7c0d1220 0x7f0e7c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:33.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e811ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0c9610 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e811ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0ca130 con 0x7f0e7c0d1220 2023-07-18T03:39:33.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e811ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0ca410 con 0x7f0e7c0b4530 2023-07-18T03:39:33.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e7a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0b8a60 0x7f0e7c15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:33.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e7a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0b8a60 0x7f0e7c15f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53704/0 (socket says 172.21.15.49:53704) 2023-07-18T03:39:33.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e7a59c700 1 -- 172.21.15.49:0/1060066952 learned_addr learned my addr 172.21.15.49:0/1060066952 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:33.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e7ad9d700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e7c0b4530 0x7f0e7c149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:33.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e7b59e700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e7c0d1220 0x7f0e7c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:33.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e7a59c700 1 -- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e7c0b4530 msgr2=0x7f0e7c149320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.767+0000 7f0e7a59c700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e7c0b4530 0x7f0e7c149320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.768+0000 7f0e7a59c700 1 -- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e7c0d1220 msgr2=0x7f0e7c15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.768+0000 7f0e7a59c700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e7c0d1220 0x7f0e7c15fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.768+0000 7f0e7a59c700 1 -- 172.21.15.49:0/1060066952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c009550 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.768+0000 7f0e7ad9d700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e7c0b4530 0x7f0e7c149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:33.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.768+0000 7f0e7a59c700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0b8a60 0x7f0e7c15f570 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000c520 tx=0x7f0e7000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:33.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.768+0000 7f0e637fe700 1 -- 172.21.15.49:0/1060066952 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70004c80 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.768+0000 7f0e637fe700 1 -- 172.21.15.49:0/1060066952 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e70004e00 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.768+0000 7f0e811ff700 1 -- 172.21.15.49:0/1060066952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c0ca6f0 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.768+0000 7f0e637fe700 1 -- 172.21.15.49:0/1060066952 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70007310 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.768+0000 7f0e811ff700 1 -- 172.21.15.49:0/1060066952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c0cacb0 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.769+0000 7f0e811ff700 1 -- 172.21.15.49:0/1060066952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c074350 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.770+0000 7f0e637fe700 1 -- 172.21.15.49:0/1060066952 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e70007470 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.771+0000 7f0e637fe700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0e6806ace0 0x7f0e6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.771+0000 7f0e637fe700 1 -- 172.21.15.49:0/1060066952 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0e70089280 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.772+0000 7f0e7ad9d700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0e6806ace0 0x7f0e6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.772+0000 7f0e7ad9d700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0e6806ace0 0x7f0e6806d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009130 tx=0x7f0e6c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.776+0000 7f0e637fe700 1 -- 172.21.15.49:0/1060066952 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0e700542f0 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.984+0000 7f0e811ff700 1 -- 172.21.15.49:0/1060066952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e7c0d1000 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.985+0000 7f0e637fe700 1 -- 172.21.15.49:0/1060066952 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e70057910 con 0x7f0e7c0b8a60 2023-07-18T03:39:33.981 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:39:33.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.987+0000 7f0e811ff700 1 -- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0e6806ace0 msgr2=0x7f0e6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.987+0000 7f0e811ff700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0e6806ace0 0x7f0e6806d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009130 tx=0x7f0e6c008030 comp rx=0 tx=0).stop 2023-07-18T03:39:33.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.987+0000 7f0e811ff700 1 -- 172.21.15.49:0/1060066952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0b8a60 msgr2=0x7f0e7c15f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:33.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.987+0000 7f0e811ff700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0b8a60 0x7f0e7c15f570 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000c520 tx=0x7f0e7000c8e0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.987+0000 7f0e811ff700 1 -- 172.21.15.49:0/1060066952 shutdown_connections 2023-07-18T03:39:33.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.987+0000 7f0e811ff700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e7c0d1220 0x7f0e7c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.987+0000 7f0e811ff700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e7c0b4530 0x7f0e7c149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.987+0000 7f0e811ff700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0e6806ace0 0x7f0e6806d190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.987+0000 7f0e811ff700 1 --2- 172.21.15.49:0/1060066952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e7c0b8a60 0x7f0e7c15f570 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:33.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.987+0000 7f0e811ff700 1 -- 172.21.15.49:0/1060066952 >> 172.21.15.49:0/1060066952 conn(0x7f0e7c09c1b0 msgr2=0x7f0e7c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:33.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.987+0000 7f0e811ff700 1 -- 172.21.15.49:0/1060066952 shutdown_connections 2023-07-18T03:39:33.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:33.987+0000 7f0e811ff700 1 -- 172.21.15.49:0/1060066952 wait complete. 2023-07-18T03:39:34.001 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:39:34.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.214+0000 7f3d6c7ff700 1 Processor -- start 2023-07-18T03:39:34.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.214+0000 7f3d6c7ff700 1 -- start start 2023-07-18T03:39:34.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.216+0000 7f3d6c7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3d640d0010 0x7f3d640b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.216+0000 7f3d6c7ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6415cd60 con 0x7f3d640d0480 2023-07-18T03:39:34.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.216+0000 7f3d6c7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d640cf340 con 0x7f3d640b4600 2023-07-18T03:39:34.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.216+0000 7f3d6c7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d640d1550 con 0x7f3d640d0010 2023-07-18T03:39:34.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.216+0000 7f3d6ad9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3d640d0480 0x7f3d640b6430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57762/0 (socket says 172.21.15.49:57762) 2023-07-18T03:39:34.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.216+0000 7f3d6ad9c700 1 -- 172.21.15.49:0/1617899836 learned_addr learned my addr 172.21.15.49:0/1617899836 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.216+0000 7f3d69d9a700 1 --2- 172.21.15.49:0/1617899836 >> v2:172.21.15.107:3300/0 conn(0x7f3d640d0010 0x7f3d640b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.216+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013207635 0 0) 0x7f3d6415cd60 con 0x7f3d640d0480 2023-07-18T03:39:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.216+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d500035b0 con 0x7f3d640d0480 2023-07-18T03:39:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051343940 0 0) 0x7f3d640cf340 con 0x7f3d640b4600 2023-07-18T03:39:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d6415cd60 con 0x7f3d640b4600 2023-07-18T03:39:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2778120784 0 0) 0x7f3d500035b0 con 0x7f3d640d0480 2023-07-18T03:39:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 >> v2:172.21.15.107:3300/0 conn(0x7f3d640d0010 msgr2=0x7f3d640b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 --2- 172.21.15.49:0/1617899836 >> v2:172.21.15.107:3300/0 conn(0x7f3d640d0010 0x7f3d640b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 >> v1:172.21.15.107:6789/0 conn(0x7f3d640b4600 legacy=0x7f3d640cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d640d0e90 con 0x7f3d640d0480 2023-07-18T03:39:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f3d5c0019f0 con 0x7f3d640d0480 2023-07-18T03:39:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 >> v1:172.21.15.49:6789/0 conn(0x7f3d640d0480 legacy=0x7f3d640b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 --2- 172.21.15.49:0/1617899836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 0x7f3d50003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 --2- 172.21.15.49:0/1617899836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d50004850 0x7f3d50004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 --2- 172.21.15.49:0/1617899836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d50005250 0x7f3d5000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640d1550 con 0x7f3d50003be0 2023-07-18T03:39:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5000c160 con 0x7f3d50004850 2023-07-18T03:39:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5000c320 con 0x7f3d50005250 2023-07-18T03:39:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d6a59b700 1 --2- 172.21.15.49:0/1617899836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 0x7f3d50003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d69d9a700 1 --2- 172.21.15.49:0/1617899836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d50004850 0x7f3d50004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d6ad9c700 1 --2- 172.21.15.49:0/1617899836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d50005250 0x7f3d5000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d6a59b700 1 -- 172.21.15.49:0/1617899836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d50005250 msgr2=0x7f3d5000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.217+0000 7f3d6a59b700 1 --2- 172.21.15.49:0/1617899836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d50005250 0x7f3d5000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.218+0000 7f3d6a59b700 1 -- 172.21.15.49:0/1617899836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d50004850 msgr2=0x7f3d50004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:34.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.218+0000 7f3d6a59b700 1 --2- 172.21.15.49:0/1617899836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d50004850 0x7f3d50004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.218+0000 7f3d6a59b700 1 -- 172.21.15.49:0/1617899836 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d500035b0 con 0x7f3d50003be0 2023-07-18T03:39:34.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.218+0000 7f3d6ad9c700 1 -- 172.21.15.49:0/1617899836 reap_dead start 2023-07-18T03:39:34.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.218+0000 7f3d6a59b700 1 --2- 172.21.15.49:0/1617899836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 0x7f3d50003fd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3d60007fd0 tx=0x7f3d50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b6c73058c734b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:34.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.218+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60001430 con 0x7f3d50003be0 2023-07-18T03:39:34.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.218+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d600052f0 con 0x7f3d50003be0 2023-07-18T03:39:34.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.218+0000 7f3d69599700 1 -- 172.21.15.49:0/1617899836 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60001d40 con 0x7f3d50003be0 2023-07-18T03:39:34.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.219+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/1617899836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 msgr2=0x7f3d50003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.219+0000 7f3d6c7ff700 1 --2- 172.21.15.49:0/1617899836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 0x7f3d50003fd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3d60007fd0 tx=0x7f3d50005880 comp rx=0 tx=0).stop 2023-07-18T03:39:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.219+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/1617899836 shutdown_connections 2023-07-18T03:39:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.219+0000 7f3d6c7ff700 1 --2- 172.21.15.49:0/1617899836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 0x7f3d50003fd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.219+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/1617899836 >> 172.21.15.49:0/1617899836 conn(0x7f3d6409c200 msgr2=0x7f3d640d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.219+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/1617899836 shutdown_connections 2023-07-18T03:39:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.219+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/1617899836 wait complete. 2023-07-18T03:39:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.221+0000 7f3d6c7ff700 1 Processor -- start 2023-07-18T03:39:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.221+0000 7f3d6c7ff700 1 -- start start 2023-07-18T03:39:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6c7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 0x7f3d64158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6c7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6415afb0 0x7f3d640b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6c7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d640b2a00 0x7f3d640b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6c7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640b33b0 con 0x7f3d50003be0 2023-07-18T03:39:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6c7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640b0b70 con 0x7f3d6415afb0 2023-07-18T03:39:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6c7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640b0e50 con 0x7f3d640b2a00 2023-07-18T03:39:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6a59b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 0x7f3d64158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6a59b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 0x7f3d64158290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53726/0 (socket says 172.21.15.49:53726) 2023-07-18T03:39:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6a59b700 1 -- 172.21.15.49:0/701023457 learned_addr learned my addr 172.21.15.49:0/701023457 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6a59b700 1 -- 172.21.15.49:0/701023457 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d640b2a00 msgr2=0x7f3d640b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6a59b700 1 --2- 172.21.15.49:0/701023457 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d640b2a00 0x7f3d640b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6a59b700 1 -- 172.21.15.49:0/701023457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6415afb0 msgr2=0x7f3d640b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6a59b700 1 --2- 172.21.15.49:0/701023457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6415afb0 0x7f3d640b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.222+0000 7f3d6a59b700 1 -- 172.21.15.49:0/701023457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d640b1130 con 0x7f3d50003be0 2023-07-18T03:39:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.223+0000 7f3d6a59b700 1 --2- 172.21.15.49:0/701023457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 0x7f3d64158290 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3d60015040 tx=0x7f3d600012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.223+0000 7f3d5affd700 1 -- 172.21.15.49:0/701023457 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60006a70 con 0x7f3d50003be0 2023-07-18T03:39:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.223+0000 7f3d5affd700 1 -- 172.21.15.49:0/701023457 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d60006c10 con 0x7f3d50003be0 2023-07-18T03:39:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.223+0000 7f3d5affd700 1 -- 172.21.15.49:0/701023457 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60007040 con 0x7f3d50003be0 2023-07-18T03:39:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.223+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/701023457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d640b1410 con 0x7f3d50003be0 2023-07-18T03:39:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.223+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/701023457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d641587d0 con 0x7f3d50003be0 2023-07-18T03:39:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.224+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/701023457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d640b1a60 con 0x7f3d50003be0 2023-07-18T03:39:34.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.225+0000 7f3d5affd700 1 -- 172.21.15.49:0/701023457 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3d600071a0 con 0x7f3d50003be0 2023-07-18T03:39:34.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.226+0000 7f3d5affd700 1 --2- 172.21.15.49:0/701023457 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d4806ace0 0x7f3d4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.226+0000 7f3d5affd700 1 -- 172.21.15.49:0/701023457 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3d6008a0c0 con 0x7f3d50003be0 2023-07-18T03:39:34.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.226+0000 7f3d69d9a700 1 --2- 172.21.15.49:0/701023457 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d4806ace0 0x7f3d4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.227+0000 7f3d69d9a700 1 --2- 172.21.15.49:0/701023457 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d4806ace0 0x7f3d4806d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3d54006f10 tx=0x7f3d54006e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:34.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.230+0000 7f3d5affd700 1 -- 172.21.15.49:0/701023457 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3d60055240 con 0x7f3d50003be0 2023-07-18T03:39:34.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.435+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/701023457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d64074350 con 0x7f3d50003be0 2023-07-18T03:39:34.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.436+0000 7f3d5affd700 1 -- 172.21.15.49:0/701023457 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3d60021020 con 0x7f3d50003be0 2023-07-18T03:39:34.432 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:39:34.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.438+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/701023457 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d4806ace0 msgr2=0x7f3d4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:34.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.438+0000 7f3d6c7ff700 1 --2- 172.21.15.49:0/701023457 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d4806ace0 0x7f3d4806d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3d54006f10 tx=0x7f3d54006e80 comp rx=0 tx=0).stop 2023-07-18T03:39:34.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.438+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/701023457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 msgr2=0x7f3d64158290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:34.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.438+0000 7f3d6c7ff700 1 --2- 172.21.15.49:0/701023457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 0x7f3d64158290 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3d60015040 tx=0x7f3d600012d0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.438+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/701023457 shutdown_connections 2023-07-18T03:39:34.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.438+0000 7f3d6c7ff700 1 --2- 172.21.15.49:0/701023457 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d640b2a00 0x7f3d640b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.438+0000 7f3d6c7ff700 1 --2- 172.21.15.49:0/701023457 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d4806ace0 0x7f3d4806d190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.438+0000 7f3d6c7ff700 1 --2- 172.21.15.49:0/701023457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d50003be0 0x7f3d64158290 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.438+0000 7f3d6c7ff700 1 --2- 172.21.15.49:0/701023457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d6415afb0 0x7f3d640b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.438+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/701023457 >> 172.21.15.49:0/701023457 conn(0x7f3d6409c200 msgr2=0x7f3d6415ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:34.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.438+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/701023457 shutdown_connections 2023-07-18T03:39:34.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.438+0000 7f3d6c7ff700 1 -- 172.21.15.49:0/701023457 wait complete. 2023-07-18T03:39:34.452 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:39:34.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.664+0000 7fa8e49ff700 1 Processor -- start 2023-07-18T03:39:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.664+0000 7fa8e49ff700 1 -- start start 2023-07-18T03:39:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.665+0000 7fa8e49ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa8e00b8a60 0x7fa8e00b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.665+0000 7fa8e49ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e00b4530 0x7fa8e00d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.665+0000 7fa8e49ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa8e00d1050 0x7fa8e00b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8e49ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e00d14d0 con 0x7fa8e00d1050 2023-07-18T03:39:34.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8e49ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e00b8e30 con 0x7fa8e00b8a60 2023-07-18T03:39:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8e49ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00afaa0 con 0x7fa8e00b4530 2023-07-18T03:39:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8ded9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa8e00d1050 0x7fa8e00b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8ded9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa8e00d1050 0x7fa8e00b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53728/0 (socket says 172.21.15.49:53728) 2023-07-18T03:39:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8ded9d700 1 -- 172.21.15.49:0/3862294984 learned_addr learned my addr 172.21.15.49:0/3862294984 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8d7fff700 1 --2- 172.21.15.49:0/3862294984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e00b4530 0x7fa8e00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8de59c700 1 --2- 172.21.15.49:0/3862294984 >> v2:172.21.15.107:3300/0 conn(0x7fa8e00b8a60 0x7fa8e00b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8ded9d700 1 -- 172.21.15.49:0/3862294984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e00b4530 msgr2=0x7fa8e00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:34.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8ded9d700 1 --2- 172.21.15.49:0/3862294984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e00b4530 0x7fa8e00d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8ded9d700 1 -- 172.21.15.49:0/3862294984 >> v2:172.21.15.107:3300/0 conn(0x7fa8e00b8a60 msgr2=0x7fa8e00b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:34.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8ded9d700 1 --2- 172.21.15.49:0/3862294984 >> v2:172.21.15.107:3300/0 conn(0x7fa8e00b8a60 0x7fa8e00b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.666+0000 7fa8ded9d700 1 -- 172.21.15.49:0/3862294984 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e00afd80 con 0x7fa8e00d1050 2023-07-18T03:39:34.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.667+0000 7fa8ded9d700 1 --2- 172.21.15.49:0/3862294984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8e00d1050 0x7fa8e00b25d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00b5b00 tx=0x7fa8d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69df2be3cdfad5c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:34.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.667+0000 7fa8ddd9b700 1 -- 172.21.15.49:0/3862294984 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d800e070 con 0x7fa8e00d1050 2023-07-18T03:39:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.667+0000 7fa8ddd9b700 1 -- 172.21.15.49:0/3862294984 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d800b910 con 0x7fa8e00d1050 2023-07-18T03:39:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.667+0000 7fa8ddd9b700 1 -- 172.21.15.49:0/3862294984 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d800bcc0 con 0x7fa8e00d1050 2023-07-18T03:39:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.667+0000 7fa8e49ff700 1 -- 172.21.15.49:0/3862294984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8e00d1050 msgr2=0x7fa8e00b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.667+0000 7fa8e49ff700 1 --2- 172.21.15.49:0/3862294984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8e00d1050 0x7fa8e00b25d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00b5b00 tx=0x7fa8d800b330 comp rx=0 tx=0).stop 2023-07-18T03:39:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.668+0000 7fa8e49ff700 1 -- 172.21.15.49:0/3862294984 shutdown_connections 2023-07-18T03:39:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.668+0000 7fa8e49ff700 1 --2- 172.21.15.49:0/3862294984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8e00d1050 0x7fa8e00b25d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.668+0000 7fa8e49ff700 1 --2- 172.21.15.49:0/3862294984 >> v2:172.21.15.107:3300/0 conn(0x7fa8e00b8a60 0x7fa8e00b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.668+0000 7fa8e49ff700 1 --2- 172.21.15.49:0/3862294984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e00b4530 0x7fa8e00d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.668+0000 7fa8e49ff700 1 -- 172.21.15.49:0/3862294984 >> 172.21.15.49:0/3862294984 conn(0x7fa8e009c1b0 msgr2=0x7fa8e00d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:34.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.668+0000 7fa8e49ff700 1 -- 172.21.15.49:0/3862294984 shutdown_connections 2023-07-18T03:39:34.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.668+0000 7fa8e49ff700 1 -- 172.21.15.49:0/3862294984 wait complete. 2023-07-18T03:39:34.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.670+0000 7fa8e49ff700 1 Processor -- start 2023-07-18T03:39:34.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.670+0000 7fa8e49ff700 1 -- start start 2023-07-18T03:39:34.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.671+0000 7fa8e49ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8e00b4530 0x7fa8e015a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.671+0000 7fa8e49ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e00b8a60 0x7fa8e01f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.671+0000 7fa8e49ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e00d1050 0x7fa8e006b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.671+0000 7fa8e49ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e01561e0 con 0x7fa8e00b4530 2023-07-18T03:39:34.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.671+0000 7fa8e49ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00cc250 con 0x7fa8e00d1050 2023-07-18T03:39:34.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.671+0000 7fa8e49ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e01f6350 con 0x7fa8e00b8a60 2023-07-18T03:39:34.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.671+0000 7fa8de59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8e00b4530 0x7fa8e015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.671+0000 7fa8de59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8e00b4530 0x7fa8e015a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53738/0 (socket says 172.21.15.49:53738) 2023-07-18T03:39:34.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.671+0000 7fa8de59c700 1 -- 172.21.15.49:0/189581978 learned_addr learned my addr 172.21.15.49:0/189581978 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:34.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.671+0000 7fa8d7fff700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e00b8a60 0x7fa8e01f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.671+0000 7fa8ded9d700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e00d1050 0x7fa8e006b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.672+0000 7fa8de59c700 1 -- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e00b8a60 msgr2=0x7fa8e01f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:34.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.672+0000 7fa8de59c700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e00b8a60 0x7fa8e01f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.672+0000 7fa8de59c700 1 -- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e00d1050 msgr2=0x7fa8e006b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:34.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.672+0000 7fa8de59c700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e00d1050 0x7fa8e006b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.672+0000 7fa8de59c700 1 -- 172.21.15.49:0/189581978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d8009b30 con 0x7fa8e00b4530 2023-07-18T03:39:34.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.672+0000 7fa8ded9d700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e00d1050 0x7fa8e006b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:34.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.672+0000 7fa8de59c700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8e00b4530 0x7fa8e015a490 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000e9b0 tx=0x7fa8d000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:34.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.672+0000 7fa8d6ffd700 1 -- 172.21.15.49:0/189581978 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0009710 con 0x7fa8e00b4530 2023-07-18T03:39:34.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.672+0000 7fa8d6ffd700 1 -- 172.21.15.49:0/189581978 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d000c850 con 0x7fa8e00b4530 2023-07-18T03:39:34.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.672+0000 7fa8d6ffd700 1 -- 172.21.15.49:0/189581978 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d000cbc0 con 0x7fa8e00b4530 2023-07-18T03:39:34.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.672+0000 7fa8e49ff700 1 -- 172.21.15.49:0/189581978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e01f6630 con 0x7fa8e00b4530 2023-07-18T03:39:34.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.672+0000 7fa8e49ff700 1 -- 172.21.15.49:0/189581978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e0159c90 con 0x7fa8e00b4530 2023-07-18T03:39:34.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.673+0000 7fa8e49ff700 1 -- 172.21.15.49:0/189581978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e0074350 con 0x7fa8e00b4530 2023-07-18T03:39:34.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.674+0000 7fa8d6ffd700 1 -- 172.21.15.49:0/189581978 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa8d000cd20 con 0x7fa8e00b4530 2023-07-18T03:39:34.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.676+0000 7fa8d6ffd700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa8c006acb0 0x7fa8c006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:34.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.676+0000 7fa8d6ffd700 1 -- 172.21.15.49:0/189581978 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa8d00894b0 con 0x7fa8e00b4530 2023-07-18T03:39:34.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.676+0000 7fa8d7fff700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa8c006acb0 0x7fa8c006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:34.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.677+0000 7fa8d7fff700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa8c006acb0 0x7fa8c006d160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00d08a0 tx=0x7fa8c8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:34.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.680+0000 7fa8d6ffd700 1 -- 172.21.15.49:0/189581978 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa8d0054510 con 0x7fa8e00b4530 2023-07-18T03:39:34.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.886+0000 7fa8e49ff700 1 -- 172.21.15.49:0/189581978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa8e01e55e0 con 0x7fa8e00b4530 2023-07-18T03:39:34.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.886+0000 7fa8d6ffd700 1 -- 172.21.15.49:0/189581978 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa8d0057b30 con 0x7fa8e00b4530 2023-07-18T03:39:34.882 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:39:34.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.889+0000 7fa8e49ff700 1 -- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa8c006acb0 msgr2=0x7fa8c006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:34.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.889+0000 7fa8e49ff700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa8c006acb0 0x7fa8c006d160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00d08a0 tx=0x7fa8c8009210 comp rx=0 tx=0).stop 2023-07-18T03:39:34.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.889+0000 7fa8e49ff700 1 -- 172.21.15.49:0/189581978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8e00b4530 msgr2=0x7fa8e015a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:34.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.889+0000 7fa8e49ff700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8e00b4530 0x7fa8e015a490 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000e9b0 tx=0x7fa8d000ecc0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.889+0000 7fa8e49ff700 1 -- 172.21.15.49:0/189581978 shutdown_connections 2023-07-18T03:39:34.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.889+0000 7fa8e49ff700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e00d1050 0x7fa8e006b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.889+0000 7fa8e49ff700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa8c006acb0 0x7fa8c006d160 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.889+0000 7fa8e49ff700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8e00b4530 0x7fa8e015a490 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.889+0000 7fa8e49ff700 1 --2- 172.21.15.49:0/189581978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e00b8a60 0x7fa8e01f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:34.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.889+0000 7fa8e49ff700 1 -- 172.21.15.49:0/189581978 >> 172.21.15.49:0/189581978 conn(0x7fa8e009c1b0 msgr2=0x7fa8e00d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:34.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.889+0000 7fa8e49ff700 1 -- 172.21.15.49:0/189581978 shutdown_connections 2023-07-18T03:39:34.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:34.889+0000 7fa8e49ff700 1 -- 172.21.15.49:0/189581978 wait complete. 2023-07-18T03:39:34.903 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:39:35.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.113+0000 7f960b3ff700 1 Processor -- start 2023-07-18T03:39:35.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.113+0000 7f960b3ff700 1 -- start start 2023-07-18T03:39:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.114+0000 7f960b3ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f960415c950 0x7f96040b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.115+0000 7f960b3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f96040d1220 0x7f96040afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.115+0000 7f960b3ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96040d2ce0 con 0x7f96040b1e00 2023-07-18T03:39:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.115+0000 7f960b3ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96040b3350 con 0x7f960415c950 2023-07-18T03:39:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.115+0000 7f960b3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96040b3630 con 0x7f96040d1220 2023-07-18T03:39:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.115+0000 7f960919b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f960415c950 0x7f96040b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.115+0000 7f960919b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f960415c950 0x7f96040b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53744/0 (socket says 172.21.15.49:53744) 2023-07-18T03:39:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.115+0000 7f960919b700 1 -- 172.21.15.49:0/479800530 learned_addr learned my addr 172.21.15.49:0/479800530 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.115+0000 7f960999c700 1 --2- 172.21.15.49:0/479800530 >> v2:172.21.15.107:3300/0 conn(0x7f96040d1220 0x7f96040afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.115+0000 7f960919b700 1 -- 172.21.15.49:0/479800530 >> v2:172.21.15.107:3300/0 conn(0x7f96040d1220 msgr2=0x7f96040afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.115+0000 7f960919b700 1 --2- 172.21.15.49:0/479800530 >> v2:172.21.15.107:3300/0 conn(0x7f96040d1220 0x7f96040afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.115+0000 7f960919b700 1 -- 172.21.15.49:0/479800530 >> v1:172.21.15.49:6789/0 conn(0x7f96040b1e00 legacy=0x7f96040d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.115+0000 7f960919b700 1 -- 172.21.15.49:0/479800530 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96040b3910 con 0x7f960415c950 2023-07-18T03:39:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.116+0000 7f960919b700 1 --2- 172.21.15.49:0/479800530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960415c950 0x7f96040b43f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f960409c8e0 tx=0x7f9600009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2ac1a81b774cdb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.116+0000 7f95fbfff700 1 -- 172.21.15.49:0/479800530 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960000b8a0 con 0x7f960415c950 2023-07-18T03:39:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.116+0000 7f95fbfff700 1 -- 172.21.15.49:0/479800530 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960000ba00 con 0x7f960415c950 2023-07-18T03:39:35.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.116+0000 7f95fbfff700 1 -- 172.21.15.49:0/479800530 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960000bdb0 con 0x7f960415c950 2023-07-18T03:39:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.116+0000 7f960b3ff700 1 -- 172.21.15.49:0/479800530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960415c950 msgr2=0x7f96040b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.116+0000 7f960b3ff700 1 --2- 172.21.15.49:0/479800530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960415c950 0x7f96040b43f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f960409c8e0 tx=0x7f9600009bc0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.116+0000 7f960b3ff700 1 -- 172.21.15.49:0/479800530 shutdown_connections 2023-07-18T03:39:35.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.117+0000 7f960b3ff700 1 --2- 172.21.15.49:0/479800530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960415c950 0x7f96040b43f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.117+0000 7f960b3ff700 1 --2- 172.21.15.49:0/479800530 >> v2:172.21.15.107:3300/0 conn(0x7f96040d1220 0x7f96040afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.117+0000 7f960b3ff700 1 -- 172.21.15.49:0/479800530 >> 172.21.15.49:0/479800530 conn(0x7f960409c1d0 msgr2=0x7f96040d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:35.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.117+0000 7f960b3ff700 1 -- 172.21.15.49:0/479800530 shutdown_connections 2023-07-18T03:39:35.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.117+0000 7f960b3ff700 1 -- 172.21.15.49:0/479800530 wait complete. 2023-07-18T03:39:35.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.118+0000 7f960b3ff700 1 Processor -- start 2023-07-18T03:39:35.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.119+0000 7f960b3ff700 1 -- start start 2023-07-18T03:39:35.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.119+0000 7f960b3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96040b1e00 0x7f96040b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.119+0000 7f960b3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96040d1220 0x7f96040cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960b3ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960415c950 0x7f96040c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960b3ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96040cc440 con 0x7f960415c950 2023-07-18T03:39:35.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960b3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96040b6820 con 0x7f96040d1220 2023-07-18T03:39:35.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960b3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96040c8a30 con 0x7f96040b1e00 2023-07-18T03:39:35.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960999c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960415c950 0x7f96040c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960999c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960415c950 0x7f96040c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53752/0 (socket says 172.21.15.49:53752) 2023-07-18T03:39:35.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960999c700 1 -- 172.21.15.49:0/2801161316 learned_addr learned my addr 172.21.15.49:0/2801161316 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:35.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960919b700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96040b1e00 0x7f96040b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960899a700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96040d1220 0x7f96040cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960999c700 1 -- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96040b1e00 msgr2=0x7f96040b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960999c700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96040b1e00 0x7f96040b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960999c700 1 -- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96040d1220 msgr2=0x7f96040cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960999c700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96040d1220 0x7f96040cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.120+0000 7f960999c700 1 -- 172.21.15.49:0/2801161316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96000094f0 con 0x7f960415c950 2023-07-18T03:39:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.121+0000 7f960999c700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960415c950 0x7f96040c84f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f95fc007e30 tx=0x7f95fc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.121+0000 7f95f9ffb700 1 -- 172.21.15.49:0/2801161316 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00ed30 con 0x7f960415c950 2023-07-18T03:39:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.121+0000 7f95f9ffb700 1 -- 172.21.15.49:0/2801161316 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc00ee90 con 0x7f960415c950 2023-07-18T03:39:35.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.121+0000 7f95f9ffb700 1 -- 172.21.15.49:0/2801161316 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc014660 con 0x7f960415c950 2023-07-18T03:39:35.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.121+0000 7f960b3ff700 1 -- 172.21.15.49:0/2801161316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96040b6ac0 con 0x7f960415c950 2023-07-18T03:39:35.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.121+0000 7f960b3ff700 1 -- 172.21.15.49:0/2801161316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96040b7080 con 0x7f960415c950 2023-07-18T03:39:35.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.122+0000 7f960b3ff700 1 -- 172.21.15.49:0/2801161316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960404e280 con 0x7f960415c950 2023-07-18T03:39:35.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.123+0000 7f95f9ffb700 1 -- 172.21.15.49:0/2801161316 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f95fc0147c0 con 0x7f960415c950 2023-07-18T03:39:35.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.124+0000 7f95f9ffb700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f95e806ace0 0x7f95e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.124+0000 7f95f9ffb700 1 -- 172.21.15.49:0/2801161316 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f95fc089c10 con 0x7f960415c950 2023-07-18T03:39:35.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.125+0000 7f960919b700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f95e806ace0 0x7f95e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.126+0000 7f960919b700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f95e806ace0 0x7f95e806d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9600014040 tx=0x7f9600008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:35.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.129+0000 7f95f9ffb700 1 -- 172.21.15.49:0/2801161316 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f95fc058670 con 0x7f960415c950 2023-07-18T03:39:35.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.381+0000 7f960b3ff700 1 -- 172.21.15.49:0/2801161316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f960408c5a0 con 0x7f960415c950 2023-07-18T03:39:35.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.381+0000 7f95f9ffb700 1 -- 172.21.15.49:0/2801161316 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4709 (secure 0 0 0) 0x7f95fc058200 con 0x7f960415c950 2023-07-18T03:39:35.377 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:35.377 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1477016053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1477016053},{"type":"v1","addr":"172.21.15.107:6835","nonce":1477016053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2998451500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2998451500},{"type":"v1","addr":"172.21.15.49:6835","nonce":2998451500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4247281109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4247281109},{"type":"v1","addr":"172.21.15.49:6833","nonce":4247281109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:38:36.025442+0000","modified":"2023-07-18T03:38:39.535929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1127901907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1127901907},{"type":"v1","addr":"172.21.15.107:6837","nonce":1127901907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:39:35.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.384+0000 7f960b3ff700 1 -- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f95e806ace0 msgr2=0x7f95e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.384+0000 7f960b3ff700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f95e806ace0 0x7f95e806d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9600014040 tx=0x7f9600008030 comp rx=0 tx=0).stop 2023-07-18T03:39:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.384+0000 7f960b3ff700 1 -- 172.21.15.49:0/2801161316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960415c950 msgr2=0x7f96040c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.384+0000 7f960b3ff700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960415c950 0x7f96040c84f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f95fc007e30 tx=0x7f95fc00e3c0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.384+0000 7f960b3ff700 1 -- 172.21.15.49:0/2801161316 shutdown_connections 2023-07-18T03:39:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.384+0000 7f960b3ff700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f95e806ace0 0x7f95e806d190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.384+0000 7f960b3ff700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f960415c950 0x7f96040c84f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.384+0000 7f960b3ff700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96040b1e00 0x7f96040b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.384+0000 7f960b3ff700 1 --2- 172.21.15.49:0/2801161316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96040d1220 0x7f96040cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.384+0000 7f960b3ff700 1 -- 172.21.15.49:0/2801161316 >> 172.21.15.49:0/2801161316 conn(0x7f960409c1d0 msgr2=0x7f96040afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:35.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.384+0000 7f960b3ff700 1 -- 172.21.15.49:0/2801161316 shutdown_connections 2023-07-18T03:39:35.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.384+0000 7f960b3ff700 1 -- 172.21.15.49:0/2801161316 wait complete. 2023-07-18T03:39:35.381 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2023-07-18T03:39:35.397 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:35.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.606+0000 7f9064fff700 1 Processor -- start 2023-07-18T03:39:35.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.606+0000 7f9064fff700 1 -- start start 2023-07-18T03:39:35.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.607+0000 7f9064fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f906015c950 0x7f90600b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.607+0000 7f9064fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90600d1220 0x7f90600afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.607+0000 7f9064fff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90600d2ce0 con 0x7f90600b1e00 2023-07-18T03:39:35.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.607+0000 7f9064fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90600b3350 con 0x7f906015c950 2023-07-18T03:39:35.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.607+0000 7f9064fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90600b3630 con 0x7f90600d1220 2023-07-18T03:39:35.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.607+0000 7f905e59c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f906015c950 0x7f90600b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.607+0000 7f9057fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f90600b1e00 0x7f90600d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57786/0 (socket says 172.21.15.49:57786) 2023-07-18T03:39:35.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.607+0000 7f9057fff700 1 -- 172.21.15.49:0/1369425648 learned_addr learned my addr 172.21.15.49:0/1369425648 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:35.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.607+0000 7f905ed9d700 1 --2- 172.21.15.49:0/1369425648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90600d1220 0x7f90600afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.608+0000 7f905e59c700 1 -- 172.21.15.49:0/1369425648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90600d1220 msgr2=0x7f90600afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:35.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.608+0000 7f905e59c700 1 --2- 172.21.15.49:0/1369425648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90600d1220 0x7f90600afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.608+0000 7f905e59c700 1 -- 172.21.15.49:0/1369425648 >> v1:172.21.15.49:6789/0 conn(0x7f90600b1e00 legacy=0x7f90600d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.608+0000 7f905e59c700 1 -- 172.21.15.49:0/1369425648 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90600b3910 con 0x7f906015c950 2023-07-18T03:39:35.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.608+0000 7f905dd9b700 1 -- 172.21.15.49:0/1369425648 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821456698 0 0) 0x7f90600d2ce0 con 0x7f90600b1e00 2023-07-18T03:39:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.608+0000 7f905e59c700 1 --2- 172.21.15.49:0/1369425648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f906015c950 0x7f90600b43f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9050009830 tx=0x7f9050009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19921c56a8bd1208 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.608+0000 7f905dd9b700 1 -- 172.21.15.49:0/1369425648 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050011830 con 0x7f906015c950 2023-07-18T03:39:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.608+0000 7f905dd9b700 1 -- 172.21.15.49:0/1369425648 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9050011990 con 0x7f906015c950 2023-07-18T03:39:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.608+0000 7f905dd9b700 1 -- 172.21.15.49:0/1369425648 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050011d40 con 0x7f906015c950 2023-07-18T03:39:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.609+0000 7f9064fff700 1 -- 172.21.15.49:0/1369425648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f906015c950 msgr2=0x7f90600b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.609+0000 7f9064fff700 1 --2- 172.21.15.49:0/1369425648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f906015c950 0x7f90600b43f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9050009830 tx=0x7f9050009b40 comp rx=0 tx=0).stop 2023-07-18T03:39:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.609+0000 7f9064fff700 1 -- 172.21.15.49:0/1369425648 shutdown_connections 2023-07-18T03:39:35.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.609+0000 7f9064fff700 1 --2- 172.21.15.49:0/1369425648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90600d1220 0x7f90600afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.609+0000 7f9064fff700 1 --2- 172.21.15.49:0/1369425648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f906015c950 0x7f90600b43f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.609+0000 7f9064fff700 1 -- 172.21.15.49:0/1369425648 >> 172.21.15.49:0/1369425648 conn(0x7f906009c1d0 msgr2=0x7f90600d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.609+0000 7f9064fff700 1 -- 172.21.15.49:0/1369425648 shutdown_connections 2023-07-18T03:39:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.609+0000 7f9064fff700 1 -- 172.21.15.49:0/1369425648 wait complete. 2023-07-18T03:39:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.611+0000 7f9064fff700 1 Processor -- start 2023-07-18T03:39:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.611+0000 7f9064fff700 1 -- start start 2023-07-18T03:39:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.612+0000 7f9064fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90600b1e00 0x7f90601568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.612+0000 7f9064fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90600d1220 0x7f906006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.612+0000 7f9064fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f906015c950 0x7f906006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.612+0000 7f9064fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9060156e30 con 0x7f90600b1e00 2023-07-18T03:39:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.612+0000 7f9064fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90600cc6e0 con 0x7f906015c950 2023-07-18T03:39:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.612+0000 7f9064fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90600cc9c0 con 0x7f90600d1220 2023-07-18T03:39:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.612+0000 7f905e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90600b1e00 0x7f90601568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.612+0000 7f905e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90600b1e00 0x7f90601568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53770/0 (socket says 172.21.15.49:53770) 2023-07-18T03:39:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.612+0000 7f905e59c700 1 -- 172.21.15.49:0/1704162370 learned_addr learned my addr 172.21.15.49:0/1704162370 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f9057fff700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90600d1220 0x7f906006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f905e59c700 1 -- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90600d1220 msgr2=0x7f906006b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f905e59c700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90600d1220 0x7f906006b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f905ed9d700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f906015c950 0x7f906006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f905e59c700 1 -- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f906015c950 msgr2=0x7f906006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f905e59c700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f906015c950 0x7f906006bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f905e59c700 1 -- 172.21.15.49:0/1704162370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9050009440 con 0x7f90600b1e00 2023-07-18T03:39:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f9057fff700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90600d1220 0x7f906006b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f905ed9d700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f906015c950 0x7f906006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f905e59c700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90600b1e00 0x7f90601568f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f90500090b0 tx=0x7f9050004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f9056ffd700 1 -- 172.21.15.49:0/1704162370 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905001a7a0 con 0x7f90600b1e00 2023-07-18T03:39:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f9056ffd700 1 -- 172.21.15.49:0/1704162370 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905001a900 con 0x7f90600b1e00 2023-07-18T03:39:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f9064fff700 1 -- 172.21.15.49:0/1704162370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90600ccca0 con 0x7f90600b1e00 2023-07-18T03:39:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.613+0000 7f9056ffd700 1 -- 172.21.15.49:0/1704162370 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905001acb0 con 0x7f90600b1e00 2023-07-18T03:39:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.614+0000 7f9064fff700 1 -- 172.21.15.49:0/1704162370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90600ca5d0 con 0x7f90600b1e00 2023-07-18T03:39:35.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.614+0000 7f9064fff700 1 -- 172.21.15.49:0/1704162370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90600cd740 con 0x7f90600b1e00 2023-07-18T03:39:35.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.615+0000 7f9056ffd700 1 -- 172.21.15.49:0/1704162370 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9050022070 con 0x7f90600b1e00 2023-07-18T03:39:35.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.616+0000 7f9056ffd700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f904006ace0 0x7f904006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.616+0000 7f9056ffd700 1 -- 172.21.15.49:0/1704162370 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f905008a550 con 0x7f90600b1e00 2023-07-18T03:39:35.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.617+0000 7f9057fff700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f904006ace0 0x7f904006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.618+0000 7f9057fff700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f904006ace0 0x7f904006d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9060157b80 tx=0x7f9048009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:35.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.620+0000 7f9056ffd700 1 -- 172.21.15.49:0/1704162370 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9050055650 con 0x7f90600b1e00 2023-07-18T03:39:35.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.830+0000 7f9064fff700 1 -- 172.21.15.49:0/1704162370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f906004e280 con 0x7f90600b1e00 2023-07-18T03:39:35.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.831+0000 7f9056ffd700 1 -- 172.21.15.49:0/1704162370 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f9050058c70 con 0x7f90600b1e00 2023-07-18T03:39:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:35.827 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:23.937174+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:35.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.833+0000 7f9064fff700 1 -- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f904006ace0 msgr2=0x7f904006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.833+0000 7f9064fff700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f904006ace0 0x7f904006d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9060157b80 tx=0x7f9048009040 comp rx=0 tx=0).stop 2023-07-18T03:39:35.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.833+0000 7f9064fff700 1 -- 172.21.15.49:0/1704162370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90600b1e00 msgr2=0x7f90601568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.833+0000 7f9064fff700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90600b1e00 0x7f90601568f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f90500090b0 tx=0x7f9050004750 comp rx=0 tx=0).stop 2023-07-18T03:39:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.833+0000 7f9064fff700 1 -- 172.21.15.49:0/1704162370 shutdown_connections 2023-07-18T03:39:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.833+0000 7f9064fff700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f906015c950 0x7f906006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.833+0000 7f9064fff700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f904006ace0 0x7f904006d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.833+0000 7f9064fff700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90600b1e00 0x7f90601568f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.833+0000 7f9064fff700 1 --2- 172.21.15.49:0/1704162370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90600d1220 0x7f906006b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.833+0000 7f9064fff700 1 -- 172.21.15.49:0/1704162370 >> 172.21.15.49:0/1704162370 conn(0x7f906009c1d0 msgr2=0x7f90600afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:35.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.834+0000 7f9064fff700 1 -- 172.21.15.49:0/1704162370 shutdown_connections 2023-07-18T03:39:35.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:35.834+0000 7f9064fff700 1 -- 172.21.15.49:0/1704162370 wait complete. 2023-07-18T03:39:35.847 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-18T03:39:35.848 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:39:36.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.057+0000 7f84ff9ff700 1 Processor -- start 2023-07-18T03:39:36.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.057+0000 7f84ff9ff700 1 -- start start 2023-07-18T03:39:36.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.059+0000 7f84ff9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f84f815c710 0x7f84f80cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:36.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.059+0000 7f84ff9ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f84f806db30 0x7f84f806a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:36.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.059+0000 7f84ff9ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f80cd320 con 0x7f84f8069f30 2023-07-18T03:39:36.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.059+0000 7f84ff9ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f80ce750 con 0x7f84f806db30 2023-07-18T03:39:36.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.059+0000 7f84ff9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f80cea30 con 0x7f84f815c710 2023-07-18T03:39:36.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.059+0000 7f84fcf9a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f84f8069f30 0x7f84f806d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57798/0 (socket says 172.21.15.49:57798) 2023-07-18T03:39:36.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.059+0000 7f84fdf9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f84f806db30 0x7f84f806a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:36.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.059+0000 7f84fcf9a700 1 -- 172.21.15.49:0/2556142527 learned_addr learned my addr 172.21.15.49:0/2556142527 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:36.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.059+0000 7f84fd79b700 1 --2- 172.21.15.49:0/2556142527 >> v2:172.21.15.107:3300/0 conn(0x7f84f815c710 0x7f84f80cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:36.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.060+0000 7f84fdf9c700 1 -- 172.21.15.49:0/2556142527 >> v2:172.21.15.107:3300/0 conn(0x7f84f815c710 msgr2=0x7f84f80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.060+0000 7f84fdf9c700 1 --2- 172.21.15.49:0/2556142527 >> v2:172.21.15.107:3300/0 conn(0x7f84f815c710 0x7f84f80cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.060+0000 7f84fdf9c700 1 -- 172.21.15.49:0/2556142527 >> v1:172.21.15.49:6789/0 conn(0x7f84f8069f30 legacy=0x7f84f806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.059+0000 7f84effff700 1 -- 172.21.15.49:0/2556142527 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044085213 0 0) 0x7f84f80cd320 con 0x7f84f8069f30 2023-07-18T03:39:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.060+0000 7f84fdf9c700 1 -- 172.21.15.49:0/2556142527 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f80ced10 con 0x7f84f806db30 2023-07-18T03:39:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.060+0000 7f84fdf9c700 1 --2- 172.21.15.49:0/2556142527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f806db30 0x7f84f806a3a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f84f8072690 tx=0x7f84f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18879b51fbae3684 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.060+0000 7f84effff700 1 -- 172.21.15.49:0/2556142527 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f000e070 con 0x7f84f806db30 2023-07-18T03:39:36.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.060+0000 7f84effff700 1 -- 172.21.15.49:0/2556142527 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f000bb20 con 0x7f84f806db30 2023-07-18T03:39:36.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.060+0000 7f84effff700 1 -- 172.21.15.49:0/2556142527 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f0012430 con 0x7f84f806db30 2023-07-18T03:39:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.061+0000 7f84ff9ff700 1 -- 172.21.15.49:0/2556142527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f806db30 msgr2=0x7f84f806a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.061+0000 7f84ff9ff700 1 --2- 172.21.15.49:0/2556142527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f806db30 0x7f84f806a3a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f84f8072690 tx=0x7f84f000b330 comp rx=0 tx=0).stop 2023-07-18T03:39:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.061+0000 7f84ff9ff700 1 -- 172.21.15.49:0/2556142527 shutdown_connections 2023-07-18T03:39:36.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.061+0000 7f84ff9ff700 1 --2- 172.21.15.49:0/2556142527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f806db30 0x7f84f806a3a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.061+0000 7f84ff9ff700 1 --2- 172.21.15.49:0/2556142527 >> v2:172.21.15.107:3300/0 conn(0x7f84f815c710 0x7f84f80cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.061+0000 7f84ff9ff700 1 -- 172.21.15.49:0/2556142527 >> 172.21.15.49:0/2556142527 conn(0x7f84f80b16c0 msgr2=0x7f84f80b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:36.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.061+0000 7f84ff9ff700 1 -- 172.21.15.49:0/2556142527 shutdown_connections 2023-07-18T03:39:36.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.061+0000 7f84ff9ff700 1 -- 172.21.15.49:0/2556142527 wait complete. 2023-07-18T03:39:36.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.063+0000 7f84ff9ff700 1 Processor -- start 2023-07-18T03:39:36.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.063+0000 7f84ff9ff700 1 -- start start 2023-07-18T03:39:36.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84ff9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f8069f30 0x7f84f80b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:36.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84ff9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84f806db30 0x7f84f81599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:36.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84ff9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f8159f00 0x7f84f80b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:36.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84ff9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f815e3f0 con 0x7f84f8159f00 2023-07-18T03:39:36.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84ff9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f80b5aa0 con 0x7f84f8069f30 2023-07-18T03:39:36.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84ff9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f815d770 con 0x7f84f806db30 2023-07-18T03:39:36.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84fdf9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f8159f00 0x7f84f80b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:36.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84fdf9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f8159f00 0x7f84f80b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53784/0 (socket says 172.21.15.49:53784) 2023-07-18T03:39:36.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84fdf9c700 1 -- 172.21.15.49:0/4173096449 learned_addr learned my addr 172.21.15.49:0/4173096449 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:36.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84fcf9a700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84f806db30 0x7f84f81599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:36.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84fd79b700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f8069f30 0x7f84f80b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:36.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84fdf9c700 1 -- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84f806db30 msgr2=0x7f84f81599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84fdf9c700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84f806db30 0x7f84f81599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84fdf9c700 1 -- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f8069f30 msgr2=0x7f84f80b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84fdf9c700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f8069f30 0x7f84f80b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84fdf9c700 1 -- 172.21.15.49:0/4173096449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f0009a70 con 0x7f84f8159f00 2023-07-18T03:39:36.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.064+0000 7f84fcf9a700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84f806db30 0x7f84f81599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:36.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.065+0000 7f84fdf9c700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f8159f00 0x7f84f80b2ae0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f84f0014040 tx=0x7f84f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:36.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.065+0000 7f84edffb700 1 -- 172.21.15.49:0/4173096449 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f0012430 con 0x7f84f8159f00 2023-07-18T03:39:36.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.065+0000 7f84edffb700 1 -- 172.21.15.49:0/4173096449 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f0020070 con 0x7f84f8159f00 2023-07-18T03:39:36.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.065+0000 7f84edffb700 1 -- 172.21.15.49:0/4173096449 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f0004280 con 0x7f84f8159f00 2023-07-18T03:39:36.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.065+0000 7f84ff9ff700 1 -- 172.21.15.49:0/4173096449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f806ac20 con 0x7f84f8159f00 2023-07-18T03:39:36.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.065+0000 7f84ff9ff700 1 -- 172.21.15.49:0/4173096449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f815dd30 con 0x7f84f8159f00 2023-07-18T03:39:36.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.066+0000 7f84ff9ff700 1 -- 172.21.15.49:0/4173096449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f815e530 con 0x7f84f8159f00 2023-07-18T03:39:36.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.067+0000 7f84edffb700 1 -- 172.21.15.49:0/4173096449 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f84f00043e0 con 0x7f84f8159f00 2023-07-18T03:39:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.068+0000 7f84edffb700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f84dc06ad10 0x7f84dc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.068+0000 7f84edffb700 1 -- 172.21.15.49:0/4173096449 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f84f0022070 con 0x7f84f8159f00 2023-07-18T03:39:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.069+0000 7f84fd79b700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f84dc06ad10 0x7f84dc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:36.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.069+0000 7f84fd79b700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f84dc06ad10 0x7f84dc06d1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009b40 tx=0x7f84f4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.072+0000 7f84edffb700 1 -- 172.21.15.49:0/4173096449 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f84f815e530 con 0x7f84f8159f00 2023-07-18T03:39:36.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.327+0000 7f84ff9ff700 1 -- 172.21.15.49:0/4173096449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84f815e530 con 0x7f84f8159f00 2023-07-18T03:39:36.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.328+0000 7f84edffb700 1 -- 172.21.15.49:0/4173096449 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4709 (secure 0 0 0) 0x7f84f815e530 con 0x7f84f8159f00 2023-07-18T03:39:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:36.324 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1477016053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1477016053},{"type":"v1","addr":"172.21.15.107:6835","nonce":1477016053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2998451500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2998451500},{"type":"v1","addr":"172.21.15.49:6835","nonce":2998451500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4247281109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4247281109},{"type":"v1","addr":"172.21.15.49:6833","nonce":4247281109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:38:36.025442+0000","modified":"2023-07-18T03:38:39.535929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1127901907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1127901907},{"type":"v1","addr":"172.21.15.107:6837","nonce":1127901907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:39:36.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.331+0000 7f84ff9ff700 1 -- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f84dc06ad10 msgr2=0x7f84dc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.331+0000 7f84ff9ff700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f84dc06ad10 0x7f84dc06d1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009b40 tx=0x7f84f4009210 comp rx=0 tx=0).stop 2023-07-18T03:39:36.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.331+0000 7f84ff9ff700 1 -- 172.21.15.49:0/4173096449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f8159f00 msgr2=0x7f84f80b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.331+0000 7f84ff9ff700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f8159f00 0x7f84f80b2ae0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f84f0014040 tx=0x7f84f0004ad0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.331+0000 7f84ff9ff700 1 -- 172.21.15.49:0/4173096449 shutdown_connections 2023-07-18T03:39:36.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.331+0000 7f84ff9ff700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f84dc06ad10 0x7f84dc06d1c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.331+0000 7f84ff9ff700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f8159f00 0x7f84f80b2ae0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.331+0000 7f84ff9ff700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f8069f30 0x7f84f80b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.331+0000 7f84ff9ff700 1 --2- 172.21.15.49:0/4173096449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84f806db30 0x7f84f81599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.331+0000 7f84ff9ff700 1 -- 172.21.15.49:0/4173096449 >> 172.21.15.49:0/4173096449 conn(0x7f84f80b16c0 msgr2=0x7f84f80cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:36.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.331+0000 7f84ff9ff700 1 -- 172.21.15.49:0/4173096449 shutdown_connections 2023-07-18T03:39:36.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.331+0000 7f84ff9ff700 1 -- 172.21.15.49:0/4173096449 wait complete. 2023-07-18T03:39:36.328 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2023-07-18T03:39:36.345 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:39:36.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.554+0000 7fd593bff700 1 Processor -- start 2023-07-18T03:39:36.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.554+0000 7fd593bff700 1 -- start start 2023-07-18T03:39:36.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.555+0000 7fd593bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd58c0d1160 0x7fd58c06c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:36.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.555+0000 7fd593bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd58c0d2cc0 con 0x7fd58c0b5cd0 2023-07-18T03:39:36.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.556+0000 7fd593bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd58c0afbb0 con 0x7fd58c160030 2023-07-18T03:39:36.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.556+0000 7fd593bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58c0afe90 con 0x7fd58c0d1160 2023-07-18T03:39:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.556+0000 7fd59219c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd58c0b5cd0 0x7fd58c0b0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57814/0 (socket says 172.21.15.49:57814) 2023-07-18T03:39:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.556+0000 7fd59219c700 1 -- 172.21.15.49:0/1120108027 learned_addr learned my addr 172.21.15.49:0/1120108027 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.556+0000 7fd59119a700 1 --2- 172.21.15.49:0/1120108027 >> v2:172.21.15.107:3300/0 conn(0x7fd58c0d1160 0x7fd58c06c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:36.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.556+0000 7fd590999700 1 -- 172.21.15.49:0/1120108027 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984615726 0 0) 0x7fd58c0d2cc0 con 0x7fd58c0b5cd0 2023-07-18T03:39:36.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.556+0000 7fd590999700 1 -- 172.21.15.49:0/1120108027 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5780035b0 con 0x7fd58c0b5cd0 2023-07-18T03:39:36.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.556+0000 7fd590999700 1 -- 172.21.15.49:0/1120108027 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814806055 0 0) 0x7fd58c0afbb0 con 0x7fd58c160030 2023-07-18T03:39:36.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.556+0000 7fd590999700 1 -- 172.21.15.49:0/1120108027 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd58c0d2cc0 con 0x7fd58c160030 2023-07-18T03:39:36.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.557+0000 7fd59119a700 1 -- 172.21.15.49:0/1120108027 >> v1:172.21.15.107:6789/0 conn(0x7fd58c160030 legacy=0x7fd58c0d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.557+0000 7fd59119a700 1 -- 172.21.15.49:0/1120108027 >> v1:172.21.15.49:6789/0 conn(0x7fd58c0b5cd0 legacy=0x7fd58c0b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.557+0000 7fd59119a700 1 -- 172.21.15.49:0/1120108027 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c0b0170 con 0x7fd58c0d1160 2023-07-18T03:39:36.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.557+0000 7fd59119a700 1 --2- 172.21.15.49:0/1120108027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd58c0d1160 0x7fd58c06c0f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009840 tx=0x7fd57c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aa142eeb7df09a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:36.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.557+0000 7fd590999700 1 -- 172.21.15.49:0/1120108027 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c011830 con 0x7fd58c0d1160 2023-07-18T03:39:36.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.557+0000 7fd590999700 1 -- 172.21.15.49:0/1120108027 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c011990 con 0x7fd58c0d1160 2023-07-18T03:39:36.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.557+0000 7fd590999700 1 -- 172.21.15.49:0/1120108027 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c011d20 con 0x7fd58c0d1160 2023-07-18T03:39:36.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.558+0000 7fd593bff700 1 -- 172.21.15.49:0/1120108027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd58c0d1160 msgr2=0x7fd58c06c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.558+0000 7fd593bff700 1 --2- 172.21.15.49:0/1120108027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd58c0d1160 0x7fd58c06c0f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009840 tx=0x7fd57c009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:36.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.558+0000 7fd593bff700 1 -- 172.21.15.49:0/1120108027 shutdown_connections 2023-07-18T03:39:36.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.558+0000 7fd593bff700 1 --2- 172.21.15.49:0/1120108027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd58c0d1160 0x7fd58c06c0f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.558+0000 7fd593bff700 1 -- 172.21.15.49:0/1120108027 >> 172.21.15.49:0/1120108027 conn(0x7fd58c09c190 msgr2=0x7fd58c0d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:36.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.558+0000 7fd593bff700 1 -- 172.21.15.49:0/1120108027 shutdown_connections 2023-07-18T03:39:36.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.558+0000 7fd593bff700 1 -- 172.21.15.49:0/1120108027 wait complete. 2023-07-18T03:39:36.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.560+0000 7fd593bff700 1 Processor -- start 2023-07-18T03:39:36.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.560+0000 7fd593bff700 1 -- start start 2023-07-18T03:39:36.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd593bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd58c0b5cd0 0x7fd58c15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:36.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd593bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c0d1160 0x7fd58c1e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:36.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd593bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd58c160030 0x7fd58c0c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:36.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd593bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1e6cc0 con 0x7fd58c0d1160 2023-07-18T03:39:36.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd593bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1e6fa0 con 0x7fd58c0b5cd0 2023-07-18T03:39:36.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd593bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1e7280 con 0x7fd58c160030 2023-07-18T03:39:36.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd59119a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c0d1160 0x7fd58c1e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:36.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd59119a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c0d1160 0x7fd58c1e7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53792/0 (socket says 172.21.15.49:53792) 2023-07-18T03:39:36.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd59119a700 1 -- 172.21.15.49:0/2840947860 learned_addr learned my addr 172.21.15.49:0/2840947860 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:36.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd59219c700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd58c160030 0x7fd58c0c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:36.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd59199b700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd58c0b5cd0 0x7fd58c15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:36.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd59119a700 1 -- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd58c160030 msgr2=0x7fd58c0c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd59119a700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd58c160030 0x7fd58c0c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd59119a700 1 -- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd58c0b5cd0 msgr2=0x7fd58c15e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.561+0000 7fd59119a700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd58c0b5cd0 0x7fd58c15e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.562+0000 7fd59119a700 1 -- 172.21.15.49:0/2840947860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c0094a0 con 0x7fd58c0d1160 2023-07-18T03:39:36.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.562+0000 7fd59199b700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd58c0b5cd0 0x7fd58c15e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:36.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.562+0000 7fd59219c700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd58c160030 0x7fd58c0c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:36.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.562+0000 7fd59119a700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c0d1160 0x7fd58c1e7d60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0090b0 tx=0x7fd57c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:36.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.562+0000 7fd5827fc700 1 -- 172.21.15.49:0/2840947860 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c01a860 con 0x7fd58c0d1160 2023-07-18T03:39:36.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.562+0000 7fd5827fc700 1 -- 172.21.15.49:0/2840947860 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c01a9c0 con 0x7fd58c0d1160 2023-07-18T03:39:36.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.562+0000 7fd593bff700 1 -- 172.21.15.49:0/2840947860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58c1e7560 con 0x7fd58c0d1160 2023-07-18T03:39:36.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.562+0000 7fd5827fc700 1 -- 172.21.15.49:0/2840947860 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c01ad30 con 0x7fd58c0d1160 2023-07-18T03:39:36.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.562+0000 7fd593bff700 1 -- 172.21.15.49:0/2840947860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c15e7e0 con 0x7fd58c0d1160 2023-07-18T03:39:36.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.563+0000 7fd593bff700 1 -- 172.21.15.49:0/2840947860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58c1e7a80 con 0x7fd58c0d1160 2023-07-18T03:39:36.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.564+0000 7fd5827fc700 1 -- 172.21.15.49:0/2840947860 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd57c022070 con 0x7fd58c0d1160 2023-07-18T03:39:36.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.565+0000 7fd5827fc700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd57006ace0 0x7fd57006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:36.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.565+0000 7fd5827fc700 1 -- 172.21.15.49:0/2840947860 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd57c08a0e0 con 0x7fd58c0d1160 2023-07-18T03:39:36.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.566+0000 7fd59199b700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd57006ace0 0x7fd57006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:36.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.566+0000 7fd59199b700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd57006ace0 0x7fd57006d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5880011e0 tx=0x7fd588009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:36.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.569+0000 7fd5827fc700 1 -- 172.21.15.49:0/2840947860 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd58c1e7a80 con 0x7fd58c0d1160 2023-07-18T03:39:36.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.822+0000 7fd593bff700 1 -- 172.21.15.49:0/2840947860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd58c1e7a80 con 0x7fd58c0d1160 2023-07-18T03:39:36.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.823+0000 7fd5827fc700 1 -- 172.21.15.49:0/2840947860 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4709 (secure 0 0 0) 0x7fd58c1e7a80 con 0x7fd58c0d1160 2023-07-18T03:39:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:36.819 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1477016053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1477016053},{"type":"v1","addr":"172.21.15.107:6835","nonce":1477016053}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2998451500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2998451500},{"type":"v1","addr":"172.21.15.49:6835","nonce":2998451500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4247281109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4247281109},{"type":"v1","addr":"172.21.15.49:6833","nonce":4247281109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:38:36.025442+0000","modified":"2023-07-18T03:38:39.535929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1127901907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1127901907},{"type":"v1","addr":"172.21.15.107:6837","nonce":1127901907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:39:36.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.825+0000 7fd593bff700 1 -- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd57006ace0 msgr2=0x7fd57006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.825+0000 7fd593bff700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd57006ace0 0x7fd57006d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5880011e0 tx=0x7fd588009040 comp rx=0 tx=0).stop 2023-07-18T03:39:36.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.825+0000 7fd593bff700 1 -- 172.21.15.49:0/2840947860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c0d1160 msgr2=0x7fd58c1e7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:36.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.825+0000 7fd593bff700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c0d1160 0x7fd58c1e7d60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0090b0 tx=0x7fd57c004750 comp rx=0 tx=0).stop 2023-07-18T03:39:36.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.825+0000 7fd593bff700 1 -- 172.21.15.49:0/2840947860 shutdown_connections 2023-07-18T03:39:36.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.825+0000 7fd593bff700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd58c160030 0x7fd58c0c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.825+0000 7fd593bff700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd58c0b5cd0 0x7fd58c15e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.825+0000 7fd593bff700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd57006ace0 0x7fd57006d190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.825+0000 7fd593bff700 1 --2- 172.21.15.49:0/2840947860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c0d1160 0x7fd58c1e7d60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:36.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.825+0000 7fd593bff700 1 -- 172.21.15.49:0/2840947860 >> 172.21.15.49:0/2840947860 conn(0x7fd58c09c190 msgr2=0x7fd58c0b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:36.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.825+0000 7fd593bff700 1 -- 172.21.15.49:0/2840947860 shutdown_connections 2023-07-18T03:39:36.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:36.825+0000 7fd593bff700 1 -- 172.21.15.49:0/2840947860 wait complete. 2023-07-18T03:39:36.823 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2023-07-18T03:39:36.839 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:37.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.050+0000 7feb70bff700 1 Processor -- start 2023-07-18T03:39:37.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.050+0000 7feb70bff700 1 -- start start 2023-07-18T03:39:37.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.051+0000 7feb70bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb6c15c950 0x7feb6c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.051+0000 7feb70bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7feb6c0d1220 0x7feb6c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.052+0000 7feb70bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6c0d2ce0 con 0x7feb6c0b1e00 2023-07-18T03:39:37.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.052+0000 7feb70bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6c0b3350 con 0x7feb6c0d1220 2023-07-18T03:39:37.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.052+0000 7feb70bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c0b3630 con 0x7feb6c15c950 2023-07-18T03:39:37.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.052+0000 7feb63fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7feb6c0b1e00 0x7feb6c0d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57830/0 (socket says 172.21.15.49:57830) 2023-07-18T03:39:37.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.052+0000 7feb63fff700 1 -- 172.21.15.49:0/822961240 learned_addr learned my addr 172.21.15.49:0/822961240 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:37.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.052+0000 7feb6ad9d700 1 --2- 172.21.15.49:0/822961240 >> v2:172.21.15.107:3300/0 conn(0x7feb6c0d1220 0x7feb6c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.052+0000 7feb6a59c700 1 --2- 172.21.15.49:0/822961240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb6c15c950 0x7feb6c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.052+0000 7feb69d9b700 1 -- 172.21.15.49:0/822961240 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168387471 0 0) 0x7feb6c0d2ce0 con 0x7feb6c0b1e00 2023-07-18T03:39:37.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.052+0000 7feb69d9b700 1 -- 172.21.15.49:0/822961240 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb580035b0 con 0x7feb6c0b1e00 2023-07-18T03:39:37.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.053+0000 7feb6a59c700 1 -- 172.21.15.49:0/822961240 >> v2:172.21.15.107:3300/0 conn(0x7feb6c0d1220 msgr2=0x7feb6c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:37.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.053+0000 7feb6a59c700 1 --2- 172.21.15.49:0/822961240 >> v2:172.21.15.107:3300/0 conn(0x7feb6c0d1220 0x7feb6c0afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.053+0000 7feb6a59c700 1 -- 172.21.15.49:0/822961240 >> v1:172.21.15.49:6789/0 conn(0x7feb6c0b1e00 legacy=0x7feb6c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:37.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.053+0000 7feb6a59c700 1 -- 172.21.15.49:0/822961240 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6c0b3910 con 0x7feb6c15c950 2023-07-18T03:39:37.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.053+0000 7feb6ad9d700 1 --2- 172.21.15.49:0/822961240 >> v2:172.21.15.107:3300/0 conn(0x7feb6c0d1220 0x7feb6c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:37.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.053+0000 7feb6a59c700 1 --2- 172.21.15.49:0/822961240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb6c15c950 0x7feb6c0b43f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009820 tx=0x7feb5c009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=417cfd85e3de249 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:37.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.053+0000 7feb69d9b700 1 -- 172.21.15.49:0/822961240 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c005e30 con 0x7feb6c15c950 2023-07-18T03:39:37.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.053+0000 7feb69d9b700 1 -- 172.21.15.49:0/822961240 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c006030 con 0x7feb6c15c950 2023-07-18T03:39:37.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.053+0000 7feb69d9b700 1 -- 172.21.15.49:0/822961240 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0063c0 con 0x7feb6c15c950 2023-07-18T03:39:37.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.054+0000 7feb70bff700 1 -- 172.21.15.49:0/822961240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb6c15c950 msgr2=0x7feb6c0b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:37.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.054+0000 7feb70bff700 1 --2- 172.21.15.49:0/822961240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb6c15c950 0x7feb6c0b43f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009820 tx=0x7feb5c009b30 comp rx=0 tx=0).stop 2023-07-18T03:39:37.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.054+0000 7feb70bff700 1 -- 172.21.15.49:0/822961240 shutdown_connections 2023-07-18T03:39:37.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.054+0000 7feb70bff700 1 --2- 172.21.15.49:0/822961240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb6c15c950 0x7feb6c0b43f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.054+0000 7feb70bff700 1 --2- 172.21.15.49:0/822961240 >> v2:172.21.15.107:3300/0 conn(0x7feb6c0d1220 0x7feb6c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.054+0000 7feb70bff700 1 -- 172.21.15.49:0/822961240 >> 172.21.15.49:0/822961240 conn(0x7feb6c09c1d0 msgr2=0x7feb6c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:37.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.054+0000 7feb70bff700 1 -- 172.21.15.49:0/822961240 shutdown_connections 2023-07-18T03:39:37.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.054+0000 7feb70bff700 1 -- 172.21.15.49:0/822961240 wait complete. 2023-07-18T03:39:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.056+0000 7feb70bff700 1 Processor -- start 2023-07-18T03:39:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.056+0000 7feb70bff700 1 -- start start 2023-07-18T03:39:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb70bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb6c0b1e00 0x7feb6c157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb70bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb6c0d1220 0x7feb6c151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb70bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb6c15c950 0x7feb6c0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb70bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c1f6390 con 0x7feb6c0b1e00 2023-07-18T03:39:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb70bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c151760 con 0x7feb6c0d1220 2023-07-18T03:39:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb70bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c1f54a0 con 0x7feb6c15c950 2023-07-18T03:39:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb6a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb6c0b1e00 0x7feb6c157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb6a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb6c0b1e00 0x7feb6c157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53802/0 (socket says 172.21.15.49:53802) 2023-07-18T03:39:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb6a59c700 1 -- 172.21.15.49:0/1272524770 learned_addr learned my addr 172.21.15.49:0/1272524770 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb63fff700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb6c0d1220 0x7feb6c151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb6ad9d700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb6c15c950 0x7feb6c0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb6a59c700 1 -- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb6c15c950 msgr2=0x7feb6c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb6a59c700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb6c15c950 0x7feb6c0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb6a59c700 1 -- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb6c0d1220 msgr2=0x7feb6c151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.057+0000 7feb6a59c700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb6c0d1220 0x7feb6c151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.058+0000 7feb6a59c700 1 -- 172.21.15.49:0/1272524770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb64009a70 con 0x7feb6c0b1e00 2023-07-18T03:39:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.058+0000 7feb63fff700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb6c0d1220 0x7feb6c151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.058+0000 7feb6a59c700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb6c0b1e00 0x7feb6c157910 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb5c000c00 tx=0x7feb5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.058+0000 7feb62ffd700 1 -- 172.21.15.49:0/1272524770 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0063a0 con 0x7feb6c0b1e00 2023-07-18T03:39:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.058+0000 7feb62ffd700 1 -- 172.21.15.49:0/1272524770 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c020070 con 0x7feb6c0b1e00 2023-07-18T03:39:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.058+0000 7feb70bff700 1 -- 172.21.15.49:0/1272524770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5c009430 con 0x7feb6c0b1e00 2023-07-18T03:39:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.058+0000 7feb62ffd700 1 -- 172.21.15.49:0/1272524770 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c006760 con 0x7feb6c0b1e00 2023-07-18T03:39:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.058+0000 7feb70bff700 1 -- 172.21.15.49:0/1272524770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6c1f5a60 con 0x7feb6c0b1e00 2023-07-18T03:39:37.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.059+0000 7feb70bff700 1 -- 172.21.15.49:0/1272524770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6c04e280 con 0x7feb6c0b1e00 2023-07-18T03:39:37.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.060+0000 7feb62ffd700 1 -- 172.21.15.49:0/1272524770 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb5c022030 con 0x7feb6c0b1e00 2023-07-18T03:39:37.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.061+0000 7feb62ffd700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7feb4c06ace0 0x7feb4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.061+0000 7feb62ffd700 1 -- 172.21.15.49:0/1272524770 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7feb5c08ab00 con 0x7feb6c0b1e00 2023-07-18T03:39:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.062+0000 7feb63fff700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7feb4c06ace0 0x7feb4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.062+0000 7feb63fff700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7feb4c06ace0 0x7feb4c06d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0b8e90 tx=0x7feb54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.065+0000 7feb62ffd700 1 -- 172.21.15.49:0/1272524770 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7feb5c08ce60 con 0x7feb6c0b1e00 2023-07-18T03:39:37.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.274+0000 7feb70bff700 1 -- 172.21.15.49:0/1272524770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb6c0b3090 con 0x7feb6c0b1e00 2023-07-18T03:39:37.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.276+0000 7feb62ffd700 1 -- 172.21.15.49:0/1272524770 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18280 (secure 0 0 0) 0x7feb5c055c80 con 0x7feb6c0b1e00 2023-07-18T03:39:37.271 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:37.272 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":24,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:23.937174+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:38:33.773236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-18T03:38:35.023692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:37.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.278+0000 7feb70bff700 1 -- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7feb4c06ace0 msgr2=0x7feb4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:37.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.278+0000 7feb70bff700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7feb4c06ace0 0x7feb4c06d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0b8e90 tx=0x7feb54009040 comp rx=0 tx=0).stop 2023-07-18T03:39:37.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.278+0000 7feb70bff700 1 -- 172.21.15.49:0/1272524770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb6c0b1e00 msgr2=0x7feb6c157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:37.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.278+0000 7feb70bff700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb6c0b1e00 0x7feb6c157910 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb5c000c00 tx=0x7feb5c004750 comp rx=0 tx=0).stop 2023-07-18T03:39:37.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.278+0000 7feb70bff700 1 -- 172.21.15.49:0/1272524770 shutdown_connections 2023-07-18T03:39:37.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.278+0000 7feb70bff700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb6c15c950 0x7feb6c0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.278+0000 7feb70bff700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7feb4c06ace0 0x7feb4c06d190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.278+0000 7feb70bff700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb6c0b1e00 0x7feb6c157910 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.278+0000 7feb70bff700 1 --2- 172.21.15.49:0/1272524770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb6c0d1220 0x7feb6c151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.278+0000 7feb70bff700 1 -- 172.21.15.49:0/1272524770 >> 172.21.15.49:0/1272524770 conn(0x7feb6c09c1d0 msgr2=0x7feb6c0afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:37.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.278+0000 7feb70bff700 1 -- 172.21.15.49:0/1272524770 shutdown_connections 2023-07-18T03:39:37.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.278+0000 7feb70bff700 1 -- 172.21.15.49:0/1272524770 wait complete. 2023-07-18T03:39:37.292 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-18T03:39:37.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.502+0000 7f68e9fff700 1 Processor -- start 2023-07-18T03:39:37.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.502+0000 7f68e9fff700 1 -- start start 2023-07-18T03:39:37.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.503+0000 7f68e9fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68e40b8a60 0x7f68e40b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.503+0000 7f68e9fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e40b4530 0x7f68e40d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.503+0000 7f68e9fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f68e40d1050 0x7f68e40b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.503+0000 7f68e9fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e40d14d0 con 0x7f68e40b8a60 2023-07-18T03:39:37.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.503+0000 7f68e9fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e40b8e30 con 0x7f68e40d1050 2023-07-18T03:39:37.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.503+0000 7f68e9fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40afaa0 con 0x7f68e40b4530 2023-07-18T03:39:37.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.504+0000 7f68e37fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68e40b8a60 0x7f68e40b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.504+0000 7f68e37fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68e40b8a60 0x7f68e40b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53814/0 (socket says 172.21.15.49:53814) 2023-07-18T03:39:37.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.504+0000 7f68e37fe700 1 -- 172.21.15.49:0/761600447 learned_addr learned my addr 172.21.15.49:0/761600447 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:37.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.504+0000 7f68e2ffd700 1 --2- 172.21.15.49:0/761600447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e40b4530 0x7f68e40d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.504+0000 7f68e3fff700 1 --2- 172.21.15.49:0/761600447 >> v2:172.21.15.107:3300/0 conn(0x7f68e40d1050 0x7f68e40b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.504+0000 7f68e37fe700 1 -- 172.21.15.49:0/761600447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e40b4530 msgr2=0x7f68e40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.504+0000 7f68e37fe700 1 --2- 172.21.15.49:0/761600447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e40b4530 0x7f68e40d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.504+0000 7f68e37fe700 1 -- 172.21.15.49:0/761600447 >> v2:172.21.15.107:3300/0 conn(0x7f68e40d1050 msgr2=0x7f68e40b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.504+0000 7f68e37fe700 1 --2- 172.21.15.49:0/761600447 >> v2:172.21.15.107:3300/0 conn(0x7f68e40d1050 0x7f68e40b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.504+0000 7f68e37fe700 1 -- 172.21.15.49:0/761600447 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e40afd80 con 0x7f68e40b8a60 2023-07-18T03:39:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.504+0000 7f68e37fe700 1 --2- 172.21.15.49:0/761600447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e40b8a60 0x7f68e40b3ff0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f68e408c2e0 tx=0x7f68d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b72de5ab705284 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.505+0000 7f68e27fc700 1 -- 172.21.15.49:0/761600447 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4011830 con 0x7f68e40b8a60 2023-07-18T03:39:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.505+0000 7f68e27fc700 1 -- 172.21.15.49:0/761600447 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d4011990 con 0x7f68e40b8a60 2023-07-18T03:39:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.505+0000 7f68e27fc700 1 -- 172.21.15.49:0/761600447 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4011d20 con 0x7f68e40b8a60 2023-07-18T03:39:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.505+0000 7f68e9fff700 1 -- 172.21.15.49:0/761600447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e40b8a60 msgr2=0x7f68e40b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.505+0000 7f68e9fff700 1 --2- 172.21.15.49:0/761600447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e40b8a60 0x7f68e40b3ff0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f68e408c2e0 tx=0x7f68d4009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.505+0000 7f68e9fff700 1 -- 172.21.15.49:0/761600447 shutdown_connections 2023-07-18T03:39:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.505+0000 7f68e9fff700 1 --2- 172.21.15.49:0/761600447 >> v2:172.21.15.107:3300/0 conn(0x7f68e40d1050 0x7f68e40b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.505+0000 7f68e9fff700 1 --2- 172.21.15.49:0/761600447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e40b8a60 0x7f68e40b3ff0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.505+0000 7f68e9fff700 1 --2- 172.21.15.49:0/761600447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e40b4530 0x7f68e40d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.505+0000 7f68e9fff700 1 -- 172.21.15.49:0/761600447 >> 172.21.15.49:0/761600447 conn(0x7f68e409c1b0 msgr2=0x7f68e40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:37.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.505+0000 7f68e9fff700 1 -- 172.21.15.49:0/761600447 shutdown_connections 2023-07-18T03:39:37.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.505+0000 7f68e9fff700 1 -- 172.21.15.49:0/761600447 wait complete. 2023-07-18T03:39:37.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.507+0000 7f68e9fff700 1 Processor -- start 2023-07-18T03:39:37.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.507+0000 7f68e9fff700 1 -- start start 2023-07-18T03:39:37.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.508+0000 7f68e9fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e40b4530 0x7f68e4151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.508+0000 7f68e9fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e40b8a60 0x7f68e41f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.508+0000 7f68e9fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e40d1050 0x7f68e41f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.508+0000 7f68e9fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e415cdb0 con 0x7f68e40b4530 2023-07-18T03:39:37.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.508+0000 7f68e9fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e415d090 con 0x7f68e40b8a60 2023-07-18T03:39:37.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.508+0000 7f68e9fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4152290 con 0x7f68e40d1050 2023-07-18T03:39:37.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.508+0000 7f68e37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e40b4530 0x7f68e4151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.508+0000 7f68e37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e40b4530 0x7f68e4151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53828/0 (socket says 172.21.15.49:53828) 2023-07-18T03:39:37.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.508+0000 7f68e37fe700 1 -- 172.21.15.49:0/1506077278 learned_addr learned my addr 172.21.15.49:0/1506077278 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:37.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68e2ffd700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e40b8a60 0x7f68e41f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68e3fff700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e40d1050 0x7f68e41f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68e37fe700 1 -- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e40d1050 msgr2=0x7f68e41f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68e37fe700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e40d1050 0x7f68e41f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68e37fe700 1 -- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e40b8a60 msgr2=0x7f68e41f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68e37fe700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e40b8a60 0x7f68e41f5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68e37fe700 1 -- 172.21.15.49:0/1506077278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d4009550 con 0x7f68e40b4530 2023-07-18T03:39:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68e3fff700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e40d1050 0x7f68e41f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68e37fe700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e40b4530 0x7f68e4151d50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f68e40681d0 tx=0x7f68d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68cbfff700 1 -- 172.21.15.49:0/1506077278 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4004240 con 0x7f68e40b4530 2023-07-18T03:39:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68cbfff700 1 -- 172.21.15.49:0/1506077278 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d401f070 con 0x7f68e40b4530 2023-07-18T03:39:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68cbfff700 1 -- 172.21.15.49:0/1506077278 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d40045e0 con 0x7f68e40b4530 2023-07-18T03:39:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.509+0000 7f68e9fff700 1 -- 172.21.15.49:0/1506077278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e415f6e0 con 0x7f68e40b4530 2023-07-18T03:39:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.510+0000 7f68e9fff700 1 -- 172.21.15.49:0/1506077278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e415fca0 con 0x7f68e40b4530 2023-07-18T03:39:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.511+0000 7f68e9fff700 1 -- 172.21.15.49:0/1506077278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e4074350 con 0x7f68e40b4530 2023-07-18T03:39:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.512+0000 7f68cbfff700 1 -- 172.21.15.49:0/1506077278 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f68d4021030 con 0x7f68e40b4530 2023-07-18T03:39:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.512+0000 7f68cbfff700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f68d006ace0 0x7f68d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.513+0000 7f68cbfff700 1 -- 172.21.15.49:0/1506077278 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f68d408a680 con 0x7f68e40b4530 2023-07-18T03:39:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.513+0000 7f68e2ffd700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f68d006ace0 0x7f68d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.514+0000 7f68e2ffd700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f68d006ace0 0x7f68d006d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f68e408d080 tx=0x7f68d8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.517+0000 7f68cbfff700 1 -- 172.21.15.49:0/1506077278 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f68d40556e0 con 0x7f68e40b4530 2023-07-18T03:39:37.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.767+0000 7f68e9fff700 1 -- 172.21.15.49:0/1506077278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f68e40b08f0 con 0x7f68e40b4530 2023-07-18T03:39:37.763 INFO:tasks.ceph.mon.a.smithi049.stderr:2023-07-18T03:39:37.768+0000 7ff4cc5f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-18T03:39:37.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.772+0000 7f68cbfff700 1 -- 172.21.15.49:0/1506077278 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f68d4058d00 con 0x7f68e40b4530 2023-07-18T03:39:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.775+0000 7f68e9fff700 1 -- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f68d006ace0 msgr2=0x7f68d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.775+0000 7f68e9fff700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f68d006ace0 0x7f68d006d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f68e408d080 tx=0x7f68d8009210 comp rx=0 tx=0).stop 2023-07-18T03:39:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.775+0000 7f68e9fff700 1 -- 172.21.15.49:0/1506077278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e40b4530 msgr2=0x7f68e4151d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.775+0000 7f68e9fff700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e40b4530 0x7f68e4151d50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f68e40681d0 tx=0x7f68d4004750 comp rx=0 tx=0).stop 2023-07-18T03:39:37.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.775+0000 7f68e9fff700 1 -- 172.21.15.49:0/1506077278 shutdown_connections 2023-07-18T03:39:37.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.775+0000 7f68e9fff700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e40d1050 0x7f68e41f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.775+0000 7f68e9fff700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f68d006ace0 0x7f68d006d190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.775+0000 7f68e9fff700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e40b4530 0x7f68e4151d50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.775+0000 7f68e9fff700 1 --2- 172.21.15.49:0/1506077278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e40b8a60 0x7f68e41f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:37.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.775+0000 7f68e9fff700 1 -- 172.21.15.49:0/1506077278 >> 172.21.15.49:0/1506077278 conn(0x7f68e409c1b0 msgr2=0x7f68e406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:37.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.776+0000 7f68e9fff700 1 -- 172.21.15.49:0/1506077278 shutdown_connections 2023-07-18T03:39:37.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:37.776+0000 7f68e9fff700 1 -- 172.21.15.49:0/1506077278 wait complete. 2023-07-18T03:39:37.773 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-18T03:39:37.779 INFO:tasks.ceph.mds.d.smithi107.stderr: -6284> 2023-07-18T03:38:40.549+0000 7f98bddf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-18T03:39:37.789 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-18T03:39:37.836 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2023-07-18T03:39:37.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.002+0000 7f3bcb7ff700 1 Processor -- start 2023-07-18T03:39:37.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.002+0000 7f3bcb7ff700 1 -- start start 2023-07-18T03:39:37.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.003+0000 7f3bcb7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3bc40b8a60 0x7f3bc40b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.003+0000 7f3bcb7ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3bc40b4530 0x7f3bc40d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:37.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.004+0000 7f3bcb7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc40d1490 con 0x7f3bc40d1050 2023-07-18T03:39:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.004+0000 7f3bcb7ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc40b8e30 con 0x7f3bc40b4530 2023-07-18T03:39:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.004+0000 7f3bcb7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc40b1e30 con 0x7f3bc40b8a60 2023-07-18T03:39:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.004+0000 7f3bc8d9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3bc40b4530 0x7f3bc40d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.004+0000 7f3bc8d9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3bc40b4530 0x7f3bc40d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53832/0 (socket says 172.21.15.49:53832) 2023-07-18T03:39:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.004+0000 7f3bc8d9a700 1 -- 172.21.15.49:0/190322685 learned_addr learned my addr 172.21.15.49:0/190322685 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:38.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.004+0000 7f3bc959b700 1 --2- 172.21.15.49:0/190322685 >> v2:172.21.15.107:3300/0 conn(0x7f3bc40b8a60 0x7f3bc40b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:38.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.004+0000 7f3bc8d9a700 1 -- 172.21.15.49:0/190322685 >> v2:172.21.15.107:3300/0 conn(0x7f3bc40b8a60 msgr2=0x7f3bc40b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:38.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.004+0000 7f3bc8d9a700 1 --2- 172.21.15.49:0/190322685 >> v2:172.21.15.107:3300/0 conn(0x7f3bc40b8a60 0x7f3bc40b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:38.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.004+0000 7f3bc8d9a700 1 -- 172.21.15.49:0/190322685 >> v1:172.21.15.107:6789/0 conn(0x7f3bc40d1050 legacy=0x7f3bc40b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:38.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.004+0000 7f3bc8d9a700 1 -- 172.21.15.49:0/190322685 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc40afaa0 con 0x7f3bc40b4530 2023-07-18T03:39:38.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.005+0000 7f3bc8d9a700 1 --2- 172.21.15.49:0/190322685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc40b4530 0x7f3bc40d0b10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40b5b00 tx=0x7f3bb4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883e4dedc3e56672 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:38.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.005+0000 7f3bbbfff700 1 -- 172.21.15.49:0/190322685 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb400b870 con 0x7f3bc40b4530 2023-07-18T03:39:38.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.005+0000 7f3bbbfff700 1 -- 172.21.15.49:0/190322685 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb400b9d0 con 0x7f3bc40b4530 2023-07-18T03:39:38.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.005+0000 7f3bbbfff700 1 -- 172.21.15.49:0/190322685 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb400bd60 con 0x7f3bc40b4530 2023-07-18T03:39:38.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.005+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/190322685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc40b4530 msgr2=0x7f3bc40d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:38.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.005+0000 7f3bcb7ff700 1 --2- 172.21.15.49:0/190322685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc40b4530 0x7f3bc40d0b10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40b5b00 tx=0x7f3bb4009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:38.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.006+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/190322685 shutdown_connections 2023-07-18T03:39:38.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.006+0000 7f3bcb7ff700 1 --2- 172.21.15.49:0/190322685 >> v2:172.21.15.107:3300/0 conn(0x7f3bc40b8a60 0x7f3bc40b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:38.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.006+0000 7f3bcb7ff700 1 --2- 172.21.15.49:0/190322685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc40b4530 0x7f3bc40d0b10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:38.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.006+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/190322685 >> 172.21.15.49:0/190322685 conn(0x7f3bc409c1b0 msgr2=0x7f3bc40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:38.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.006+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/190322685 shutdown_connections 2023-07-18T03:39:38.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.006+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/190322685 wait complete. 2023-07-18T03:39:38.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.007+0000 7f3bcb7ff700 1 Processor -- start 2023-07-18T03:39:38.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.008+0000 7f3bcb7ff700 1 -- start start 2023-07-18T03:39:38.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.008+0000 7f3bcb7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bc40b4530 0x7f3bc4156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:38.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.008+0000 7f3bcb7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc40b8a60 0x7f3bc415f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:38.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.008+0000 7f3bcb7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bc40d1050 0x7f3bc406b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:38.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.008+0000 7f3bcb7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40cb840 con 0x7f3bc40b8a60 2023-07-18T03:39:38.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.008+0000 7f3bcb7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4156ac0 con 0x7f3bc40d1050 2023-07-18T03:39:38.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.008+0000 7f3bcb7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc415dba0 con 0x7f3bc40b4530 2023-07-18T03:39:38.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.009+0000 7f3bc8d9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc40b8a60 0x7f3bc415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:38.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.009+0000 7f3bc8d9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc40b8a60 0x7f3bc415f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53840/0 (socket says 172.21.15.49:53840) 2023-07-18T03:39:38.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.009+0000 7f3bc8d9a700 1 -- 172.21.15.49:0/2452371151 learned_addr learned my addr 172.21.15.49:0/2452371151 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:38.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.009+0000 7f3bc959b700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bc40b4530 0x7f3bc4156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:38.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.009+0000 7f3bc8d9a700 1 -- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bc40b4530 msgr2=0x7f3bc4156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:38.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.009+0000 7f3bc9d9c700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bc40d1050 0x7f3bc406b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:38.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.009+0000 7f3bc8d9a700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bc40b4530 0x7f3bc4156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:38.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.009+0000 7f3bc8d9a700 1 -- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bc40d1050 msgr2=0x7f3bc406b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:38.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.009+0000 7f3bc8d9a700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bc40d1050 0x7f3bc406b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:38.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.009+0000 7f3bc8d9a700 1 -- 172.21.15.49:0/2452371151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb4009550 con 0x7f3bc40b8a60 2023-07-18T03:39:38.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.009+0000 7f3bc959b700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bc40b4530 0x7f3bc4156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:38.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.009+0000 7f3bc9d9c700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bc40d1050 0x7f3bc406b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:38.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.010+0000 7f3bc8d9a700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc40b8a60 0x7f3bc415f560 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4000c00 tx=0x7f3bb4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:38.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.010+0000 7f3bb9ffb700 1 -- 172.21.15.49:0/2452371151 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4004540 con 0x7f3bc40b8a60 2023-07-18T03:39:38.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.010+0000 7f3bb9ffb700 1 -- 172.21.15.49:0/2452371151 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb40036a0 con 0x7f3bc40b8a60 2023-07-18T03:39:38.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.010+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/2452371151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc415de80 con 0x7f3bc40b8a60 2023-07-18T03:39:38.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.010+0000 7f3bb9ffb700 1 -- 172.21.15.49:0/2452371151 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4003a30 con 0x7f3bc40b8a60 2023-07-18T03:39:38.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.010+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/2452371151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc415e440 con 0x7f3bc40b8a60 2023-07-18T03:39:38.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.011+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/2452371151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc40cb980 con 0x7f3bc40b8a60 2023-07-18T03:39:38.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.012+0000 7f3bb9ffb700 1 -- 172.21.15.49:0/2452371151 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3bb4021070 con 0x7f3bc40b8a60 2023-07-18T03:39:38.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.013+0000 7f3bb9ffb700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3ba806ad10 0x7f3ba806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:38.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.014+0000 7f3bb9ffb700 1 -- 172.21.15.49:0/2452371151 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f3bb408a9e0 con 0x7f3bc40b8a60 2023-07-18T03:39:38.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.014+0000 7f3bc959b700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3ba806ad10 0x7f3ba806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:38.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.015+0000 7f3bc959b700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3ba806ad10 0x7f3ba806d1c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0009a00 tx=0x7f3bc0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:38.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.018+0000 7f3bb9ffb700 1 -- 172.21.15.49:0/2452371151 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3bc40cb980 con 0x7f3bc40b8a60 2023-07-18T03:39:38.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.273+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/2452371151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3bc40cb980 con 0x7f3bc40b8a60 2023-07-18T03:39:38.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.774+0000 7f3bb9ffb700 1 -- 172.21.15.49:0/2452371151 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3bc40cb980 con 0x7f3bc40b8a60 2023-07-18T03:39:38.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.776+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3ba806ad10 msgr2=0x7f3ba806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:38.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.776+0000 7f3bcb7ff700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3ba806ad10 0x7f3ba806d1c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0009a00 tx=0x7f3bc0008040 comp rx=0 tx=0).stop 2023-07-18T03:39:38.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.776+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/2452371151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc40b8a60 msgr2=0x7f3bc415f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:38.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.776+0000 7f3bcb7ff700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc40b8a60 0x7f3bc415f560 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4000c00 tx=0x7f3bb4004750 comp rx=0 tx=0).stop 2023-07-18T03:39:38.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.777+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/2452371151 shutdown_connections 2023-07-18T03:39:38.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.777+0000 7f3bcb7ff700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bc40d1050 0x7f3bc406b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:38.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.777+0000 7f3bcb7ff700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bc40b4530 0x7f3bc4156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:38.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.777+0000 7f3bcb7ff700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3ba806ad10 0x7f3ba806d1c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:38.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.777+0000 7f3bcb7ff700 1 --2- 172.21.15.49:0/2452371151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bc40b8a60 0x7f3bc415f560 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:38.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.777+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/2452371151 >> 172.21.15.49:0/2452371151 conn(0x7f3bc409c1b0 msgr2=0x7f3bc415c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:38.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.777+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/2452371151 shutdown_connections 2023-07-18T03:39:38.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:38.777+0000 7f3bcb7ff700 1 -- 172.21.15.49:0/2452371151 wait complete. 2023-07-18T03:39:38.792 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-18T03:39:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.010+0000 7fad307ff700 1 Processor -- start 2023-07-18T03:39:39.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.010+0000 7fad307ff700 1 -- start start 2023-07-18T03:39:39.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.011+0000 7fad307ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad2815c7b0 0x7fad280d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:39.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.011+0000 7fad307ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fad280b1e50 0x7fad280d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:39.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.011+0000 7fad307ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fad280d0f70 0x7fad28160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:39.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.011+0000 7fad307ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280d3de0 con 0x7fad280b1e50 2023-07-18T03:39:39.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.011+0000 7fad307ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280d13f0 con 0x7fad280d0f70 2023-07-18T03:39:39.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.011+0000 7fad307ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280d0190 con 0x7fad2815c7b0 2023-07-18T03:39:39.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.011+0000 7fad2dd9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fad280b1e50 0x7fad280d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:39.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.011+0000 7fad2dd9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fad280b1e50 0x7fad280d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37098/0 (socket says 172.21.15.49:37098) 2023-07-18T03:39:39.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.011+0000 7fad2dd9a700 1 -- 172.21.15.49:0/2434246142 learned_addr learned my addr 172.21.15.49:0/2434246142 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:39.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.011+0000 7fad2ed9c700 1 --2- 172.21.15.49:0/2434246142 >> v2:172.21.15.107:3300/0 conn(0x7fad280d0f70 0x7fad28160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:39.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.011+0000 7fad2e59b700 1 --2- 172.21.15.49:0/2434246142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad2815c7b0 0x7fad280d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:39.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.012+0000 7fad2dd9a700 1 -- 172.21.15.49:0/2434246142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad2815c7b0 msgr2=0x7fad280d28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:39.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.012+0000 7fad2dd9a700 1 --2- 172.21.15.49:0/2434246142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad2815c7b0 0x7fad280d28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:39.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.012+0000 7fad2dd9a700 1 -- 172.21.15.49:0/2434246142 >> v2:172.21.15.107:3300/0 conn(0x7fad280d0f70 msgr2=0x7fad28160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:39.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.012+0000 7fad2dd9a700 1 --2- 172.21.15.49:0/2434246142 >> v2:172.21.15.107:3300/0 conn(0x7fad280d0f70 0x7fad28160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:39.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.012+0000 7fad2dd9a700 1 -- 172.21.15.49:0/2434246142 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280d0470 con 0x7fad280b1e50 2023-07-18T03:39:39.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.012+0000 7fad2dd9a700 1 --2- 172.21.15.49:0/2434246142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad280b1e50 0x7fad280d0a30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fad2808d670 tx=0x7fad18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d125defed96be3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:39.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.012+0000 7fad2d599700 1 -- 172.21.15.49:0/2434246142 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1800b870 con 0x7fad280b1e50 2023-07-18T03:39:39.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.012+0000 7fad2d599700 1 -- 172.21.15.49:0/2434246142 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1800b9d0 con 0x7fad280b1e50 2023-07-18T03:39:39.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.012+0000 7fad2d599700 1 -- 172.21.15.49:0/2434246142 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1800bd60 con 0x7fad280b1e50 2023-07-18T03:39:39.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.013+0000 7fad307ff700 1 -- 172.21.15.49:0/2434246142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad280b1e50 msgr2=0x7fad280d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.013+0000 7fad307ff700 1 --2- 172.21.15.49:0/2434246142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad280b1e50 0x7fad280d0a30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fad2808d670 tx=0x7fad18009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.013+0000 7fad307ff700 1 -- 172.21.15.49:0/2434246142 shutdown_connections 2023-07-18T03:39:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.013+0000 7fad307ff700 1 --2- 172.21.15.49:0/2434246142 >> v2:172.21.15.107:3300/0 conn(0x7fad280d0f70 0x7fad28160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.013+0000 7fad307ff700 1 --2- 172.21.15.49:0/2434246142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad2815c7b0 0x7fad280d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.013+0000 7fad307ff700 1 --2- 172.21.15.49:0/2434246142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad280b1e50 0x7fad280d0a30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.013+0000 7fad307ff700 1 -- 172.21.15.49:0/2434246142 >> 172.21.15.49:0/2434246142 conn(0x7fad2809c1b0 msgr2=0x7fad280d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.013+0000 7fad307ff700 1 -- 172.21.15.49:0/2434246142 shutdown_connections 2023-07-18T03:39:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.013+0000 7fad307ff700 1 -- 172.21.15.49:0/2434246142 wait complete. 2023-07-18T03:39:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.015+0000 7fad307ff700 1 Processor -- start 2023-07-18T03:39:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.015+0000 7fad307ff700 1 -- start start 2023-07-18T03:39:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.016+0000 7fad307ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad280b1e50 0x7fad2815adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.016+0000 7fad307ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad280d0f70 0x7fad2815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.016+0000 7fad307ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad2815c7b0 0x7fad281f1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.016+0000 7fad307ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad280b72b0 con 0x7fad2815c7b0 2023-07-18T03:39:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.016+0000 7fad307ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad280cb5d0 con 0x7fad280d0f70 2023-07-18T03:39:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.016+0000 7fad307ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280cb8b0 con 0x7fad280b1e50 2023-07-18T03:39:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.016+0000 7fad2ed9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad2815c7b0 0x7fad281f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.016+0000 7fad2ed9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad2815c7b0 0x7fad281f1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37112/0 (socket says 172.21.15.49:37112) 2023-07-18T03:39:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.016+0000 7fad2ed9c700 1 -- 172.21.15.49:0/1824184494 learned_addr learned my addr 172.21.15.49:0/1824184494 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.016+0000 7fad2e59b700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad280b1e50 0x7fad2815adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.016+0000 7fad2dd9a700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad280d0f70 0x7fad2815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.017+0000 7fad2ed9c700 1 -- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad280b1e50 msgr2=0x7fad2815adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.017+0000 7fad2ed9c700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad280b1e50 0x7fad2815adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.017+0000 7fad2ed9c700 1 -- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad280d0f70 msgr2=0x7fad2815b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.017+0000 7fad2ed9c700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad280d0f70 0x7fad2815b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.017+0000 7fad2ed9c700 1 -- 172.21.15.49:0/1824184494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad18009550 con 0x7fad2815c7b0 2023-07-18T03:39:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.017+0000 7fad2ed9c700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad2815c7b0 0x7fad281f1780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fad20007e60 tx=0x7fad2000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.017+0000 7fad1effd700 1 -- 172.21.15.49:0/1824184494 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2000ed30 con 0x7fad2815c7b0 2023-07-18T03:39:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.017+0000 7fad1effd700 1 -- 172.21.15.49:0/1824184494 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad20014420 con 0x7fad2815c7b0 2023-07-18T03:39:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.017+0000 7fad1effd700 1 -- 172.21.15.49:0/1824184494 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad200147d0 con 0x7fad2815c7b0 2023-07-18T03:39:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.017+0000 7fad307ff700 1 -- 172.21.15.49:0/1824184494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad280cbb90 con 0x7fad2815c7b0 2023-07-18T03:39:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.017+0000 7fad307ff700 1 -- 172.21.15.49:0/1824184494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad281f1ea0 con 0x7fad2815c7b0 2023-07-18T03:39:39.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.018+0000 7fad307ff700 1 -- 172.21.15.49:0/1824184494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad280cbd80 con 0x7fad2815c7b0 2023-07-18T03:39:39.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.020+0000 7fad1effd700 1 -- 172.21.15.49:0/1824184494 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad20014930 con 0x7fad2815c7b0 2023-07-18T03:39:39.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.021+0000 7fad1effd700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad0c06ace0 0x7fad0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.021+0000 7fad1effd700 1 -- 172.21.15.49:0/1824184494 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fad20089bf0 con 0x7fad2815c7b0 2023-07-18T03:39:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.021+0000 7fad2e59b700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad0c06ace0 0x7fad0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:39.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.022+0000 7fad2e59b700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad0c06ace0 0x7fad0c06d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad24006b20 tx=0x7fad24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:39.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.026+0000 7fad1effd700 1 -- 172.21.15.49:0/1824184494 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fad20054b70 con 0x7fad2815c7b0 2023-07-18T03:39:39.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:39.232+0000 7fad307ff700 1 -- 172.21.15.49:0/1824184494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad28074350 con 0x7fad2815c7b0 2023-07-18T03:39:40.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.006+0000 7fad1effd700 1 -- 172.21.15.49:0/1824184494 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7fad20058190 con 0x7fad2815c7b0 2023-07-18T03:39:40.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.011+0000 7fad307ff700 1 -- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad0c06ace0 msgr2=0x7fad0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:40.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.011+0000 7fad307ff700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad0c06ace0 0x7fad0c06d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad24006b20 tx=0x7fad24008040 comp rx=0 tx=0).stop 2023-07-18T03:39:40.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.011+0000 7fad307ff700 1 -- 172.21.15.49:0/1824184494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad2815c7b0 msgr2=0x7fad281f1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:40.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.011+0000 7fad307ff700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad2815c7b0 0x7fad281f1780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fad20007e60 tx=0x7fad2000e3c0 comp rx=0 tx=0).stop 2023-07-18T03:39:40.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.011+0000 7fad307ff700 1 -- 172.21.15.49:0/1824184494 shutdown_connections 2023-07-18T03:39:40.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.011+0000 7fad307ff700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fad0c06ace0 0x7fad0c06d190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:40.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.011+0000 7fad307ff700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad2815c7b0 0x7fad281f1780 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:40.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.011+0000 7fad307ff700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad280b1e50 0x7fad2815adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:40.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.011+0000 7fad307ff700 1 --2- 172.21.15.49:0/1824184494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad280d0f70 0x7fad2815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:40.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.011+0000 7fad307ff700 1 -- 172.21.15.49:0/1824184494 >> 172.21.15.49:0/1824184494 conn(0x7fad2809c1b0 msgr2=0x7fad280afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:40.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.011+0000 7fad307ff700 1 -- 172.21.15.49:0/1824184494 shutdown_connections 2023-07-18T03:39:40.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.011+0000 7fad307ff700 1 -- 172.21.15.49:0/1824184494 wait complete. 2023-07-18T03:39:40.010 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2023-07-18T03:39:40.031 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-18T03:39:40.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.248+0000 7fcffa3ff700 1 Processor -- start 2023-07-18T03:39:40.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.248+0000 7fcffa3ff700 1 -- start start 2023-07-18T03:39:40.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.249+0000 7fcffa3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcff415c8f0 0x7fcff40d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:40.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.249+0000 7fcffa3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcff40b1e10 0x7fcff40d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:40.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.249+0000 7fcffa3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff40af9b0 con 0x7fcff40b2740 2023-07-18T03:39:40.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.249+0000 7fcffa3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff40b45b0 con 0x7fcff40b1e10 2023-07-18T03:39:40.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.249+0000 7fcffa3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40afc50 con 0x7fcff415c8f0 2023-07-18T03:39:40.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.250+0000 7fcff3fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcff415c8f0 0x7fcff40d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:40.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.250+0000 7fcff37fe700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcff40b1e10 0x7fcff40d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:40.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.250+0000 7fcff899c700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fcff40b2740 0x7fcff40b2bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:52238/0 (socket says 172.21.15.49:52238) 2023-07-18T03:39:40.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.250+0000 7fcff899c700 1 -- 172.21.15.49:0/552389641 learned_addr learned my addr 172.21.15.49:0/552389641 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:40.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.250+0000 7fcff3fff700 1 -- 172.21.15.49:0/552389641 >> v2:172.21.15.107:3300/0 conn(0x7fcff40b1e10 msgr2=0x7fcff40d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:40.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.250+0000 7fcff3fff700 1 --2- 172.21.15.49:0/552389641 >> v2:172.21.15.107:3300/0 conn(0x7fcff40b1e10 0x7fcff40d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:40.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.250+0000 7fcff3fff700 1 -- 172.21.15.49:0/552389641 >> v1:172.21.15.107:6789/0 conn(0x7fcff40b2740 legacy=0x7fcff40b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:40.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.250+0000 7fcff3fff700 1 -- 172.21.15.49:0/552389641 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff40aff30 con 0x7fcff415c8f0 2023-07-18T03:39:40.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.250+0000 7fcff37fe700 1 --2- 172.21.15.49:0/552389641 >> v2:172.21.15.107:3300/0 conn(0x7fcff40b1e10 0x7fcff40d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:40.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.250+0000 7fcff3fff700 1 --2- 172.21.15.49:0/552389641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcff415c8f0 0x7fcff40d09b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009870 tx=0x7fcfe4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=421945e30a12a704 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:40.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.251+0000 7fcff2ffd700 1 -- 172.21.15.49:0/552389641 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4011830 con 0x7fcff415c8f0 2023-07-18T03:39:40.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.251+0000 7fcff2ffd700 1 -- 172.21.15.49:0/552389641 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe4011990 con 0x7fcff415c8f0 2023-07-18T03:39:40.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.251+0000 7fcff2ffd700 1 -- 172.21.15.49:0/552389641 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4011d20 con 0x7fcff415c8f0 2023-07-18T03:39:40.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.251+0000 7fcffa3ff700 1 -- 172.21.15.49:0/552389641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcff415c8f0 msgr2=0x7fcff40d09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:40.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.251+0000 7fcffa3ff700 1 --2- 172.21.15.49:0/552389641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcff415c8f0 0x7fcff40d09b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009870 tx=0x7fcfe4009b80 comp rx=0 tx=0).stop 2023-07-18T03:39:40.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.251+0000 7fcffa3ff700 1 -- 172.21.15.49:0/552389641 shutdown_connections 2023-07-18T03:39:40.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.251+0000 7fcffa3ff700 1 --2- 172.21.15.49:0/552389641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcff415c8f0 0x7fcff40d09b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:40.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.251+0000 7fcffa3ff700 1 --2- 172.21.15.49:0/552389641 >> v2:172.21.15.107:3300/0 conn(0x7fcff40b1e10 0x7fcff40d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:40.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.251+0000 7fcffa3ff700 1 -- 172.21.15.49:0/552389641 >> 172.21.15.49:0/552389641 conn(0x7fcff409c150 msgr2=0x7fcff40d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:40.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.251+0000 7fcffa3ff700 1 -- 172.21.15.49:0/552389641 shutdown_connections 2023-07-18T03:39:40.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.252+0000 7fcffa3ff700 1 -- 172.21.15.49:0/552389641 wait complete. 2023-07-18T03:39:40.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.253+0000 7fcffa3ff700 1 Processor -- start 2023-07-18T03:39:40.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.253+0000 7fcffa3ff700 1 -- start start 2023-07-18T03:39:40.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.254+0000 7fcffa3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcff40b1e10 0x7fcff40b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:40.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.254+0000 7fcffa3ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff40b2740 0x7fcff41f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:40.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.254+0000 7fcffa3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff415c8f0 0x7fcff40b8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:40.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.254+0000 7fcffa3ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40b6ce0 con 0x7fcff40b2740 2023-07-18T03:39:40.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.254+0000 7fcffa3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40b6060 con 0x7fcff415c8f0 2023-07-18T03:39:40.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.254+0000 7fcffa3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40b6340 con 0x7fcff40b1e10 2023-07-18T03:39:40.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.255+0000 7fcff37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff40b2740 0x7fcff41f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:40.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.255+0000 7fcff37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff40b2740 0x7fcff41f5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37128/0 (socket says 172.21.15.49:37128) 2023-07-18T03:39:40.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.255+0000 7fcff37fe700 1 -- 172.21.15.49:0/1952918226 learned_addr learned my addr 172.21.15.49:0/1952918226 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:40.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.255+0000 7fcff899c700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff415c8f0 0x7fcff40b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:40.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.255+0000 7fcff3fff700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcff40b1e10 0x7fcff40b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:40.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.255+0000 7fcff37fe700 1 -- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcff40b1e10 msgr2=0x7fcff40b7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:40.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.255+0000 7fcff37fe700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcff40b1e10 0x7fcff40b7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:40.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.255+0000 7fcff37fe700 1 -- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff415c8f0 msgr2=0x7fcff40b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:40.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.255+0000 7fcff37fe700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff415c8f0 0x7fcff40b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:40.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.255+0000 7fcff37fe700 1 -- 172.21.15.49:0/1952918226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe80094a0 con 0x7fcff40b2740 2023-07-18T03:39:40.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.255+0000 7fcff899c700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff415c8f0 0x7fcff40b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:40.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.255+0000 7fcff37fe700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff40b2740 0x7fcff41f5240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800ebe0 tx=0x7fcfe8009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:40.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.256+0000 7fcff0ff9700 1 -- 172.21.15.49:0/1952918226 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800c8b0 con 0x7fcff40b2740 2023-07-18T03:39:40.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.256+0000 7fcff0ff9700 1 -- 172.21.15.49:0/1952918226 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe800ca10 con 0x7fcff40b2740 2023-07-18T03:39:40.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.256+0000 7fcff0ff9700 1 -- 172.21.15.49:0/1952918226 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800cda0 con 0x7fcff40b2740 2023-07-18T03:39:40.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.256+0000 7fcffa3ff700 1 -- 172.21.15.49:0/1952918226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe40094a0 con 0x7fcff40b2740 2023-07-18T03:39:40.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.256+0000 7fcffa3ff700 1 -- 172.21.15.49:0/1952918226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff40b6900 con 0x7fcff40b2740 2023-07-18T03:39:40.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.257+0000 7fcffa3ff700 1 -- 172.21.15.49:0/1952918226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff40b6e20 con 0x7fcff40b2740 2023-07-18T03:39:40.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.258+0000 7fcff0ff9700 1 -- 172.21.15.49:0/1952918226 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcfe8013470 con 0x7fcff40b2740 2023-07-18T03:39:40.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.259+0000 7fcff0ff9700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcfe006ace0 0x7fcfe006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:40.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.259+0000 7fcff0ff9700 1 -- 172.21.15.49:0/1952918226 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5768+0+0 (secure 0 0 0) 0x7fcfe80895a0 con 0x7fcff40b2740 2023-07-18T03:39:40.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.259+0000 7fcff3fff700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcfe006ace0 0x7fcfe006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:40.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.260+0000 7fcff3fff700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcfe006ace0 0x7fcfe006d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40090b0 tx=0x7fcfe4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:40.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.263+0000 7fcff0ff9700 1 -- 172.21.15.49:0/1952918226 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcff40b6e20 con 0x7fcff40b2740 2023-07-18T03:39:40.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:40.469+0000 7fcffa3ff700 1 -- 172.21.15.49:0/1952918226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcff40b6e20 con 0x7fcff40b2740 2023-07-18T03:39:41.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.014+0000 7fcff0ff9700 1 -- 172.21.15.49:0/1952918226 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcff40b6e20 con 0x7fcff40b2740 2023-07-18T03:39:41.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.020+0000 7fcffa3ff700 1 -- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcfe006ace0 msgr2=0x7fcfe006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:41.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.020+0000 7fcffa3ff700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcfe006ace0 0x7fcfe006d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40090b0 tx=0x7fcfe4008030 comp rx=0 tx=0).stop 2023-07-18T03:39:41.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.020+0000 7fcffa3ff700 1 -- 172.21.15.49:0/1952918226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff40b2740 msgr2=0x7fcff41f5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:41.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.020+0000 7fcffa3ff700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff40b2740 0x7fcff41f5240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800ebe0 tx=0x7fcfe8009e20 comp rx=0 tx=0).stop 2023-07-18T03:39:41.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.021+0000 7fcffa3ff700 1 -- 172.21.15.49:0/1952918226 shutdown_connections 2023-07-18T03:39:41.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.021+0000 7fcffa3ff700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff415c8f0 0x7fcff40b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:41.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.021+0000 7fcffa3ff700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcff40b1e10 0x7fcff40b7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:41.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.021+0000 7fcffa3ff700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcfe006ace0 0x7fcfe006d190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:41.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.021+0000 7fcffa3ff700 1 --2- 172.21.15.49:0/1952918226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcff40b2740 0x7fcff41f5240 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:41.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.021+0000 7fcffa3ff700 1 -- 172.21.15.49:0/1952918226 >> 172.21.15.49:0/1952918226 conn(0x7fcff409c150 msgr2=0x7fcff40b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:41.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.022+0000 7fcffa3ff700 1 -- 172.21.15.49:0/1952918226 shutdown_connections 2023-07-18T03:39:41.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:41.022+0000 7fcffa3ff700 1 -- 172.21.15.49:0/1952918226 wait complete. 2023-07-18T03:39:41.021 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2023-07-18T03:39:41.041 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-18T03:39:41.041 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-18T03:39:41.041 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-18T03:39:41.042 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:39:41.042 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-18T03:39:41.042 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-18T03:39:41.042 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-18T03:39:41.042 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:39:41.043 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-18T03:39:41.043 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-18T03:39:41.043 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-18T03:39:41.044 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:39:41.044 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-18T03:39:41.044 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-18T03:39:41.044 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-18T03:39:41.045 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:39:41.056 INFO:tasks.ceph.mds.a:Stopped 2023-07-18T03:39:41.057 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-18T03:39:41.058 INFO:tasks.ceph.mds.c:Stopped 2023-07-18T03:39:41.058 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-18T03:39:41.062 INFO:tasks.ceph.mds.d:Stopped 2023-07-18T03:39:41.063 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-18T03:39:41.102 INFO:tasks.ceph.mds.a:Started 2023-07-18T03:39:41.103 DEBUG:teuthology.parallel:result is None 2023-07-18T03:39:41.106 INFO:tasks.ceph.mds.c:Started 2023-07-18T03:39:41.107 DEBUG:teuthology.parallel:result is None 2023-07-18T03:39:41.107 INFO:tasks.ceph.mds.d:Started 2023-07-18T03:39:41.108 DEBUG:teuthology.parallel:result is None 2023-07-18T03:39:41.253 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2023-07-18T03:39:41.254 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2023-07-18T03:39:41.255 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-07-18T03:39:41.257 INFO:tasks.ceph.mds.a.smithi049.stderr:2023-07-18T03:39:41.262+0000 7f1a575fd700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6833/0: (98) Address already in use 2023-07-18T03:39:43.505 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-07-18T03:39:47.063 INFO:tasks.ceph.mds.b:Stopped 2023-07-18T03:39:47.063 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-18T03:39:47.069 INFO:tasks.ceph.mds.b:Started 2023-07-18T03:39:47.069 DEBUG:teuthology.parallel:result is None 2023-07-18T03:39:47.069 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-18T03:39:47.218 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2023-07-18T03:39:47.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.293+0000 7fcee97ff700 1 Processor -- start 2023-07-18T03:39:47.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.293+0000 7fcee97ff700 1 -- start start 2023-07-18T03:39:47.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.294+0000 7fcee97ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcee415d9d0 0x7fcee406b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:47.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.295+0000 7fcee97ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee406c4e0 con 0x7fcee4071170 2023-07-18T03:39:47.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.295+0000 7fcee97ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee40b2020 con 0x7fcee406c9f0 2023-07-18T03:39:47.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.295+0000 7fcee97ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee415dda0 con 0x7fcee415d9d0 2023-07-18T03:39:47.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.295+0000 7fcee2ffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcee415d9d0 0x7fcee406b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:47.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.295+0000 7fcee37fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fcee4071170 0x7fcee40715f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47580/0 (socket says 172.21.15.49:47580) 2023-07-18T03:39:47.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.295+0000 7fcee37fe700 1 -- 172.21.15.49:0/2442127043 learned_addr learned my addr 172.21.15.49:0/2442127043 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:47.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.295+0000 7fcee2ffd700 1 -- 172.21.15.49:0/2442127043 >> v1:172.21.15.107:6789/0 conn(0x7fcee406c9f0 legacy=0x7fcee40b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:47.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.295+0000 7fcee2ffd700 1 -- 172.21.15.49:0/2442127043 >> v1:172.21.15.49:6789/0 conn(0x7fcee4071170 legacy=0x7fcee40715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:47.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.295+0000 7fcee1ffb700 1 -- 172.21.15.49:0/2442127043 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043283384 0 0) 0x7fcee40b2020 con 0x7fcee406c9f0 2023-07-18T03:39:47.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.295+0000 7fcee2ffd700 1 -- 172.21.15.49:0/2442127043 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee406bf20 con 0x7fcee415d9d0 2023-07-18T03:39:47.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.296+0000 7fcee2ffd700 1 --2- 172.21.15.49:0/2442127043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee415d9d0 0x7fcee406b5b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fced4009840 tx=0x7fced4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d319354aed6a0b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:47.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.296+0000 7fcee1ffb700 1 -- 172.21.15.49:0/2442127043 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4004230 con 0x7fcee415d9d0 2023-07-18T03:39:47.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.296+0000 7fcee1ffb700 1 -- 172.21.15.49:0/2442127043 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced4004390 con 0x7fcee415d9d0 2023-07-18T03:39:47.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.296+0000 7fcee1ffb700 1 -- 172.21.15.49:0/2442127043 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced40036a0 con 0x7fcee415d9d0 2023-07-18T03:39:47.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.296+0000 7fcee97ff700 1 -- 172.21.15.49:0/2442127043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee415d9d0 msgr2=0x7fcee406b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:47.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.296+0000 7fcee97ff700 1 --2- 172.21.15.49:0/2442127043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee415d9d0 0x7fcee406b5b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fced4009840 tx=0x7fced4009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:47.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.296+0000 7fcee97ff700 1 -- 172.21.15.49:0/2442127043 shutdown_connections 2023-07-18T03:39:47.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.296+0000 7fcee97ff700 1 --2- 172.21.15.49:0/2442127043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee415d9d0 0x7fcee406b5b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.296+0000 7fcee97ff700 1 -- 172.21.15.49:0/2442127043 >> 172.21.15.49:0/2442127043 conn(0x7fcee40b05d0 msgr2=0x7fcee40b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.297+0000 7fcee97ff700 1 -- 172.21.15.49:0/2442127043 shutdown_connections 2023-07-18T03:39:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.297+0000 7fcee97ff700 1 -- 172.21.15.49:0/2442127043 wait complete. 2023-07-18T03:39:47.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.298+0000 7fcee97ff700 1 Processor -- start 2023-07-18T03:39:47.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.298+0000 7fcee97ff700 1 -- start start 2023-07-18T03:39:47.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.299+0000 7fcee97ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcee406c9f0 0x7fcee4151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:47.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.299+0000 7fcee97ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcee4071170 0x7fcee41f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:47.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.299+0000 7fcee97ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee415d9d0 0x7fcee40cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:47.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.299+0000 7fcee97ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee41524d0 con 0x7fcee415d9d0 2023-07-18T03:39:47.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.299+0000 7fcee97ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee41f63b0 con 0x7fcee406c9f0 2023-07-18T03:39:47.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.299+0000 7fcee97ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee41f5df0 con 0x7fcee4071170 2023-07-18T03:39:47.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee415d9d0 0x7fcee40cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:47.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee415d9d0 0x7fcee40cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37174/0 (socket says 172.21.15.49:37174) 2023-07-18T03:39:47.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee37fe700 1 -- 172.21.15.49:0/1360617365 learned_addr learned my addr 172.21.15.49:0/1360617365 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:47.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee2ffd700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcee406c9f0 0x7fcee4151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:47.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee37fe700 1 -- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcee4071170 msgr2=0x7fcee41f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:47.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee27fc700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcee4071170 0x7fcee41f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:47.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee37fe700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcee4071170 0x7fcee41f5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:47.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee37fe700 1 -- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcee406c9f0 msgr2=0x7fcee4151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:47.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee37fe700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcee406c9f0 0x7fcee4151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:47.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee37fe700 1 -- 172.21.15.49:0/1360617365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced40094a0 con 0x7fcee415d9d0 2023-07-18T03:39:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee27fc700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcee4071170 0x7fcee41f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee2ffd700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcee406c9f0 0x7fcee4151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:47.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.300+0000 7fcee37fe700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee415d9d0 0x7fcee40cea20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00cca0 tx=0x7fcedc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.301+0000 7fcecb7fe700 1 -- 172.21.15.49:0/1360617365 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc00cea0 con 0x7fcee415d9d0 2023-07-18T03:39:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.301+0000 7fcecb7fe700 1 -- 172.21.15.49:0/1360617365 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc00ae00 con 0x7fcee415d9d0 2023-07-18T03:39:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.301+0000 7fcecb7fe700 1 -- 172.21.15.49:0/1360617365 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc007310 con 0x7fcee415d9d0 2023-07-18T03:39:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.301+0000 7fcee97ff700 1 -- 172.21.15.49:0/1360617365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee41f60d0 con 0x7fcee415d9d0 2023-07-18T03:39:47.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.301+0000 7fcee97ff700 1 -- 172.21.15.49:0/1360617365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee41f3460 con 0x7fcee415d9d0 2023-07-18T03:39:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.302+0000 7fcee97ff700 1 -- 172.21.15.49:0/1360617365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee4074350 con 0x7fcee415d9d0 2023-07-18T03:39:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.303+0000 7fcecb7fe700 1 -- 172.21.15.49:0/1360617365 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcedc007470 con 0x7fcee415d9d0 2023-07-18T03:39:47.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.304+0000 7fcecb7fe700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fced006ace0 0x7fced006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:47.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.304+0000 7fcecb7fe700 1 -- 172.21.15.49:0/1360617365 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcedc0891a0 con 0x7fcee415d9d0 2023-07-18T03:39:47.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.305+0000 7fcee2ffd700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fced006ace0 0x7fced006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:47.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.305+0000 7fcee2ffd700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fced006ace0 0x7fced006d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fced40090b0 tx=0x7fced4013040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:47.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.309+0000 7fcecb7fe700 1 -- 172.21.15.49:0/1360617365 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcedc054560 con 0x7fcee415d9d0 2023-07-18T03:39:47.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:47.533+0000 7fcee97ff700 1 -- 172.21.15.49:0/1360617365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcee41f3650 con 0x7fcee415d9d0 2023-07-18T03:39:48.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.300+0000 7fcecb7fe700 1 -- 172.21.15.49:0/1360617365 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcedc057b80 con 0x7fcee415d9d0 2023-07-18T03:39:48.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.303+0000 7fcee97ff700 1 -- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fced006ace0 msgr2=0x7fced006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.303+0000 7fcee97ff700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fced006ace0 0x7fced006d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fced40090b0 tx=0x7fced4013040 comp rx=0 tx=0).stop 2023-07-18T03:39:48.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.303+0000 7fcee97ff700 1 -- 172.21.15.49:0/1360617365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee415d9d0 msgr2=0x7fcee40cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.303+0000 7fcee97ff700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee415d9d0 0x7fcee40cea20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00cca0 tx=0x7fcedc00a340 comp rx=0 tx=0).stop 2023-07-18T03:39:48.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.304+0000 7fcee97ff700 1 -- 172.21.15.49:0/1360617365 shutdown_connections 2023-07-18T03:39:48.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.304+0000 7fcee97ff700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fced006ace0 0x7fced006d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.304+0000 7fcee97ff700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee415d9d0 0x7fcee40cea20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.304+0000 7fcee97ff700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcee406c9f0 0x7fcee4151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.304+0000 7fcee97ff700 1 --2- 172.21.15.49:0/1360617365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcee4071170 0x7fcee41f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.304+0000 7fcee97ff700 1 -- 172.21.15.49:0/1360617365 >> 172.21.15.49:0/1360617365 conn(0x7fcee40b05d0 msgr2=0x7fcee40cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:48.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.304+0000 7fcee97ff700 1 -- 172.21.15.49:0/1360617365 shutdown_connections 2023-07-18T03:39:48.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.304+0000 7fcee97ff700 1 -- 172.21.15.49:0/1360617365 wait complete. 2023-07-18T03:39:48.300 INFO:teuthology.orchestra.run.smithi049.stderr: removed all blocklist entries 2023-07-18T03:39:48.319 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-18T03:39:48.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.535+0000 7ff85fbff700 1 Processor -- start 2023-07-18T03:39:48.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.535+0000 7ff85fbff700 1 -- start start 2023-07-18T03:39:48.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.536+0000 7ff85fbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff85815c950 0x7ff8580b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.536+0000 7ff85fbff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff8580d1220 0x7ff8580afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.536+0000 7ff85fbff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8580d2ce0 con 0x7ff8580b1e00 2023-07-18T03:39:48.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.536+0000 7ff85fbff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8580b3350 con 0x7ff8580d1220 2023-07-18T03:39:48.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.536+0000 7ff85fbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580b3630 con 0x7ff85815c950 2023-07-18T03:39:48.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85d19a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff8580b1e00 0x7ff8580d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47582/0 (socket says 172.21.15.49:47582) 2023-07-18T03:39:48.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85d19a700 1 -- 172.21.15.49:0/908797636 learned_addr learned my addr 172.21.15.49:0/908797636 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:48.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866403116 0 0) 0x7ff8580d2ce0 con 0x7ff8580b1e00 2023-07-18T03:39:48.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85e19c700 1 --2- 172.21.15.49:0/908797636 >> v2:172.21.15.107:3300/0 conn(0x7ff8580d1220 0x7ff8580afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85d99b700 1 --2- 172.21.15.49:0/908797636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff85815c950 0x7ff8580b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8440035b0 con 0x7ff8580b1e00 2023-07-18T03:39:48.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1413378335 0 0) 0x7ff8440035b0 con 0x7ff8580b1e00 2023-07-18T03:39:48.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff85815c950 msgr2=0x7ff8580b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:48.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 --2- 172.21.15.49:0/908797636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff85815c950 0x7ff8580b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 >> v2:172.21.15.107:3300/0 conn(0x7ff8580d1220 msgr2=0x7ff8580afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:48.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 --2- 172.21.15.49:0/908797636 >> v2:172.21.15.107:3300/0 conn(0x7ff8580d1220 0x7ff8580afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8580b3910 con 0x7ff8580b1e00 2023-07-18T03:39:48.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7ff848001670 con 0x7ff8580b1e00 2023-07-18T03:39:48.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85e19c700 1 --2- 172.21.15.49:0/908797636 >> v2:172.21.15.107:3300/0 conn(0x7ff8580d1220 0x7ff8580afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:48.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 >> v1:172.21.15.49:6789/0 conn(0x7ff8580b1e00 legacy=0x7ff8580d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 --2- 172.21.15.49:0/908797636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff844003a00 0x7ff844003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 --2- 172.21.15.49:0/908797636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff85815c950 0x7ff844004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.537+0000 7ff85c999700 1 --2- 172.21.15.49:0/908797636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff844004d90 0x7ff844005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.538+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580b3350 con 0x7ff844004d90 2023-07-18T03:39:48.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.538+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580b3630 con 0x7ff844003a00 2023-07-18T03:39:48.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.538+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84400ba90 con 0x7ff85815c950 2023-07-18T03:39:48.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.538+0000 7ff85e19c700 1 --2- 172.21.15.49:0/908797636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff844004d90 0x7ff844005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.538+0000 7ff85d19a700 1 --2- 172.21.15.49:0/908797636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff85815c950 0x7ff844004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.538+0000 7ff85d99b700 1 --2- 172.21.15.49:0/908797636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff844003a00 0x7ff844003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.538+0000 7ff85e19c700 1 -- 172.21.15.49:0/908797636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff85815c950 msgr2=0x7ff844004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:48.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.538+0000 7ff85e19c700 1 --2- 172.21.15.49:0/908797636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff85815c950 0x7ff844004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.538+0000 7ff85e19c700 1 -- 172.21.15.49:0/908797636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff844003a00 msgr2=0x7ff844003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:48.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.538+0000 7ff85e19c700 1 --2- 172.21.15.49:0/908797636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff844003a00 0x7ff844003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.538+0000 7ff85e19c700 1 -- 172.21.15.49:0/908797636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8440035b0 con 0x7ff844004d90 2023-07-18T03:39:48.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.538+0000 7ff85e19c700 1 --2- 172.21.15.49:0/908797636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff844004d90 0x7ff844005200 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff8500073b0 tx=0x7ff844005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f1d8b04dce72ed server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:48.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.539+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850014400 con 0x7ff844004d90 2023-07-18T03:39:48.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.539+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff850014560 con 0x7ff844004d90 2023-07-18T03:39:48.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.539+0000 7ff85c999700 1 -- 172.21.15.49:0/908797636 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8500148d0 con 0x7ff844004d90 2023-07-18T03:39:48.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.539+0000 7ff85fbff700 1 -- 172.21.15.49:0/908797636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff844004d90 msgr2=0x7ff844005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.539+0000 7ff85fbff700 1 --2- 172.21.15.49:0/908797636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff844004d90 0x7ff844005200 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff8500073b0 tx=0x7ff844005880 comp rx=0 tx=0).stop 2023-07-18T03:39:48.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.539+0000 7ff85e19c700 1 -- 172.21.15.49:0/908797636 reap_dead start 2023-07-18T03:39:48.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.539+0000 7ff85fbff700 1 -- 172.21.15.49:0/908797636 shutdown_connections 2023-07-18T03:39:48.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.539+0000 7ff85fbff700 1 -- 172.21.15.49:0/908797636 >> 172.21.15.49:0/908797636 conn(0x7ff85809c1d0 msgr2=0x7ff8580d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:48.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.539+0000 7ff85fbff700 1 -- 172.21.15.49:0/908797636 shutdown_connections 2023-07-18T03:39:48.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.539+0000 7ff85fbff700 1 -- 172.21.15.49:0/908797636 wait complete. 2023-07-18T03:39:48.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.541+0000 7ff85fbff700 1 Processor -- start 2023-07-18T03:39:48.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.541+0000 7ff85fbff700 1 -- start start 2023-07-18T03:39:48.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.542+0000 7ff85fbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff844004d90 0x7ff858151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.542+0000 7ff85fbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8580c7db0 0x7ff8580c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.542+0000 7ff85fbff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8581e7340 0x7ff8581e77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.542+0000 7ff85fbff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8581e7cf0 con 0x7ff8581e7340 2023-07-18T03:39:48.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.542+0000 7ff85fbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580c9530 con 0x7ff8580c7db0 2023-07-18T03:39:48.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.542+0000 7ff85fbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8581e6ee0 con 0x7ff844004d90 2023-07-18T03:39:48.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.543+0000 7ff85e19c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8581e7340 0x7ff8581e77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.543+0000 7ff85e19c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8581e7340 0x7ff8581e77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37186/0 (socket says 172.21.15.49:37186) 2023-07-18T03:39:48.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.543+0000 7ff85e19c700 1 -- 172.21.15.49:0/533018306 learned_addr learned my addr 172.21.15.49:0/533018306 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:48.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.543+0000 7ff85d19a700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8580c7db0 0x7ff8580c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.543+0000 7ff85d99b700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff844004d90 0x7ff858151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.543+0000 7ff85e19c700 1 -- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff844004d90 msgr2=0x7ff858151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.543+0000 7ff85e19c700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff844004d90 0x7ff858151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.543+0000 7ff85e19c700 1 -- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8580c7db0 msgr2=0x7ff8580c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.543+0000 7ff85e19c700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8580c7db0 0x7ff8580c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.543+0000 7ff85e19c700 1 -- 172.21.15.49:0/533018306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8580c8920 con 0x7ff8581e7340 2023-07-18T03:39:48.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.543+0000 7ff85d19a700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8580c7db0 0x7ff8580c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:48.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.543+0000 7ff85e19c700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8581e7340 0x7ff8581e77b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff8440047d0 tx=0x7ff850004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:48.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.544+0000 7ff84e7fc700 1 -- 172.21.15.49:0/533018306 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850014400 con 0x7ff8581e7340 2023-07-18T03:39:48.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.544+0000 7ff84e7fc700 1 -- 172.21.15.49:0/533018306 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff850003e10 con 0x7ff8581e7340 2023-07-18T03:39:48.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.544+0000 7ff85fbff700 1 -- 172.21.15.49:0/533018306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8580c8c00 con 0x7ff8581e7340 2023-07-18T03:39:48.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.544+0000 7ff84e7fc700 1 -- 172.21.15.49:0/533018306 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850007000 con 0x7ff8581e7340 2023-07-18T03:39:48.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.544+0000 7ff85fbff700 1 -- 172.21.15.49:0/533018306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8580c91c0 con 0x7ff8581e7340 2023-07-18T03:39:48.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.545+0000 7ff85fbff700 1 -- 172.21.15.49:0/533018306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8581e8200 con 0x7ff8581e7340 2023-07-18T03:39:48.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.546+0000 7ff84e7fc700 1 -- 172.21.15.49:0/533018306 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff850004040 con 0x7ff8581e7340 2023-07-18T03:39:48.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.547+0000 7ff84e7fc700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff83c06ace0 0x7ff83c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.547+0000 7ff84e7fc700 1 -- 172.21.15.49:0/533018306 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff85008aaa0 con 0x7ff8581e7340 2023-07-18T03:39:48.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.547+0000 7ff85d99b700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff83c06ace0 0x7ff83c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.548+0000 7ff85d99b700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff83c06ace0 0x7ff83c06d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff854009b00 tx=0x7ff854008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:48.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.551+0000 7ff84e7fc700 1 -- 172.21.15.49:0/533018306 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff850055fb0 con 0x7ff8581e7340 2023-07-18T03:39:48.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.839+0000 7ff85fbff700 1 -- 172.21.15.49:0/533018306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff85804e280 con 0x7ff8581e7340 2023-07-18T03:39:48.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.840+0000 7ff84e7fc700 1 -- 172.21.15.49:0/533018306 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7ff850019090 con 0x7ff8581e7340 2023-07-18T03:39:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:48.834 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:39:48.834 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_dump": [ 2023-07-18T03:39:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.a", 2023-07-18T03:39:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD+CLZk0A9wAxAAPyxogWvN53TNCPyBYAgGgQ==", 2023-07-18T03:39:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:39:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-07-18T03:39:48.835 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-07-18T03:39:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.b", 2023-07-18T03:39:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD+CLZkWB96GxAAepmZ6nF9qh75BBmZTY1W4w==", 2023-07-18T03:39:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.836 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:39:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-07-18T03:39:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-07-18T03:39:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.837 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.c", 2023-07-18T03:39:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD+CLZkCKiYDxAAhmsGhQF1Uusa7eSwZKWIWw==", 2023-07-18T03:39:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:39:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-07-18T03:39:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-07-18T03:39:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.d", 2023-07-18T03:39:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD+CLZkKJwHKBAAusYdoX3eXJdWmNNsa7RuZA==", 2023-07-18T03:39:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:39:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-07-18T03:39:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-07-18T03:39:48.839 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.0", 2023-07-18T03:39:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAACbZkV1wVMhAAqSqjoEtRctkKCcFiUUOTcw==", 2023-07-18T03:39:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.840 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:39:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:39:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.1", 2023-07-18T03:39:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQADCbZk9S6TNhAAOl9IxSR58LnxX58i/sjMvg==", 2023-07-18T03:39:48.841 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.842 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:39:48.842 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:39:48.842 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.842 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.842 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.2", 2023-07-18T03:39:48.843 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAGCbZke4glOxAAASCGrrHrKsam5fv4gjdZ/A==", 2023-07-18T03:39:48.843 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.843 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:39:48.843 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:39:48.843 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.843 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.844 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.3", 2023-07-18T03:39:48.844 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAKCbZkP4iFAxAA/nPkDfrSOZeJm2QyEAGNiQ==", 2023-07-18T03:39:48.844 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.844 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:39:48.844 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:39:48.844 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.844 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.845 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.4", 2023-07-18T03:39:48.845 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAQCbZkxYX2GRAA1Y4sxGA6H05aa5jydM38WQ==", 2023-07-18T03:39:48.845 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.845 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:39:48.845 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:39:48.845 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.846 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.5", 2023-07-18T03:39:48.846 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQATCbZkt8qVIBAAeaCS5rmMJEgwZUjh3siVTw==", 2023-07-18T03:39:48.846 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.846 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:39:48.846 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:39:48.847 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.847 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.847 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.6", 2023-07-18T03:39:48.847 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAWCbZkK/qRKBAA7wpJMzxg/K+YMxEJHUeRPw==", 2023-07-18T03:39:48.847 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.847 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:39:48.848 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:39:48.848 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.848 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.7", 2023-07-18T03:39:48.848 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAZCbZk2ltrMhAATvhB8r+uw5+LDGlMtJzEbw==", 2023-07-18T03:39:48.848 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.849 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:39:48.849 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:39:48.849 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.849 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.849 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.0", 2023-07-18T03:39:48.849 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD+CLZkI5fMMhAAisfdJO7kEPCDAngS1ywP+g==", 2023-07-18T03:39:48.850 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.850 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:39:48.850 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-07-18T03:39:48.850 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-07-18T03:39:48.850 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-07-18T03:39:48.850 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.851 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.1", 2023-07-18T03:39:48.851 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD/CLZkgQfPBhAAIkgoL4P4YFEqTZqP/8hYHQ==", 2023-07-18T03:39:48.851 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.851 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:39:48.851 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-07-18T03:39:48.851 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-07-18T03:39:48.851 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-07-18T03:39:48.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.852 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.2", 2023-07-18T03:39:48.852 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD+CLZk1mmtOhAAYC8ALN9saaob9naZYRC6Vg==", 2023-07-18T03:39:48.852 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.852 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:39:48.852 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-07-18T03:39:48.853 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-07-18T03:39:48.853 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-07-18T03:39:48.853 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.853 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.3", 2023-07-18T03:39:48.853 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD/CLZk1OaMDhAARMs8iCxSvTEiQomODnCtIg==", 2023-07-18T03:39:48.853 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.854 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:39:48.854 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-07-18T03:39:48.854 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-07-18T03:39:48.854 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-07-18T03:39:48.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.854 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.admin", 2023-07-18T03:39:48.855 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD9CLZk9BpYBhAAGOejpn6CQyH1hCyRmbyKrQ==", 2023-07-18T03:39:48.855 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.855 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-07-18T03:39:48.855 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-07-18T03:39:48.855 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-07-18T03:39:48.855 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.855 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.856 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-mds", 2023-07-18T03:39:48.856 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAnCbZkw/JHBhAAdEKJMUn3DBJyO9oR2iGyuA==", 2023-07-18T03:39:48.856 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.856 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-mds" 2023-07-18T03:39:48.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.857 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-mgr", 2023-07-18T03:39:48.857 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAnCbZkQAhIBhAAkZ0lYMugq8mkJ4H6j92Yhg==", 2023-07-18T03:39:48.857 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.857 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-mgr" 2023-07-18T03:39:48.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.858 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-osd", 2023-07-18T03:39:48.858 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAnCbZkSxlIBhAAHa0ZE221uamAsbWk7furZQ==", 2023-07-18T03:39:48.858 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.858 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-osd" 2023-07-18T03:39:48.858 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.858 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-rbd", 2023-07-18T03:39:48.859 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAnCbZkNChIBhAASKu6Pjv/kehj7+ClyxTIeQ==", 2023-07-18T03:39:48.859 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.859 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-rbd" 2023-07-18T03:39:48.859 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.859 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-07-18T03:39:48.859 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAnCbZk5zJIBhAAiWI8dYh5/P8R47RevC4cTg==", 2023-07-18T03:39:48.860 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.860 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-07-18T03:39:48.860 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.860 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-rgw", 2023-07-18T03:39:48.860 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAnCbZkxT1IBhAAK+JsavTfiNfS0/p6rE53Gw==", 2023-07-18T03:39:48.860 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.861 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-rgw" 2023-07-18T03:39:48.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.861 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.x", 2023-07-18T03:39:48.861 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD9CLZk6hvxNhAAMALkYp+kAehBGNNCw/BJfg==", 2023-07-18T03:39:48.861 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.861 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-07-18T03:39:48.862 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2023-07-18T03:39:48.862 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.862 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:48.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:48.862 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.y", 2023-07-18T03:39:48.862 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD9CLZkATFALhAAqRgQzaCzvqgqsvTwhZ/+rw==", 2023-07-18T03:39:48.863 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:39:48.863 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-07-18T03:39:48.863 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2023-07-18T03:39:48.863 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:39:48.863 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.863 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:48.863 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-07-18T03:39:48.863 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:39:48.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.842+0000 7ff85fbff700 1 -- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff83c06ace0 msgr2=0x7ff83c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.842+0000 7ff85fbff700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff83c06ace0 0x7ff83c06d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff854009b00 tx=0x7ff854008040 comp rx=0 tx=0).stop 2023-07-18T03:39:48.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.842+0000 7ff85fbff700 1 -- 172.21.15.49:0/533018306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8581e7340 msgr2=0x7ff8581e77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.842+0000 7ff85fbff700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8581e7340 0x7ff8581e77b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff8440047d0 tx=0x7ff850004b40 comp rx=0 tx=0).stop 2023-07-18T03:39:48.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.842+0000 7ff85fbff700 1 -- 172.21.15.49:0/533018306 shutdown_connections 2023-07-18T03:39:48.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.842+0000 7ff85fbff700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff83c06ace0 0x7ff83c06d190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.842+0000 7ff85fbff700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8581e7340 0x7ff8581e77b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.842+0000 7ff85fbff700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff844004d90 0x7ff858151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.842+0000 7ff85fbff700 1 --2- 172.21.15.49:0/533018306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8580c7db0 0x7ff8580c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.842+0000 7ff85fbff700 1 -- 172.21.15.49:0/533018306 >> 172.21.15.49:0/533018306 conn(0x7ff85809c1d0 msgr2=0x7ff8580b8370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:48.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.842+0000 7ff85fbff700 1 -- 172.21.15.49:0/533018306 shutdown_connections 2023-07-18T03:39:48.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:48.842+0000 7ff85fbff700 1 -- 172.21.15.49:0/533018306 wait complete. 2023-07-18T03:39:48.867 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-07-18T03:39:49.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.078+0000 7f97f19ff700 1 Processor -- start 2023-07-18T03:39:49.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.078+0000 7f97f19ff700 1 -- start start 2023-07-18T03:39:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.080+0000 7f97f19ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97ec158bb0 0x7f97ec0cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.080+0000 7f97f19ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec0ce8c0 con 0x7f97ec0ce150 2023-07-18T03:39:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.080+0000 7f97f19ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec0cc2b0 con 0x7f97ec0cbe40 2023-07-18T03:39:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.080+0000 7f97f19ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec06b5b0 con 0x7f97ec158bb0 2023-07-18T03:39:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.080+0000 7f97eb7fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f97ec0ce150 0x7f97ec06ba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37608/0 (socket says 172.21.15.49:37608) 2023-07-18T03:39:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.080+0000 7f97eb7fe700 1 -- 172.21.15.49:0/3326853174 learned_addr learned my addr 172.21.15.49:0/3326853174 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.080+0000 7f97eaffd700 1 --2- 172.21.15.49:0/3326853174 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97ec158bb0 0x7f97ec0cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:49.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.081+0000 7f97e9ffb700 1 -- 172.21.15.49:0/3326853174 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153253865 0 0) 0x7f97ec0ce8c0 con 0x7f97ec0ce150 2023-07-18T03:39:49.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.081+0000 7f97e9ffb700 1 -- 172.21.15.49:0/3326853174 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97d40035b0 con 0x7f97ec0ce150 2023-07-18T03:39:49.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.081+0000 7f97eaffd700 1 -- 172.21.15.49:0/3326853174 >> v1:172.21.15.107:6789/0 conn(0x7f97ec0cbe40 legacy=0x7f97ec0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.081+0000 7f97e9ffb700 1 -- 172.21.15.49:0/3326853174 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573537295 0 0) 0x7f97ec0cc2b0 con 0x7f97ec0cbe40 2023-07-18T03:39:49.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.081+0000 7f97eaffd700 1 -- 172.21.15.49:0/3326853174 >> v1:172.21.15.49:6789/0 conn(0x7f97ec0ce150 legacy=0x7f97ec06ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.081+0000 7f97eaffd700 1 -- 172.21.15.49:0/3326853174 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec0cae30 con 0x7f97ec158bb0 2023-07-18T03:39:49.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.081+0000 7f97eaffd700 1 --2- 172.21.15.49:0/3326853174 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97ec158bb0 0x7f97ec0cd0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00aa10 tx=0x7f97dc00ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b7430a2a47b3eff server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:49.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.081+0000 7f97e9ffb700 1 -- 172.21.15.49:0/3326853174 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00cb70 con 0x7f97ec158bb0 2023-07-18T03:39:49.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.081+0000 7f97e9ffb700 1 -- 172.21.15.49:0/3326853174 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc00ccd0 con 0x7f97ec158bb0 2023-07-18T03:39:49.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.081+0000 7f97e9ffb700 1 -- 172.21.15.49:0/3326853174 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00ae50 con 0x7f97ec158bb0 2023-07-18T03:39:49.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.082+0000 7f97f19ff700 1 -- 172.21.15.49:0/3326853174 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97ec158bb0 msgr2=0x7f97ec0cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.082+0000 7f97f19ff700 1 --2- 172.21.15.49:0/3326853174 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97ec158bb0 0x7f97ec0cd0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00aa10 tx=0x7f97dc00ad20 comp rx=0 tx=0).stop 2023-07-18T03:39:49.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.082+0000 7f97f19ff700 1 -- 172.21.15.49:0/3326853174 shutdown_connections 2023-07-18T03:39:49.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.082+0000 7f97f19ff700 1 --2- 172.21.15.49:0/3326853174 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97ec158bb0 0x7f97ec0cd0a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.082+0000 7f97f19ff700 1 -- 172.21.15.49:0/3326853174 >> 172.21.15.49:0/3326853174 conn(0x7f97ec0b05d0 msgr2=0x7f97ec0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:49.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.082+0000 7f97f19ff700 1 -- 172.21.15.49:0/3326853174 shutdown_connections 2023-07-18T03:39:49.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.082+0000 7f97f19ff700 1 -- 172.21.15.49:0/3326853174 wait complete. 2023-07-18T03:39:49.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.084+0000 7f97f19ff700 1 Processor -- start 2023-07-18T03:39:49.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.084+0000 7f97f19ff700 1 -- start start 2023-07-18T03:39:49.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.085+0000 7f97f19ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97ec0cbe40 0x7f97ec148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:49.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.085+0000 7f97f19ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97ec0ce150 0x7f97ec0c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:49.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.085+0000 7f97f19ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97ec158bb0 0x7f97ec0c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:49.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.085+0000 7f97f19ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0c53d0 con 0x7f97ec0ce150 2023-07-18T03:39:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.085+0000 7f97f19ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0c50f0 con 0x7f97ec0cbe40 2023-07-18T03:39:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.085+0000 7f97f19ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec1504e0 con 0x7f97ec158bb0 2023-07-18T03:39:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.085+0000 7f97ea7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97ec0ce150 0x7f97ec0c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.085+0000 7f97ea7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97ec0ce150 0x7f97ec0c4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39940/0 (socket says 172.21.15.49:39940) 2023-07-18T03:39:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.085+0000 7f97ea7fc700 1 -- 172.21.15.49:0/3988659751 learned_addr learned my addr 172.21.15.49:0/3988659751 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.086+0000 7f97eaffd700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97ec0cbe40 0x7f97ec148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:49.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.086+0000 7f97eb7fe700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97ec158bb0 0x7f97ec0c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:49.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.086+0000 7f97eaffd700 1 -- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97ec158bb0 msgr2=0x7f97ec0c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.086+0000 7f97eaffd700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97ec158bb0 0x7f97ec0c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.086+0000 7f97eaffd700 1 -- 172.21.15.49:0/3988659751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97ec0ce150 msgr2=0x7f97ec0c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.086+0000 7f97eaffd700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97ec0ce150 0x7f97ec0c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.086+0000 7f97eaffd700 1 -- 172.21.15.49:0/3988659751 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc00a640 con 0x7f97ec0cbe40 2023-07-18T03:39:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.086+0000 7f97eb7fe700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97ec158bb0 0x7f97ec0c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.086+0000 7f97ea7fc700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97ec0ce150 0x7f97ec0c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.086+0000 7f97eaffd700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97ec0cbe40 0x7f97ec148ab0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f97dc018000 tx=0x7f97dc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.087+0000 7f97d3fff700 1 -- 172.21.15.49:0/3988659751 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00a4c0 con 0x7f97ec0cbe40 2023-07-18T03:39:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.087+0000 7f97d3fff700 1 -- 172.21.15.49:0/3988659751 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc022440 con 0x7f97ec0cbe40 2023-07-18T03:39:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.087+0000 7f97f19ff700 1 -- 172.21.15.49:0/3988659751 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec1507c0 con 0x7f97ec0cbe40 2023-07-18T03:39:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.087+0000 7f97d3fff700 1 -- 172.21.15.49:0/3988659751 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0227b0 con 0x7f97ec0cbe40 2023-07-18T03:39:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.087+0000 7f97f19ff700 1 -- 172.21.15.49:0/3988659751 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec1494d0 con 0x7f97ec0cbe40 2023-07-18T03:39:49.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.088+0000 7f97f19ff700 1 -- 172.21.15.49:0/3988659751 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ec0714e0 con 0x7f97ec0cbe40 2023-07-18T03:39:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.089+0000 7f97d3fff700 1 -- 172.21.15.49:0/3988659751 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97dc022910 con 0x7f97ec0cbe40 2023-07-18T03:39:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.090+0000 7f97d3fff700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f97d806ace0 0x7f97d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.090+0000 7f97d3fff700 1 -- 172.21.15.49:0/3988659751 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f97dc013070 con 0x7f97ec0cbe40 2023-07-18T03:39:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.090+0000 7f97ea7fc700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f97d806ace0 0x7f97d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:49.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.091+0000 7f97ea7fc700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f97d806ace0 0x7f97d806d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f97ec071b40 tx=0x7f97e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.096+0000 7f97d3fff700 1 -- 172.21.15.49:0/3988659751 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f97ec0714e0 con 0x7f97ec0cbe40 2023-07-18T03:39:49.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.391+0000 7f97f19ff700 1 -- 172.21.15.49:0/3988659751 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f97ec0714e0 con 0x7f97ec0cbe40 2023-07-18T03:39:49.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.398+0000 7f97d3fff700 1 -- 172.21.15.49:0/3988659751 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f97ec0714e0 con 0x7f97ec0cbe40 2023-07-18T03:39:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.400+0000 7f97f19ff700 1 -- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f97d806ace0 msgr2=0x7f97d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.400+0000 7f97f19ff700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f97d806ace0 0x7f97d806d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f97ec071b40 tx=0x7f97e0009040 comp rx=0 tx=0).stop 2023-07-18T03:39:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.400+0000 7f97f19ff700 1 -- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97ec0cbe40 msgr2=0x7f97ec148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.400+0000 7f97f19ff700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97ec0cbe40 0x7f97ec148ab0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f97dc018000 tx=0x7f97dc004750 comp rx=0 tx=0).stop 2023-07-18T03:39:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.401+0000 7f97f19ff700 1 -- 172.21.15.49:0/3988659751 shutdown_connections 2023-07-18T03:39:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.401+0000 7f97f19ff700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97ec158bb0 0x7f97ec0c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.401+0000 7f97f19ff700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97ec0cbe40 0x7f97ec148ab0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.401+0000 7f97f19ff700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f97d806ace0 0x7f97d806d190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.401+0000 7f97f19ff700 1 --2- 172.21.15.49:0/3988659751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97ec0ce150 0x7f97ec0c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.401+0000 7f97f19ff700 1 -- 172.21.15.49:0/3988659751 >> 172.21.15.49:0/3988659751 conn(0x7f97ec0b05d0 msgr2=0x7f97ec0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.401+0000 7f97f19ff700 1 -- 172.21.15.49:0/3988659751 shutdown_connections 2023-07-18T03:39:49.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.401+0000 7f97f19ff700 1 -- 172.21.15.49:0/3988659751 wait complete. 2023-07-18T03:39:49.413 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-07-18T03:39:49.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.624+0000 7f9ddbdff700 1 Processor -- start 2023-07-18T03:39:49.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.624+0000 7f9ddbdff700 1 -- start start 2023-07-18T03:39:49.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.625+0000 7f9ddbdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9dd40d1e40 0x7f9dd40cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:49.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.625+0000 7f9ddbdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40c71b0 0x7f9dd40b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:49.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.625+0000 7f9ddbdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd40cd5a0 con 0x7f9dd415c860 2023-07-18T03:39:49.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.625+0000 7f9ddbdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd4146740 con 0x7f9dd40d1e40 2023-07-18T03:39:49.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.625+0000 7f9ddbdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd40cd7f0 con 0x7f9dd40c71b0 2023-07-18T03:39:49.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.626+0000 7f9dd939a700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9dd40d1e40 0x7f9dd40cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:49.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.626+0000 7f9dda39c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40c71b0 0x7f9dd40b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:49.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.626+0000 7f9dda39c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40c71b0 0x7f9dd40b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:40628/0 (socket says 172.21.15.49:40628) 2023-07-18T03:39:49.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.626+0000 7f9dda39c700 1 -- 172.21.15.49:0/3493790021 learned_addr learned my addr 172.21.15.49:0/3493790021 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:49.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.626+0000 7f9dd939a700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9dd40d1e40 0x7f9dd40cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:60700/0 (socket says 172.21.15.49:60700) 2023-07-18T03:39:49.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.626+0000 7f9dd9b9b700 1 --1- 172.21.15.49:0/3493790021 >> v1:172.21.15.107:6789/0 conn(0x7f9dd415c860 0x7f9dd40c6aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:37740/0 (socket says 172.21.15.49:37740) 2023-07-18T03:39:49.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.626+0000 7f9dda39c700 1 -- 172.21.15.49:0/3493790021 >> v2:172.21.15.107:3300/0 conn(0x7f9dd40d1e40 msgr2=0x7f9dd40cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:49.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.626+0000 7f9dda39c700 1 --2- 172.21.15.49:0/3493790021 >> v2:172.21.15.107:3300/0 conn(0x7f9dd40d1e40 0x7f9dd40cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.626+0000 7f9dda39c700 1 -- 172.21.15.49:0/3493790021 >> v1:172.21.15.107:6789/0 conn(0x7f9dd415c860 legacy=0x7f9dd40c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.626+0000 7f9dda39c700 1 -- 172.21.15.49:0/3493790021 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd40cdad0 con 0x7f9dd40c71b0 2023-07-18T03:39:49.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.626+0000 7f9dda39c700 1 --2- 172.21.15.49:0/3493790021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40c71b0 0x7f9dd40b49d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009e50 tx=0x7f9dcc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d1dc0f8e459f4f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:49.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.627+0000 7f9dd8b99700 1 -- 172.21.15.49:0/3493790021 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc00e070 con 0x7f9dd40c71b0 2023-07-18T03:39:49.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.627+0000 7f9dd8b99700 1 -- 172.21.15.49:0/3493790021 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dcc00b910 con 0x7f9dd40c71b0 2023-07-18T03:39:49.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.627+0000 7f9dd8b99700 1 -- 172.21.15.49:0/3493790021 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc00bcc0 con 0x7f9dd40c71b0 2023-07-18T03:39:49.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.627+0000 7f9ddbdff700 1 -- 172.21.15.49:0/3493790021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40c71b0 msgr2=0x7f9dd40b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.627+0000 7f9ddbdff700 1 --2- 172.21.15.49:0/3493790021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40c71b0 0x7f9dd40b49d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009e50 tx=0x7f9dcc00b330 comp rx=0 tx=0).stop 2023-07-18T03:39:49.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.627+0000 7f9ddbdff700 1 -- 172.21.15.49:0/3493790021 shutdown_connections 2023-07-18T03:39:49.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.627+0000 7f9ddbdff700 1 --2- 172.21.15.49:0/3493790021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40c71b0 0x7f9dd40b49d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.627+0000 7f9ddbdff700 1 --2- 172.21.15.49:0/3493790021 >> v2:172.21.15.107:3300/0 conn(0x7f9dd40d1e40 0x7f9dd40cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.627+0000 7f9ddbdff700 1 -- 172.21.15.49:0/3493790021 >> 172.21.15.49:0/3493790021 conn(0x7f9dd4065bd0 msgr2=0x7f9dd4065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:49.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.627+0000 7f9ddbdff700 1 -- 172.21.15.49:0/3493790021 shutdown_connections 2023-07-18T03:39:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.628+0000 7f9ddbdff700 1 -- 172.21.15.49:0/3493790021 wait complete. 2023-07-18T03:39:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.629+0000 7f9ddbdff700 1 Processor -- start 2023-07-18T03:39:49.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.629+0000 7f9ddbdff700 1 -- start start 2023-07-18T03:39:49.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.630+0000 7f9ddbdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd40c71b0 0x7f9dd415d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:49.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.630+0000 7f9ddbdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40d1e40 0x7f9dd40ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:49.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.630+0000 7f9ddbdff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dd415c860 0x7f9dd4156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:49.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9ddbdff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd40b74f0 con 0x7f9dd415c860 2023-07-18T03:39:49.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9ddbdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd40b77d0 con 0x7f9dd40c71b0 2023-07-18T03:39:49.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9ddbdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd40b7ab0 con 0x7f9dd40d1e40 2023-07-18T03:39:49.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9dda39c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dd415c860 0x7f9dd4156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:49.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9dda39c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dd415c860 0x7f9dd4156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39946/0 (socket says 172.21.15.49:39946) 2023-07-18T03:39:49.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9dda39c700 1 -- 172.21.15.49:0/4269812234 learned_addr learned my addr 172.21.15.49:0/4269812234 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:49.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9dd939a700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40d1e40 0x7f9dd40ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:49.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9dd9b9b700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd40c71b0 0x7f9dd415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:49.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9dda39c700 1 -- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40d1e40 msgr2=0x7f9dd40ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9dda39c700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40d1e40 0x7f9dd40ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9dda39c700 1 -- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd40c71b0 msgr2=0x7f9dd415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9dda39c700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd40c71b0 0x7f9dd415d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9dda39c700 1 -- 172.21.15.49:0/4269812234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc009b30 con 0x7f9dd415c860 2023-07-18T03:39:49.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.631+0000 7f9dd939a700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40d1e40 0x7f9dd40ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:49.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.632+0000 7f9dda39c700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dd415c860 0x7f9dd4156470 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40cd930 tx=0x7f9dcc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:49.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.632+0000 7f9dca7fc700 1 -- 172.21.15.49:0/4269812234 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0048f0 con 0x7f9dd415c860 2023-07-18T03:39:49.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.632+0000 7f9dca7fc700 1 -- 172.21.15.49:0/4269812234 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dcc0099b0 con 0x7f9dd415c860 2023-07-18T03:39:49.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.632+0000 7f9dca7fc700 1 -- 172.21.15.49:0/4269812234 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc01c680 con 0x7f9dd415c860 2023-07-18T03:39:49.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.632+0000 7f9ddbdff700 1 -- 172.21.15.49:0/4269812234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd40b7d90 con 0x7f9dd415c860 2023-07-18T03:39:49.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.632+0000 7f9ddbdff700 1 -- 172.21.15.49:0/4269812234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd40b8350 con 0x7f9dd415c860 2023-07-18T03:39:49.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.633+0000 7f9ddbdff700 1 -- 172.21.15.49:0/4269812234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd4074350 con 0x7f9dd415c860 2023-07-18T03:39:49.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.634+0000 7f9dca7fc700 1 -- 172.21.15.49:0/4269812234 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9dcc01c7e0 con 0x7f9dd415c860 2023-07-18T03:39:49.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.635+0000 7f9dca7fc700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9db806ace0 0x7f9db806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:49.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.635+0000 7f9dca7fc700 1 -- 172.21.15.49:0/4269812234 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9dcc08ae60 con 0x7f9dd415c860 2023-07-18T03:39:49.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.636+0000 7f9dd9b9b700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9db806ace0 0x7f9db806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:49.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.637+0000 7f9dd9b9b700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9db806ace0 0x7f9db806d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0000f60 tx=0x7f9dd0007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:49.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.640+0000 7f9dca7fc700 1 -- 172.21.15.49:0/4269812234 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9dcc056250 con 0x7f9dd415c860 2023-07-18T03:39:49.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.929+0000 7f9ddbdff700 1 -- 172.21.15.49:0/4269812234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f9dd40cd040 con 0x7f9dd415c860 2023-07-18T03:39:49.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.933+0000 7f9dca7fc700 1 -- 172.21.15.49:0/4269812234 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9dcc059870 con 0x7f9dd415c860 2023-07-18T03:39:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.935+0000 7f9ddbdff700 1 -- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9db806ace0 msgr2=0x7f9db806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.935+0000 7f9ddbdff700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9db806ace0 0x7f9db806d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0000f60 tx=0x7f9dd0007ed0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.935+0000 7f9ddbdff700 1 -- 172.21.15.49:0/4269812234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dd415c860 msgr2=0x7f9dd4156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.935+0000 7f9ddbdff700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dd415c860 0x7f9dd4156470 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40cd930 tx=0x7f9dcc004ad0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.936+0000 7f9ddbdff700 1 -- 172.21.15.49:0/4269812234 shutdown_connections 2023-07-18T03:39:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.936+0000 7f9ddbdff700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9db806ace0 0x7f9db806d190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.936+0000 7f9ddbdff700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dd415c860 0x7f9dd4156470 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.936+0000 7f9ddbdff700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd40c71b0 0x7f9dd415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.936+0000 7f9ddbdff700 1 --2- 172.21.15.49:0/4269812234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dd40d1e40 0x7f9dd40ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.936+0000 7f9ddbdff700 1 -- 172.21.15.49:0/4269812234 >> 172.21.15.49:0/4269812234 conn(0x7f9dd4065bd0 msgr2=0x7f9dd4159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:49.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.936+0000 7f9ddbdff700 1 -- 172.21.15.49:0/4269812234 shutdown_connections 2023-07-18T03:39:49.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:49.936+0000 7f9ddbdff700 1 -- 172.21.15.49:0/4269812234 wait complete. 2023-07-18T03:39:49.949 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-07-18T03:39:50.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.161+0000 7f48befff700 1 Processor -- start 2023-07-18T03:39:50.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.161+0000 7f48befff700 1 -- start start 2023-07-18T03:39:50.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.162+0000 7f48befff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b815c860 0x7f48b80d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:50.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.162+0000 7f48befff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f48b80c71b0 0x7f48b80b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:50.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.162+0000 7f48befff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b80cf710 con 0x7f48b80cf2a0 2023-07-18T03:39:50.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.162+0000 7f48befff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b8146740 con 0x7f48b80c71b0 2023-07-18T03:39:50.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.162+0000 7f48befff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80cd5a0 con 0x7f48b815c860 2023-07-18T03:39:50.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.163+0000 7f48bcd9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b815c860 0x7f48b80d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:50.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.163+0000 7f48bd59c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f48b80c71b0 0x7f48b80b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:50.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.163+0000 7f48bcd9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b815c860 0x7f48b80d1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:40646/0 (socket says 172.21.15.49:40646) 2023-07-18T03:39:50.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.163+0000 7f48bcd9b700 1 -- 172.21.15.49:0/2347884008 learned_addr learned my addr 172.21.15.49:0/2347884008 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:50.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.163+0000 7f48b7fff700 1 --1- 172.21.15.49:0/2347884008 >> v1:172.21.15.107:6789/0 conn(0x7f48b80cf2a0 0x7f48b80c6aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:37744/0 (socket says 172.21.15.49:37744) 2023-07-18T03:39:50.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.163+0000 7f48b77fe700 1 -- 172.21.15.49:0/2347884008 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834526606 0 0) 0x7f48a4001110 con 0x7f48b80cf2a0 2023-07-18T03:39:50.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.163+0000 7f48b77fe700 1 -- 172.21.15.49:0/2347884008 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48a80035b0 con 0x7f48b80cf2a0 2023-07-18T03:39:50.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.163+0000 7f48bcd9b700 1 -- 172.21.15.49:0/2347884008 >> v2:172.21.15.107:3300/0 conn(0x7f48b80c71b0 msgr2=0x7f48b80b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:50.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.163+0000 7f48bcd9b700 1 --2- 172.21.15.49:0/2347884008 >> v2:172.21.15.107:3300/0 conn(0x7f48b80c71b0 0x7f48b80b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:50.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.163+0000 7f48bcd9b700 1 -- 172.21.15.49:0/2347884008 >> v1:172.21.15.107:6789/0 conn(0x7f48b80cf2a0 legacy=0x7f48b80c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:50.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.163+0000 7f48bcd9b700 1 -- 172.21.15.49:0/2347884008 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b80cd7f0 con 0x7f48b815c860 2023-07-18T03:39:50.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.163+0000 7f48bd59c700 1 --2- 172.21.15.49:0/2347884008 >> v2:172.21.15.107:3300/0 conn(0x7f48b80c71b0 0x7f48b80b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:50.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.164+0000 7f48bcd9b700 1 --2- 172.21.15.49:0/2347884008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b815c860 0x7f48b80d1e40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48ac0098f0 tx=0x7f48ac009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb833a003aa087b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:50.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.164+0000 7f48b77fe700 1 -- 172.21.15.49:0/2347884008 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac011830 con 0x7f48b815c860 2023-07-18T03:39:50.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.164+0000 7f48b77fe700 1 -- 172.21.15.49:0/2347884008 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac011990 con 0x7f48b815c860 2023-07-18T03:39:50.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.164+0000 7f48b77fe700 1 -- 172.21.15.49:0/2347884008 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac011d20 con 0x7f48b815c860 2023-07-18T03:39:50.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.164+0000 7f48befff700 1 -- 172.21.15.49:0/2347884008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b815c860 msgr2=0x7f48b80d1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:50.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.164+0000 7f48befff700 1 --2- 172.21.15.49:0/2347884008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b815c860 0x7f48b80d1e40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48ac0098f0 tx=0x7f48ac009c00 comp rx=0 tx=0).stop 2023-07-18T03:39:50.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.165+0000 7f48befff700 1 -- 172.21.15.49:0/2347884008 shutdown_connections 2023-07-18T03:39:50.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.165+0000 7f48befff700 1 --2- 172.21.15.49:0/2347884008 >> v2:172.21.15.107:3300/0 conn(0x7f48b80c71b0 0x7f48b80b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:50.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.165+0000 7f48befff700 1 --2- 172.21.15.49:0/2347884008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b815c860 0x7f48b80d1e40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:50.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.165+0000 7f48befff700 1 -- 172.21.15.49:0/2347884008 >> 172.21.15.49:0/2347884008 conn(0x7f48b8065bd0 msgr2=0x7f48b8065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:50.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.165+0000 7f48befff700 1 -- 172.21.15.49:0/2347884008 shutdown_connections 2023-07-18T03:39:50.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.165+0000 7f48befff700 1 -- 172.21.15.49:0/2347884008 wait complete. 2023-07-18T03:39:50.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.166+0000 7f48befff700 1 Processor -- start 2023-07-18T03:39:50.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.167+0000 7f48befff700 1 -- start start 2023-07-18T03:39:50.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.167+0000 7f48befff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80c71b0 0x7f48b81e6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:50.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.167+0000 7f48befff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b80cf2a0 0x7f48b81f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:50.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.167+0000 7f48befff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48b815c860 0x7f48b80c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:50.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.167+0000 7f48befff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b81f2f10 con 0x7f48b80c71b0 2023-07-18T03:39:50.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.167+0000 7f48befff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b81e3c40 con 0x7f48b815c860 2023-07-18T03:39:50.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.167+0000 7f48befff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b81efb90 con 0x7f48b80cf2a0 2023-07-18T03:39:50.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48bcd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80c71b0 0x7f48b81e6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:50.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48bcd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80c71b0 0x7f48b81e6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39958/0 (socket says 172.21.15.49:39958) 2023-07-18T03:39:50.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48bcd9b700 1 -- 172.21.15.49:0/2259515117 learned_addr learned my addr 172.21.15.49:0/2259515117 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:50.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48bd59c700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48b815c860 0x7f48b80c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:50.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48b7fff700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b80cf2a0 0x7f48b81f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:50.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48bcd9b700 1 -- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b80cf2a0 msgr2=0x7f48b81f29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:50.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48bcd9b700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b80cf2a0 0x7f48b81f29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:50.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48bcd9b700 1 -- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48b815c860 msgr2=0x7f48b80c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:50.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48bcd9b700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48b815c860 0x7f48b80c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:50.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48bcd9b700 1 -- 172.21.15.49:0/2259515117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b0009a70 con 0x7f48b80c71b0 2023-07-18T03:39:50.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48bd59c700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48b815c860 0x7f48b80c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:50.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48b7fff700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b80cf2a0 0x7f48b81f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:50.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.168+0000 7f48bcd9b700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80c71b0 0x7f48b81e6b20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009c40 tx=0x7f48ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:50.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.169+0000 7f48b57fa700 1 -- 172.21.15.49:0/2259515117 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac004480 con 0x7f48b80c71b0 2023-07-18T03:39:50.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.169+0000 7f48b57fa700 1 -- 172.21.15.49:0/2259515117 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac01f070 con 0x7f48b80c71b0 2023-07-18T03:39:50.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.169+0000 7f48befff700 1 -- 172.21.15.49:0/2259515117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac009550 con 0x7f48b80c71b0 2023-07-18T03:39:50.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.169+0000 7f48b57fa700 1 -- 172.21.15.49:0/2259515117 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac0038c0 con 0x7f48b80c71b0 2023-07-18T03:39:50.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.169+0000 7f48befff700 1 -- 172.21.15.49:0/2259515117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b81e4210 con 0x7f48b80c71b0 2023-07-18T03:39:50.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.170+0000 7f48befff700 1 -- 172.21.15.49:0/2259515117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b81efd90 con 0x7f48b80c71b0 2023-07-18T03:39:50.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.171+0000 7f48b57fa700 1 -- 172.21.15.49:0/2259515117 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f48ac021030 con 0x7f48b80c71b0 2023-07-18T03:39:50.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.172+0000 7f48b57fa700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f489c06ace0 0x7f489c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:50.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.172+0000 7f48b57fa700 1 -- 172.21.15.49:0/2259515117 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f48ac08a540 con 0x7f48b80c71b0 2023-07-18T03:39:50.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.173+0000 7f48b7fff700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f489c06ace0 0x7f489c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:50.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.173+0000 7f48b7fff700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f489c06ace0 0x7f489c06d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f48a4007fd0 tx=0x7f48a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:50.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.177+0000 7f48b57fa700 1 -- 172.21.15.49:0/2259515117 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f48b81efd90 con 0x7f48b80c71b0 2023-07-18T03:39:50.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-07-18T03:39:50.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.470+0000 7f48befff700 1 -- 172.21.15.49:0/2259515117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f48b81efd60 con 0x7f48b80c71b0 2023-07-18T03:39:50.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.474+0000 7f48b57fa700 1 -- 172.21.15.49:0/2259515117 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f48ac0559e0 con 0x7f48b80c71b0 2023-07-18T03:39:50.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.476+0000 7f48befff700 1 -- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f489c06ace0 msgr2=0x7f489c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:50.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.476+0000 7f48befff700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f489c06ace0 0x7f489c06d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f48a4007fd0 tx=0x7f48a4009040 comp rx=0 tx=0).stop 2023-07-18T03:39:50.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.476+0000 7f48befff700 1 -- 172.21.15.49:0/2259515117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80c71b0 msgr2=0x7f48b81e6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:50.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.476+0000 7f48befff700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80c71b0 0x7f48b81e6b20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009c40 tx=0x7f48ac004750 comp rx=0 tx=0).stop 2023-07-18T03:39:50.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.477+0000 7f48befff700 1 -- 172.21.15.49:0/2259515117 shutdown_connections 2023-07-18T03:39:50.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.477+0000 7f48befff700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48b815c860 0x7f48b80c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:50.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.477+0000 7f48befff700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f489c06ace0 0x7f489c06d190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:50.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.477+0000 7f48befff700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80c71b0 0x7f48b81e6b20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:50.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.477+0000 7f48befff700 1 --2- 172.21.15.49:0/2259515117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f48b80cf2a0 0x7f48b81f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:50.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.477+0000 7f48befff700 1 -- 172.21.15.49:0/2259515117 >> 172.21.15.49:0/2259515117 conn(0x7f48b8065bd0 msgr2=0x7f48b8159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:50.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.477+0000 7f48befff700 1 -- 172.21.15.49:0/2259515117 shutdown_connections 2023-07-18T03:39:50.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.477+0000 7f48befff700 1 -- 172.21.15.49:0/2259515117 wait complete. 2023-07-18T03:39:50.489 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-07-18T03:39:50.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.699+0000 7f3910bff700 1 Processor -- start 2023-07-18T03:39:50.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.700+0000 7f3910bff700 1 -- start start 2023-07-18T03:39:50.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.701+0000 7f3910bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f390c0b4d00 0x7f390c0d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:50.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.701+0000 7f3910bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390c0d02a0 con 0x7f390c0b8660 2023-07-18T03:39:50.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.701+0000 7f3910bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390c0cfce0 con 0x7f390c0b4860 2023-07-18T03:39:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.701+0000 7f3910bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c0cffc0 con 0x7f390c0b4d00 2023-07-18T03:39:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.701+0000 7f390a59c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f390c0b8660 0x7f390c0cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37620/0 (socket says 172.21.15.49:37620) 2023-07-18T03:39:50.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.701+0000 7f390a59c700 1 -- 172.21.15.49:0/3662427011 learned_addr learned my addr 172.21.15.49:0/3662427011 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:50.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.701+0000 7f390ad9d700 1 --2- 172.21.15.49:0/3662427011 >> v2:172.21.15.107:3300/0 conn(0x7f390c0b4d00 0x7f390c0d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:50.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.702+0000 7f3909d9b700 1 -- 172.21.15.49:0/3662427011 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194744863 0 0) 0x7f390c0d02a0 con 0x7f390c0b8660 2023-07-18T03:39:50.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.702+0000 7f3909d9b700 1 -- 172.21.15.49:0/3662427011 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f80035b0 con 0x7f390c0b8660 2023-07-18T03:39:50.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.702+0000 7f3909d9b700 1 -- 172.21.15.49:0/3662427011 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425444494 0 0) 0x7f390c0cfce0 con 0x7f390c0b4860 2023-07-18T03:39:50.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.702+0000 7f3909d9b700 1 -- 172.21.15.49:0/3662427011 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f390c0d02a0 con 0x7f390c0b4860 2023-07-18T03:39:50.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.702+0000 7f3909d9b700 1 -- 172.21.15.49:0/3662427011 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3823169827 0 0) 0x7f38f80035b0 con 0x7f390c0b8660 2023-07-18T03:39:50.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.702+0000 7f390ad9d700 1 -- 172.21.15.49:0/3662427011 >> v1:172.21.15.107:6789/0 conn(0x7f390c0b4860 legacy=0x7f390c0d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:50.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.702+0000 7f390ad9d700 1 -- 172.21.15.49:0/3662427011 >> v1:172.21.15.49:6789/0 conn(0x7f390c0b8660 legacy=0x7f390c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:50.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.702+0000 7f390ad9d700 1 -- 172.21.15.49:0/3662427011 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0d0ac0 con 0x7f390c0b4d00 2023-07-18T03:39:50.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.702+0000 7f390ad9d700 1 --2- 172.21.15.49:0/3662427011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f390c0b4d00 0x7f390c0d19d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3904009d90 tx=0x7f390400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=296d364f5c5d0f46 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:50.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.702+0000 7f3909d9b700 1 -- 172.21.15.49:0/3662427011 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390400e070 con 0x7f390c0b4d00 2023-07-18T03:39:50.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.703+0000 7f3909d9b700 1 -- 172.21.15.49:0/3662427011 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3904003a20 con 0x7f390c0b4d00 2023-07-18T03:39:50.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.703+0000 7f3909d9b700 1 -- 172.21.15.49:0/3662427011 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc001670 con 0x7f390c0b4d00 2023-07-18T03:39:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.703+0000 7f3910bff700 1 -- 172.21.15.49:0/3662427011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f390c0b4d00 msgr2=0x7f390c0d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.703+0000 7f3910bff700 1 --2- 172.21.15.49:0/3662427011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f390c0b4d00 0x7f390c0d19d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3904009d90 tx=0x7f390400b330 comp rx=0 tx=0).stop 2023-07-18T03:39:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.703+0000 7f3910bff700 1 -- 172.21.15.49:0/3662427011 shutdown_connections 2023-07-18T03:39:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.703+0000 7f3910bff700 1 --2- 172.21.15.49:0/3662427011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f390c0b4d00 0x7f390c0d19d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.703+0000 7f3910bff700 1 -- 172.21.15.49:0/3662427011 >> 172.21.15.49:0/3662427011 conn(0x7f390c066860 msgr2=0x7f390c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:50.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.703+0000 7f3910bff700 1 -- 172.21.15.49:0/3662427011 shutdown_connections 2023-07-18T03:39:50.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.703+0000 7f3910bff700 1 -- 172.21.15.49:0/3662427011 wait complete. 2023-07-18T03:39:50.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.705+0000 7f3910bff700 1 Processor -- start 2023-07-18T03:39:50.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.705+0000 7f3910bff700 1 -- start start 2023-07-18T03:39:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.706+0000 7f3910bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f390c0b4860 0x7f390c0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.706+0000 7f3910bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390c0b4d00 0x7f390c14d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.706+0000 7f3910bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f390c0b8660 0x7f390c0c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.706+0000 7f3910bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c157d90 con 0x7f390c0b4d00 2023-07-18T03:39:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.706+0000 7f3910bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0cb050 con 0x7f390c0b8660 2023-07-18T03:39:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.706+0000 7f3910bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0ca7b0 con 0x7f390c0b4860 2023-07-18T03:39:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.706+0000 7f3903fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390c0b4d00 0x7f390c14d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:50.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.706+0000 7f3903fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390c0b4d00 0x7f390c14d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39968/0 (socket says 172.21.15.49:39968) 2023-07-18T03:39:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.706+0000 7f3903fff700 1 -- 172.21.15.49:0/3127061846 learned_addr learned my addr 172.21.15.49:0/3127061846 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.706+0000 7f390ad9d700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f390c0b8660 0x7f390c0c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.706+0000 7f390a59c700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f390c0b4860 0x7f390c0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.707+0000 7f3903fff700 1 -- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f390c0b4860 msgr2=0x7f390c0cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.707+0000 7f3903fff700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f390c0b4860 0x7f390c0cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.707+0000 7f3903fff700 1 -- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f390c0b8660 msgr2=0x7f390c0c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:50.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.707+0000 7f3903fff700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f390c0b8660 0x7f390c0c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:50.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.707+0000 7f3903fff700 1 -- 172.21.15.49:0/3127061846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3904009a70 con 0x7f390c0b4d00 2023-07-18T03:39:50.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.707+0000 7f3903fff700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390c0b4d00 0x7f390c14d190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f38f400c520 tx=0x7f38f400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:50.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.707+0000 7f3902ffd700 1 -- 172.21.15.49:0/3127061846 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f40052d0 con 0x7f390c0b4d00 2023-07-18T03:39:50.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.707+0000 7f3902ffd700 1 -- 172.21.15.49:0/3127061846 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4005430 con 0x7f390c0b4d00 2023-07-18T03:39:50.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.707+0000 7f3902ffd700 1 -- 172.21.15.49:0/3127061846 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f40038c0 con 0x7f390c0b4d00 2023-07-18T03:39:50.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.707+0000 7f3910bff700 1 -- 172.21.15.49:0/3127061846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c0caa90 con 0x7f390c0b4d00 2023-07-18T03:39:50.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.707+0000 7f3910bff700 1 -- 172.21.15.49:0/3127061846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c14d6d0 con 0x7f390c0b4d00 2023-07-18T03:39:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.708+0000 7f3910bff700 1 -- 172.21.15.49:0/3127061846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c157f00 con 0x7f390c0b4d00 2023-07-18T03:39:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.710+0000 7f3902ffd700 1 -- 172.21.15.49:0/3127061846 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38f4003a20 con 0x7f390c0b4d00 2023-07-18T03:39:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.710+0000 7f3902ffd700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f38ec06ad30 0x7f38ec06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:50.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.711+0000 7f3902ffd700 1 -- 172.21.15.49:0/3127061846 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f38f4088510 con 0x7f390c0b4d00 2023-07-18T03:39:50.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.711+0000 7f390a59c700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f38ec06ad30 0x7f38ec06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:50.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.712+0000 7f390a59c700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f38ec06ad30 0x7f38ec06d1e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0017a0 tx=0x7f38fc001670 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:50.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:50.715+0000 7f3902ffd700 1 -- 172.21.15.49:0/3127061846 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f38f40539f0 con 0x7f390c0b4d00 2023-07-18T03:39:51.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.010+0000 7f3910bff700 1 -- 172.21.15.49:0/3127061846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f390c074350 con 0x7f390c0b4d00 2023-07-18T03:39:51.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.014+0000 7f3902ffd700 1 -- 172.21.15.49:0/3127061846 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f38f4057010 con 0x7f390c0b4d00 2023-07-18T03:39:51.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.016+0000 7f3910bff700 1 -- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f38ec06ad30 msgr2=0x7f38ec06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:51.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.016+0000 7f3910bff700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f38ec06ad30 0x7f38ec06d1e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0017a0 tx=0x7f38fc001670 comp rx=0 tx=0).stop 2023-07-18T03:39:51.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.016+0000 7f3910bff700 1 -- 172.21.15.49:0/3127061846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390c0b4d00 msgr2=0x7f390c14d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.016+0000 7f3910bff700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390c0b4d00 0x7f390c14d190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f38f400c520 tx=0x7f38f400c830 comp rx=0 tx=0).stop 2023-07-18T03:39:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.017+0000 7f3910bff700 1 -- 172.21.15.49:0/3127061846 shutdown_connections 2023-07-18T03:39:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.017+0000 7f3910bff700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f390c0b8660 0x7f390c0c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.017+0000 7f3910bff700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f390c0b4860 0x7f390c0cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.017+0000 7f3910bff700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f38ec06ad30 0x7f38ec06d1e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.017+0000 7f3910bff700 1 --2- 172.21.15.49:0/3127061846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f390c0b4d00 0x7f390c14d190 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.017+0000 7f3910bff700 1 -- 172.21.15.49:0/3127061846 >> 172.21.15.49:0/3127061846 conn(0x7f390c066860 msgr2=0x7f390c0b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:51.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.017+0000 7f3910bff700 1 -- 172.21.15.49:0/3127061846 shutdown_connections 2023-07-18T03:39:51.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.017+0000 7f3910bff700 1 -- 172.21.15.49:0/3127061846 wait complete. 2023-07-18T03:39:51.029 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-07-18T03:39:51.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.241+0000 7f679afff700 1 Processor -- start 2023-07-18T03:39:51.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.241+0000 7f679afff700 1 -- start start 2023-07-18T03:39:51.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.243+0000 7f679afff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67940c92d0 0x7f67940ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:51.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.243+0000 7f679afff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f67940adea0 0x7f67940cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:51.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.243+0000 7f679afff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679415ce70 con 0x7f67940b12f0 2023-07-18T03:39:51.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.243+0000 7f679afff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679415da60 con 0x7f67940adea0 2023-07-18T03:39:51.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.243+0000 7f679afff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679415d0c0 con 0x7f67940c92d0 2023-07-18T03:39:51.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.243+0000 7f6793fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f67940b12f0 0x7f67940c96a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37628/0 (socket says 172.21.15.49:37628) 2023-07-18T03:39:51.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.243+0000 7f6793fff700 1 -- 172.21.15.49:0/3433306496 learned_addr learned my addr 172.21.15.49:0/3433306496 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:51.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.243+0000 7f679959c700 1 --2- 172.21.15.49:0/3433306496 >> v2:172.21.15.49:3300/0 conn(0x7f67940adea0 0x7f67940cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.243+0000 7f6798d9b700 1 --2- 172.21.15.49:0/3433306496 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67940c92d0 0x7f67940ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.243+0000 7f67937fe700 1 -- 172.21.15.49:0/3433306496 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003869191 0 0) 0x7f679415ce70 con 0x7f67940b12f0 2023-07-18T03:39:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.243+0000 7f67937fe700 1 -- 172.21.15.49:0/3433306496 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67840057c0 con 0x7f67940b12f0 2023-07-18T03:39:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.244+0000 7f679959c700 1 -- 172.21.15.49:0/3433306496 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67940c92d0 msgr2=0x7f67940ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.244+0000 7f679959c700 1 --2- 172.21.15.49:0/3433306496 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67940c92d0 0x7f67940ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.244+0000 7f679959c700 1 -- 172.21.15.49:0/3433306496 >> v1:172.21.15.49:6789/0 conn(0x7f67940b12f0 legacy=0x7f67940c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.244+0000 7f679959c700 1 -- 172.21.15.49:0/3433306496 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679415d3a0 con 0x7f67940adea0 2023-07-18T03:39:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.244+0000 7f6798d9b700 1 --2- 172.21.15.49:0/3433306496 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67940c92d0 0x7f67940ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.244+0000 7f679959c700 1 --2- 172.21.15.49:0/3433306496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67940adea0 0x7f67940cc690 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f678c009d90 tx=0x7f678c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffd2bd54c9d4eae server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.244+0000 7f67937fe700 1 -- 172.21.15.49:0/3433306496 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c00e070 con 0x7f67940adea0 2023-07-18T03:39:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.244+0000 7f67937fe700 1 -- 172.21.15.49:0/3433306496 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c00bb20 con 0x7f67940adea0 2023-07-18T03:39:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.244+0000 7f67937fe700 1 -- 172.21.15.49:0/3433306496 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c012430 con 0x7f67940adea0 2023-07-18T03:39:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.245+0000 7f679afff700 1 -- 172.21.15.49:0/3433306496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67940adea0 msgr2=0x7f67940cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.245+0000 7f679afff700 1 --2- 172.21.15.49:0/3433306496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67940adea0 0x7f67940cc690 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f678c009d90 tx=0x7f678c00b330 comp rx=0 tx=0).stop 2023-07-18T03:39:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.245+0000 7f679afff700 1 -- 172.21.15.49:0/3433306496 shutdown_connections 2023-07-18T03:39:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.245+0000 7f679afff700 1 --2- 172.21.15.49:0/3433306496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67940adea0 0x7f67940cc690 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.245+0000 7f679afff700 1 --2- 172.21.15.49:0/3433306496 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67940c92d0 0x7f67940ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.245+0000 7f679afff700 1 -- 172.21.15.49:0/3433306496 >> 172.21.15.49:0/3433306496 conn(0x7f67940b05d0 msgr2=0x7f67940b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.245+0000 7f679afff700 1 -- 172.21.15.49:0/3433306496 shutdown_connections 2023-07-18T03:39:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.245+0000 7f679afff700 1 -- 172.21.15.49:0/3433306496 wait complete. 2023-07-18T03:39:51.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.247+0000 7f679afff700 1 Processor -- start 2023-07-18T03:39:51.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.247+0000 7f679afff700 1 -- start start 2023-07-18T03:39:51.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.248+0000 7f679afff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67940adea0 0x7f6794070a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:51.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.248+0000 7f679afff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67940b12f0 0x7f6794152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.248+0000 7f679afff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67940c92d0 0x7f67940cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.248+0000 7f679afff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67941f2f70 con 0x7f67940c92d0 2023-07-18T03:39:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.248+0000 7f679afff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67941f23f0 con 0x7f67940adea0 2023-07-18T03:39:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.248+0000 7f679afff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67941f26d0 con 0x7f67940b12f0 2023-07-18T03:39:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.248+0000 7f679959c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67940c92d0 0x7f67940cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.248+0000 7f679959c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67940c92d0 0x7f67940cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39980/0 (socket says 172.21.15.49:39980) 2023-07-18T03:39:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.248+0000 7f679959c700 1 -- 172.21.15.49:0/2323489839 learned_addr learned my addr 172.21.15.49:0/2323489839 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:51.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.248+0000 7f6793fff700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67940b12f0 0x7f6794152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:51.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.248+0000 7f6798d9b700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67940adea0 0x7f6794070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:51.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.249+0000 7f679959c700 1 -- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67940b12f0 msgr2=0x7f6794152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:51.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.249+0000 7f679959c700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67940b12f0 0x7f6794152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.249+0000 7f679959c700 1 -- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67940adea0 msgr2=0x7f6794070a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:51.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.249+0000 7f679959c700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67940adea0 0x7f6794070a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.249+0000 7f679959c700 1 -- 172.21.15.49:0/2323489839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c009a70 con 0x7f67940c92d0 2023-07-18T03:39:51.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.249+0000 7f679959c700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67940c92d0 0x7f67940cb2c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f678c014040 tx=0x7f678c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:51.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.249+0000 7f67917fa700 1 -- 172.21.15.49:0/2323489839 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c012430 con 0x7f67940c92d0 2023-07-18T03:39:51.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.249+0000 7f67917fa700 1 -- 172.21.15.49:0/2323489839 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c020070 con 0x7f67940c92d0 2023-07-18T03:39:51.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.249+0000 7f679afff700 1 -- 172.21.15.49:0/2323489839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67941f29b0 con 0x7f67940c92d0 2023-07-18T03:39:51.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.249+0000 7f67917fa700 1 -- 172.21.15.49:0/2323489839 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c003ea0 con 0x7f67940c92d0 2023-07-18T03:39:51.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.250+0000 7f679afff700 1 -- 172.21.15.49:0/2323489839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67940ad1f0 con 0x7f67940c92d0 2023-07-18T03:39:51.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.251+0000 7f679afff700 1 -- 172.21.15.49:0/2323489839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6794153090 con 0x7f67940c92d0 2023-07-18T03:39:51.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.251+0000 7f67917fa700 1 -- 172.21.15.49:0/2323489839 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f678c004040 con 0x7f67940c92d0 2023-07-18T03:39:51.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.253+0000 7f67917fa700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f677806ace0 0x7f677806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:51.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.253+0000 7f67917fa700 1 -- 172.21.15.49:0/2323489839 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f678c022070 con 0x7f67940c92d0 2023-07-18T03:39:51.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.253+0000 7f6798d9b700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f677806ace0 0x7f677806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:51.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.254+0000 7f6798d9b700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f677806ace0 0x7f677806d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f679415cd80 tx=0x7f6788008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:51.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.257+0000 7f67917fa700 1 -- 172.21.15.49:0/2323489839 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6794153090 con 0x7f67940c92d0 2023-07-18T03:39:51.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.546+0000 7f679afff700 1 -- 172.21.15.49:0/2323489839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f6794153090 con 0x7f67940c92d0 2023-07-18T03:39:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.550+0000 7f67917fa700 1 -- 172.21.15.49:0/2323489839 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6794153090 con 0x7f67940c92d0 2023-07-18T03:39:51.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.552+0000 7f679afff700 1 -- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f677806ace0 msgr2=0x7f677806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:51.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.552+0000 7f679afff700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f677806ace0 0x7f677806d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f679415cd80 tx=0x7f6788008040 comp rx=0 tx=0).stop 2023-07-18T03:39:51.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.552+0000 7f679afff700 1 -- 172.21.15.49:0/2323489839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67940c92d0 msgr2=0x7f67940cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:51.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.552+0000 7f679afff700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67940c92d0 0x7f67940cb2c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f678c014040 tx=0x7f678c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.553+0000 7f679afff700 1 -- 172.21.15.49:0/2323489839 shutdown_connections 2023-07-18T03:39:51.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.553+0000 7f679afff700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f677806ace0 0x7f677806d190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.553+0000 7f679afff700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67940c92d0 0x7f67940cb2c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.553+0000 7f679afff700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67940adea0 0x7f6794070a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.553+0000 7f679afff700 1 --2- 172.21.15.49:0/2323489839 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67940b12f0 0x7f6794152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.553+0000 7f679afff700 1 -- 172.21.15.49:0/2323489839 >> 172.21.15.49:0/2323489839 conn(0x7f67940b05d0 msgr2=0x7f679406cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:51.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.553+0000 7f679afff700 1 -- 172.21.15.49:0/2323489839 shutdown_connections 2023-07-18T03:39:51.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.553+0000 7f679afff700 1 -- 172.21.15.49:0/2323489839 wait complete. 2023-07-18T03:39:51.565 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-07-18T03:39:51.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.776+0000 7f8e919ff700 1 Processor -- start 2023-07-18T03:39:51.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.776+0000 7f8e919ff700 1 -- start start 2023-07-18T03:39:51.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.777+0000 7f8e919ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e8c0b8a60 0x7f8e8c0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:51.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.777+0000 7f8e919ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8e8c0b4530 0x7f8e8c0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:51.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.778+0000 7f8e919ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8c0b8e30 con 0x7f8e8c0d1050 2023-07-18T03:39:51.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.778+0000 7f8e919ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8c0d14d0 con 0x7f8e8c0b4530 2023-07-18T03:39:51.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.778+0000 7f8e919ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0b1e30 con 0x7f8e8c0b8a60 2023-07-18T03:39:51.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.778+0000 7f8e8b7fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8e8c0d1050 0x7f8e8c0b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37636/0 (socket says 172.21.15.49:37636) 2023-07-18T03:39:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.778+0000 7f8e8b7fe700 1 -- 172.21.15.49:0/3549650748 learned_addr learned my addr 172.21.15.49:0/3549650748 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.778+0000 7f8e8a7fc700 1 --2- 172.21.15.49:0/3549650748 >> v2:172.21.15.107:3300/0 conn(0x7f8e8c0b4530 0x7f8e8c0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.778+0000 7f8e8affd700 1 --2- 172.21.15.49:0/3549650748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e8c0b8a60 0x7f8e8c0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.778+0000 7f8e89ffb700 1 -- 172.21.15.49:0/3549650748 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043855071 0 0) 0x7f8e8c0b8e30 con 0x7f8e8c0d1050 2023-07-18T03:39:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.778+0000 7f8e89ffb700 1 -- 172.21.15.49:0/3549650748 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e740035b0 con 0x7f8e8c0d1050 2023-07-18T03:39:51.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.779+0000 7f8e8a7fc700 1 -- 172.21.15.49:0/3549650748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e8c0b8a60 msgr2=0x7f8e8c0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.779+0000 7f8e8a7fc700 1 --2- 172.21.15.49:0/3549650748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e8c0b8a60 0x7f8e8c0b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.779+0000 7f8e89ffb700 1 -- 172.21.15.49:0/3549650748 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2255645910 0 0) 0x7f8e740035b0 con 0x7f8e8c0d1050 2023-07-18T03:39:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.779+0000 7f8e8a7fc700 1 -- 172.21.15.49:0/3549650748 >> v1:172.21.15.49:6789/0 conn(0x7f8e8c0d1050 legacy=0x7f8e8c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.779+0000 7f8e8a7fc700 1 -- 172.21.15.49:0/3549650748 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c0afaa0 con 0x7f8e8c0b4530 2023-07-18T03:39:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.779+0000 7f8e8affd700 1 --2- 172.21.15.49:0/3549650748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e8c0b8a60 0x7f8e8c0b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:51.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.779+0000 7f8e8a7fc700 1 --2- 172.21.15.49:0/3549650748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e8c0b4530 0x7f8e8c0d0b10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009840 tx=0x7f8e80009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97109e102bb288c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.779+0000 7f8e89ffb700 1 -- 172.21.15.49:0/3549650748 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e840019f0 con 0x7f8e8c0b4530 2023-07-18T03:39:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.779+0000 7f8e89ffb700 1 -- 172.21.15.49:0/3549650748 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e80004030 con 0x7f8e8c0b4530 2023-07-18T03:39:51.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.779+0000 7f8e89ffb700 1 -- 172.21.15.49:0/3549650748 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e800043c0 con 0x7f8e8c0b4530 2023-07-18T03:39:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.780+0000 7f8e919ff700 1 -- 172.21.15.49:0/3549650748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e8c0b4530 msgr2=0x7f8e8c0d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.780+0000 7f8e919ff700 1 --2- 172.21.15.49:0/3549650748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e8c0b4530 0x7f8e8c0d0b10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009840 tx=0x7f8e80009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.780+0000 7f8e919ff700 1 -- 172.21.15.49:0/3549650748 shutdown_connections 2023-07-18T03:39:51.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.780+0000 7f8e919ff700 1 --2- 172.21.15.49:0/3549650748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e8c0b8a60 0x7f8e8c0b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.780+0000 7f8e919ff700 1 --2- 172.21.15.49:0/3549650748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e8c0b4530 0x7f8e8c0d0b10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.780+0000 7f8e919ff700 1 -- 172.21.15.49:0/3549650748 >> 172.21.15.49:0/3549650748 conn(0x7f8e8c09c1b0 msgr2=0x7f8e8c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:51.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.780+0000 7f8e919ff700 1 -- 172.21.15.49:0/3549650748 shutdown_connections 2023-07-18T03:39:51.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.780+0000 7f8e919ff700 1 -- 172.21.15.49:0/3549650748 wait complete. 2023-07-18T03:39:51.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.782+0000 7f8e919ff700 1 Processor -- start 2023-07-18T03:39:51.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.782+0000 7f8e919ff700 1 -- start start 2023-07-18T03:39:51.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.783+0000 7f8e919ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e8c0b4530 0x7f8e8c0c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.783+0000 7f8e919ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c0b8a60 0x7f8e8c1f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.783+0000 7f8e919ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e8c0d1050 0x7f8e8c1f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.783+0000 7f8e919ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0cf7f0 con 0x7f8e8c0b8a60 2023-07-18T03:39:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.783+0000 7f8e919ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c1f2090 con 0x7f8e8c0b4530 2023-07-18T03:39:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.783+0000 7f8e919ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0b5d70 con 0x7f8e8c0d1050 2023-07-18T03:39:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.784+0000 7f8e8a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c0b8a60 0x7f8e8c1f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:51.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.784+0000 7f8e8a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c0b8a60 0x7f8e8c1f5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39996/0 (socket says 172.21.15.49:39996) 2023-07-18T03:39:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.784+0000 7f8e8a7fc700 1 -- 172.21.15.49:0/1692863225 learned_addr learned my addr 172.21.15.49:0/1692863225 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.784+0000 7f8e8affd700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e8c0b4530 0x7f8e8c0c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.784+0000 7f8e8b7fe700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e8c0d1050 0x7f8e8c1f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.784+0000 7f8e8a7fc700 1 -- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e8c0d1050 msgr2=0x7f8e8c1f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.784+0000 7f8e8a7fc700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e8c0d1050 0x7f8e8c1f5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.784+0000 7f8e8a7fc700 1 -- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e8c0b4530 msgr2=0x7f8e8c0c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.784+0000 7f8e8a7fc700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e8c0b4530 0x7f8e8c0c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:51.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.784+0000 7f8e8a7fc700 1 -- 172.21.15.49:0/1692863225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c0094a0 con 0x7f8e8c0b8a60 2023-07-18T03:39:51.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.784+0000 7f8e8a7fc700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c0b8a60 0x7f8e8c1f5750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000a010 tx=0x7f8e80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:51.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.785+0000 7f8e73fff700 1 -- 172.21.15.49:0/1692863225 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80003cf0 con 0x7f8e8c0b8a60 2023-07-18T03:39:51.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.785+0000 7f8e73fff700 1 -- 172.21.15.49:0/1692863225 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8001f070 con 0x7f8e8c0b8a60 2023-07-18T03:39:51.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.785+0000 7f8e73fff700 1 -- 172.21.15.49:0/1692863225 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80011870 con 0x7f8e8c0b8a60 2023-07-18T03:39:51.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.785+0000 7f8e919ff700 1 -- 172.21.15.49:0/1692863225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e800094a0 con 0x7f8e8c0b8a60 2023-07-18T03:39:51.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.785+0000 7f8e919ff700 1 -- 172.21.15.49:0/1692863225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8c0cfce0 con 0x7f8e8c0b8a60 2023-07-18T03:39:51.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.786+0000 7f8e919ff700 1 -- 172.21.15.49:0/1692863225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8c0b5f00 con 0x7f8e8c0b8a60 2023-07-18T03:39:51.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.787+0000 7f8e73fff700 1 -- 172.21.15.49:0/1692863225 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e800119d0 con 0x7f8e8c0b8a60 2023-07-18T03:39:51.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.788+0000 7f8e73fff700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8e7806ace0 0x7f8e7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:51.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.788+0000 7f8e73fff700 1 -- 172.21.15.49:0/1692863225 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8e80021070 con 0x7f8e8c0b8a60 2023-07-18T03:39:51.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.788+0000 7f8e8affd700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8e7806ace0 0x7f8e7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:51.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.789+0000 7f8e8affd700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8e7806ace0 0x7f8e7806d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009d70 tx=0x7f8e7c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:51.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:51.792+0000 7f8e73fff700 1 -- 172.21.15.49:0/1692863225 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8e80055e00 con 0x7f8e8c0b8a60 2023-07-18T03:39:52.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.087+0000 7f8e919ff700 1 -- 172.21.15.49:0/1692863225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f8e8c074350 con 0x7f8e8c0b8a60 2023-07-18T03:39:52.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.091+0000 7f8e73fff700 1 -- 172.21.15.49:0/1692863225 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8e80059420 con 0x7f8e8c0b8a60 2023-07-18T03:39:52.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.093+0000 7f8e919ff700 1 -- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8e7806ace0 msgr2=0x7f8e7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.093+0000 7f8e919ff700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8e7806ace0 0x7f8e7806d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009d70 tx=0x7f8e7c009210 comp rx=0 tx=0).stop 2023-07-18T03:39:52.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.093+0000 7f8e919ff700 1 -- 172.21.15.49:0/1692863225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c0b8a60 msgr2=0x7f8e8c1f5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.093+0000 7f8e919ff700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c0b8a60 0x7f8e8c1f5750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000a010 tx=0x7f8e80004750 comp rx=0 tx=0).stop 2023-07-18T03:39:52.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.093+0000 7f8e919ff700 1 -- 172.21.15.49:0/1692863225 shutdown_connections 2023-07-18T03:39:52.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.093+0000 7f8e919ff700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e8c0d1050 0x7f8e8c1f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.093+0000 7f8e919ff700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e8c0b4530 0x7f8e8c0c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.093+0000 7f8e919ff700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8e7806ace0 0x7f8e7806d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.093+0000 7f8e919ff700 1 --2- 172.21.15.49:0/1692863225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e8c0b8a60 0x7f8e8c1f5750 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.093+0000 7f8e919ff700 1 -- 172.21.15.49:0/1692863225 >> 172.21.15.49:0/1692863225 conn(0x7f8e8c09c1b0 msgr2=0x7f8e8c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:52.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.093+0000 7f8e919ff700 1 -- 172.21.15.49:0/1692863225 shutdown_connections 2023-07-18T03:39:52.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.093+0000 7f8e919ff700 1 -- 172.21.15.49:0/1692863225 wait complete. 2023-07-18T03:39:52.106 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:52.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.317+0000 7f5663fff700 1 Processor -- start 2023-07-18T03:39:52.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.317+0000 7f5663fff700 1 -- start start 2023-07-18T03:39:52.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.319+0000 7f5663fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f565c0b4510 0x7f565c0b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.319+0000 7f5663fff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565c0b7f50 con 0x7f565c0d0b10 2023-07-18T03:39:52.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.319+0000 7f5663fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565c0cdca0 con 0x7f565c0b5d90 2023-07-18T03:39:52.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.319+0000 7f5663fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565c0b48e0 con 0x7f565c0b4510 2023-07-18T03:39:52.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.319+0000 7f566259c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f565c0d0b10 0x7f565c0b83e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37640/0 (socket says 172.21.15.49:37640) 2023-07-18T03:39:52.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.319+0000 7f566259c700 1 -- 172.21.15.49:0/840242702 learned_addr learned my addr 172.21.15.49:0/840242702 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:52.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.319+0000 7f5661d9b700 1 --2- 172.21.15.49:0/840242702 >> v2:172.21.15.107:3300/0 conn(0x7f565c0b4510 0x7f565c0b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.319+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585314578 0 0) 0x7f565c0b7f50 con 0x7f565c0d0b10 2023-07-18T03:39:52.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.319+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56480035b0 con 0x7f565c0d0b10 2023-07-18T03:39:52.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2313612875 0 0) 0x7f56480035b0 con 0x7f565c0d0b10 2023-07-18T03:39:52.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 >> v2:172.21.15.107:3300/0 conn(0x7f565c0b4510 msgr2=0x7f565c0b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:52.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 --2- 172.21.15.49:0/840242702 >> v2:172.21.15.107:3300/0 conn(0x7f565c0b4510 0x7f565c0b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 >> v1:172.21.15.107:6789/0 conn(0x7f565c0b5d90 legacy=0x7f565c0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c0b7340 con 0x7f565c0d0b10 2023-07-18T03:39:52.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f56540019f0 con 0x7f565c0d0b10 2023-07-18T03:39:52.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 >> v1:172.21.15.49:6789/0 conn(0x7f565c0d0b10 legacy=0x7f565c0b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 --2- 172.21.15.49:0/840242702 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5648003a50 0x7f5648003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 --2- 172.21.15.49:0/840242702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5648004850 0x7f5648004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 --2- 172.21.15.49:0/840242702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56480052d0 0x7f564800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564800c1d0 con 0x7f56480052d0 2023-07-18T03:39:52.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564800c340 con 0x7f5648004850 2023-07-18T03:39:52.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564800c500 con 0x7f5648003a50 2023-07-18T03:39:52.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f566259c700 1 --2- 172.21.15.49:0/840242702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56480052d0 0x7f564800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f566159a700 1 --2- 172.21.15.49:0/840242702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5648004850 0x7f5648004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.320+0000 7f5661d9b700 1 --2- 172.21.15.49:0/840242702 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5648003a50 0x7f5648003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.321+0000 7f566259c700 1 -- 172.21.15.49:0/840242702 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5648003a50 msgr2=0x7f5648003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:52.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.321+0000 7f566259c700 1 --2- 172.21.15.49:0/840242702 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5648003a50 0x7f5648003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.321+0000 7f566259c700 1 -- 172.21.15.49:0/840242702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5648004850 msgr2=0x7f5648004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:52.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.321+0000 7f566259c700 1 --2- 172.21.15.49:0/840242702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5648004850 0x7f5648004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.321+0000 7f566259c700 1 -- 172.21.15.49:0/840242702 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56480035b0 con 0x7f56480052d0 2023-07-18T03:39:52.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.321+0000 7f566159a700 1 --2- 172.21.15.49:0/840242702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5648004850 0x7f5648004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:52.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.321+0000 7f566259c700 1 -- 172.21.15.49:0/840242702 reap_dead start 2023-07-18T03:39:52.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.321+0000 7f566259c700 1 --2- 172.21.15.49:0/840242702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56480052d0 0x7f564800ba10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f565400a760 tx=0x7f5648005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f2fc785b706383 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:52.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.321+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565400a250 con 0x7f56480052d0 2023-07-18T03:39:52.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.321+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56540072d0 con 0x7f56480052d0 2023-07-18T03:39:52.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.321+0000 7f5660d99700 1 -- 172.21.15.49:0/840242702 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654007680 con 0x7f56480052d0 2023-07-18T03:39:52.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.321+0000 7f5663fff700 1 -- 172.21.15.49:0/840242702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56480052d0 msgr2=0x7f564800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.322+0000 7f5663fff700 1 --2- 172.21.15.49:0/840242702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56480052d0 0x7f564800ba10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f565400a760 tx=0x7f5648005880 comp rx=0 tx=0).stop 2023-07-18T03:39:52.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.322+0000 7f5663fff700 1 -- 172.21.15.49:0/840242702 shutdown_connections 2023-07-18T03:39:52.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.322+0000 7f5663fff700 1 --2- 172.21.15.49:0/840242702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56480052d0 0x7f564800ba10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.322+0000 7f5663fff700 1 -- 172.21.15.49:0/840242702 >> 172.21.15.49:0/840242702 conn(0x7f565c09c1d0 msgr2=0x7f565c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:52.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.322+0000 7f5663fff700 1 -- 172.21.15.49:0/840242702 shutdown_connections 2023-07-18T03:39:52.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.322+0000 7f5663fff700 1 -- 172.21.15.49:0/840242702 wait complete. 2023-07-18T03:39:52.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.324+0000 7f5663fff700 1 Processor -- start 2023-07-18T03:39:52.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.324+0000 7f5663fff700 1 -- start start 2023-07-18T03:39:52.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f5663fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56480052d0 0x7f565c1f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f5663fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f565c0c9240 0x7f565c157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f5663fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f565c0ca010 0x7f565c0ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f5663fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0ca9d0 con 0x7f565c0ca010 2023-07-18T03:39:52.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f5663fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c1f3000 con 0x7f56480052d0 2023-07-18T03:39:52.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f5663fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0cb680 con 0x7f565c0c9240 2023-07-18T03:39:52.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f566259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f565c0ca010 0x7f565c0ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f566259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f565c0ca010 0x7f565c0ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40008/0 (socket says 172.21.15.49:40008) 2023-07-18T03:39:52.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f566259c700 1 -- 172.21.15.49:0/22939580 learned_addr learned my addr 172.21.15.49:0/22939580 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:52.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f5661d9b700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56480052d0 0x7f565c1f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f566159a700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f565c0c9240 0x7f565c157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f566259c700 1 -- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f565c0c9240 msgr2=0x7f565c157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f566259c700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f565c0c9240 0x7f565c157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f566259c700 1 -- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56480052d0 msgr2=0x7f565c1f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f566259c700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56480052d0 0x7f565c1f2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.325+0000 7f566259c700 1 -- 172.21.15.49:0/22939580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c0cb960 con 0x7f565c0ca010 2023-07-18T03:39:52.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.326+0000 7f566259c700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f565c0ca010 0x7f565c0ca490 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5654016040 tx=0x7f5654001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:52.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.326+0000 7f56527fc700 1 -- 172.21.15.49:0/22939580 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654025030 con 0x7f565c0ca010 2023-07-18T03:39:52.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.326+0000 7f56527fc700 1 -- 172.21.15.49:0/22939580 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5654007af0 con 0x7f565c0ca010 2023-07-18T03:39:52.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.326+0000 7f5663fff700 1 -- 172.21.15.49:0/22939580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565c0cbc40 con 0x7f565c0ca010 2023-07-18T03:39:52.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.326+0000 7f56527fc700 1 -- 172.21.15.49:0/22939580 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654012640 con 0x7f565c0ca010 2023-07-18T03:39:52.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.326+0000 7f5663fff700 1 -- 172.21.15.49:0/22939580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565c0cc200 con 0x7f565c0ca010 2023-07-18T03:39:52.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.327+0000 7f5663fff700 1 -- 172.21.15.49:0/22939580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565c071a00 con 0x7f565c0ca010 2023-07-18T03:39:52.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.328+0000 7f56527fc700 1 -- 172.21.15.49:0/22939580 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5654028070 con 0x7f565c0ca010 2023-07-18T03:39:52.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.329+0000 7f56527fc700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f564006ace0 0x7f564006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.329+0000 7f56527fc700 1 -- 172.21.15.49:0/22939580 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5654026070 con 0x7f565c0ca010 2023-07-18T03:39:52.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.330+0000 7f5661d9b700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f564006ace0 0x7f564006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.330+0000 7f5661d9b700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f564006ace0 0x7f564006d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f565c0c9c30 tx=0x7f5658008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:52.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.333+0000 7f56527fc700 1 -- 172.21.15.49:0/22939580 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5654055c50 con 0x7f565c0ca010 2023-07-18T03:39:52.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.542+0000 7f5663fff700 1 -- 172.21.15.49:0/22939580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f565c0cc3f0 con 0x7f565c0ca010 2023-07-18T03:39:52.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.543+0000 7f56527fc700 1 -- 172.21.15.49:0/22939580 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+14545 (secure 0 0 0) 0x7f5654059270 con 0x7f565c0ca010 2023-07-18T03:39:52.538 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:52.538 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":32,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:48.292782+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:52.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.545+0000 7f5663fff700 1 -- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f564006ace0 msgr2=0x7f564006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.545+0000 7f5663fff700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f564006ace0 0x7f564006d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f565c0c9c30 tx=0x7f5658008040 comp rx=0 tx=0).stop 2023-07-18T03:39:52.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.545+0000 7f5663fff700 1 -- 172.21.15.49:0/22939580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f565c0ca010 msgr2=0x7f565c0ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.545+0000 7f5663fff700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f565c0ca010 0x7f565c0ca490 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5654016040 tx=0x7f5654001cb0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.546+0000 7f5663fff700 1 -- 172.21.15.49:0/22939580 shutdown_connections 2023-07-18T03:39:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.546+0000 7f5663fff700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f564006ace0 0x7f564006d190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.546+0000 7f5663fff700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f565c0ca010 0x7f565c0ca490 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.546+0000 7f5663fff700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56480052d0 0x7f565c1f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.546+0000 7f5663fff700 1 --2- 172.21.15.49:0/22939580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f565c0c9240 0x7f565c157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.546+0000 7f5663fff700 1 -- 172.21.15.49:0/22939580 >> 172.21.15.49:0/22939580 conn(0x7f565c09c1d0 msgr2=0x7f565c15aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.546+0000 7f5663fff700 1 -- 172.21.15.49:0/22939580 shutdown_connections 2023-07-18T03:39:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.546+0000 7f5663fff700 1 -- 172.21.15.49:0/22939580 wait complete. 2023-07-18T03:39:52.558 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:39:52.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:39:52.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.771+0000 7f39b69ff700 1 Processor -- start 2023-07-18T03:39:52.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.771+0000 7f39b69ff700 1 -- start start 2023-07-18T03:39:52.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.772+0000 7f39b69ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f39b015fd30 0x7f39b00ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.772+0000 7f39b69ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f39b00ae730 0x7f39b00cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.772+0000 7f39b69ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39b00cd6f0 0x7f39b01603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.772+0000 7f39b69ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b006b930 con 0x7f39b015fd30 2023-07-18T03:39:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39b69ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b00cdb90 con 0x7f39b00ae730 2023-07-18T03:39:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39b69ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b00b21b0 con 0x7f39b00cd6f0 2023-07-18T03:39:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39affff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f39b015fd30 0x7f39b00ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39affff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f39b015fd30 0x7f39b00ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40016/0 (socket says 172.21.15.49:40016) 2023-07-18T03:39:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39affff700 1 -- 172.21.15.49:0/1922421388 learned_addr learned my addr 172.21.15.49:0/1922421388 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:52.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39af7fe700 1 --2- 172.21.15.49:0/1922421388 >> v2:172.21.15.107:3300/0 conn(0x7f39b00ae730 0x7f39b00cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39b4f9c700 1 --2- 172.21.15.49:0/1922421388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39b00cd6f0 0x7f39b01603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39affff700 1 -- 172.21.15.49:0/1922421388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39b00cd6f0 msgr2=0x7f39b01603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39affff700 1 --2- 172.21.15.49:0/1922421388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39b00cd6f0 0x7f39b01603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39affff700 1 -- 172.21.15.49:0/1922421388 >> v2:172.21.15.107:3300/0 conn(0x7f39b00ae730 msgr2=0x7f39b00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:52.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39affff700 1 --2- 172.21.15.49:0/1922421388 >> v2:172.21.15.107:3300/0 conn(0x7f39b00ae730 0x7f39b00cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39affff700 1 -- 172.21.15.49:0/1922421388 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b006ad20 con 0x7f39b015fd30 2023-07-18T03:39:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.773+0000 7f39b4f9c700 1 --2- 172.21.15.49:0/1922421388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39b00cd6f0 0x7f39b01603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:52.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.774+0000 7f39affff700 1 --2- 172.21.15.49:0/1922421388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b015fd30 0x7f39b00ae1f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f39b006fb60 tx=0x7f39a0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd309095aa09aead server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.774+0000 7f39aeffd700 1 -- 172.21.15.49:0/1922421388 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0011830 con 0x7f39b015fd30 2023-07-18T03:39:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.774+0000 7f39aeffd700 1 -- 172.21.15.49:0/1922421388 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a0011990 con 0x7f39b015fd30 2023-07-18T03:39:52.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.774+0000 7f39aeffd700 1 -- 172.21.15.49:0/1922421388 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0011d20 con 0x7f39b015fd30 2023-07-18T03:39:52.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.774+0000 7f39b69ff700 1 -- 172.21.15.49:0/1922421388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b015fd30 msgr2=0x7f39b00ae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.774+0000 7f39b69ff700 1 --2- 172.21.15.49:0/1922421388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b015fd30 0x7f39b00ae1f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f39b006fb60 tx=0x7f39a0009bd0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.775+0000 7f39b69ff700 1 -- 172.21.15.49:0/1922421388 shutdown_connections 2023-07-18T03:39:52.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.775+0000 7f39b69ff700 1 --2- 172.21.15.49:0/1922421388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39b00cd6f0 0x7f39b01603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.775+0000 7f39b69ff700 1 --2- 172.21.15.49:0/1922421388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b015fd30 0x7f39b00ae1f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.775+0000 7f39b69ff700 1 --2- 172.21.15.49:0/1922421388 >> v2:172.21.15.107:3300/0 conn(0x7f39b00ae730 0x7f39b00cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.775+0000 7f39b69ff700 1 -- 172.21.15.49:0/1922421388 >> 172.21.15.49:0/1922421388 conn(0x7f39b00cc990 msgr2=0x7f39b00cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:52.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.775+0000 7f39b69ff700 1 -- 172.21.15.49:0/1922421388 shutdown_connections 2023-07-18T03:39:52.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.775+0000 7f39b69ff700 1 -- 172.21.15.49:0/1922421388 wait complete. 2023-07-18T03:39:52.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.777+0000 7f39b69ff700 1 Processor -- start 2023-07-18T03:39:52.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.777+0000 7f39b69ff700 1 -- start start 2023-07-18T03:39:52.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b69ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39b00ae730 0x7f39b00c3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b69ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39b00cd6f0 0x7f39b015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b69ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b015fd30 0x7f39b00c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b69ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b015adb0 con 0x7f39b015fd30 2023-07-18T03:39:52.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b69ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b00c4040 con 0x7f39b00cd6f0 2023-07-18T03:39:52.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b69ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b01f30f0 con 0x7f39b00ae730 2023-07-18T03:39:52.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b4f9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b015fd30 0x7f39b00c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b4f9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b015fd30 0x7f39b00c2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40022/0 (socket says 172.21.15.49:40022) 2023-07-18T03:39:52.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b4f9c700 1 -- 172.21.15.49:0/1638857182 learned_addr learned my addr 172.21.15.49:0/1638857182 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39af7fe700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39b00cd6f0 0x7f39b015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39affff700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39b00ae730 0x7f39b00c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b4f9c700 1 -- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39b00ae730 msgr2=0x7f39b00c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b4f9c700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39b00ae730 0x7f39b00c3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b4f9c700 1 -- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39b00cd6f0 msgr2=0x7f39b015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b4f9c700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39b00cd6f0 0x7f39b015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39b4f9c700 1 -- 172.21.15.49:0/1638857182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a0009550 con 0x7f39b015fd30 2023-07-18T03:39:52.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.778+0000 7f39af7fe700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39b00cd6f0 0x7f39b015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:52.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.779+0000 7f39b4f9c700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b015fd30 0x7f39b00c2e70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009fd0 tx=0x7f39a800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:52.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.779+0000 7f39acff9700 1 -- 172.21.15.49:0/1638857182 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8003a20 con 0x7f39b015fd30 2023-07-18T03:39:52.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.779+0000 7f39acff9700 1 -- 172.21.15.49:0/1638857182 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a800f040 con 0x7f39b015fd30 2023-07-18T03:39:52.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.779+0000 7f39b69ff700 1 -- 172.21.15.49:0/1638857182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b00c3d60 con 0x7f39b015fd30 2023-07-18T03:39:52.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.779+0000 7f39acff9700 1 -- 172.21.15.49:0/1638857182 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8003de0 con 0x7f39b015fd30 2023-07-18T03:39:52.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.779+0000 7f39b69ff700 1 -- 172.21.15.49:0/1638857182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b01f33f0 con 0x7f39b015fd30 2023-07-18T03:39:52.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.780+0000 7f39b69ff700 1 -- 172.21.15.49:0/1638857182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b0074350 con 0x7f39b015fd30 2023-07-18T03:39:52.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.781+0000 7f39acff9700 1 -- 172.21.15.49:0/1638857182 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f39a8004030 con 0x7f39b015fd30 2023-07-18T03:39:52.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.783+0000 7f39acff9700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f399c06ace0 0x7f399c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:52.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.783+0000 7f39acff9700 1 -- 172.21.15.49:0/1638857182 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f39a8089260 con 0x7f39b015fd30 2023-07-18T03:39:52.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.783+0000 7f39affff700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f399c06ace0 0x7f399c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:52.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.784+0000 7f39affff700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f399c06ace0 0x7f399c06d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f39a0014040 tx=0x7f39a0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:52.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.787+0000 7f39acff9700 1 -- 172.21.15.49:0/1638857182 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f39a80546f0 con 0x7f39b015fd30 2023-07-18T03:39:52.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.993+0000 7f39b69ff700 1 -- 172.21.15.49:0/1638857182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f39b00cfd40 con 0x7f39b015fd30 2023-07-18T03:39:52.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.994+0000 7f39acff9700 1 -- 172.21.15.49:0/1638857182 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f39a8057d10 con 0x7f39b015fd30 2023-07-18T03:39:52.990 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:39:52.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.996+0000 7f39b69ff700 1 -- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f399c06ace0 msgr2=0x7f399c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.996+0000 7f39b69ff700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f399c06ace0 0x7f399c06d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f39a0014040 tx=0x7f39a0008030 comp rx=0 tx=0).stop 2023-07-18T03:39:52.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.996+0000 7f39b69ff700 1 -- 172.21.15.49:0/1638857182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b015fd30 msgr2=0x7f39b00c2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:52.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.996+0000 7f39b69ff700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b015fd30 0x7f39b00c2e70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009fd0 tx=0x7f39a800d460 comp rx=0 tx=0).stop 2023-07-18T03:39:52.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.996+0000 7f39b69ff700 1 -- 172.21.15.49:0/1638857182 shutdown_connections 2023-07-18T03:39:52.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.996+0000 7f39b69ff700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f399c06ace0 0x7f399c06d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.996+0000 7f39b69ff700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39b015fd30 0x7f39b00c2e70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.996+0000 7f39b69ff700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39b00ae730 0x7f39b00c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.996+0000 7f39b69ff700 1 --2- 172.21.15.49:0/1638857182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39b00cd6f0 0x7f39b015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:52.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.996+0000 7f39b69ff700 1 -- 172.21.15.49:0/1638857182 >> 172.21.15.49:0/1638857182 conn(0x7f39b00cc990 msgr2=0x7f39b00ac1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:52.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.996+0000 7f39b69ff700 1 -- 172.21.15.49:0/1638857182 shutdown_connections 2023-07-18T03:39:52.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:52.996+0000 7f39b69ff700 1 -- 172.21.15.49:0/1638857182 wait complete. 2023-07-18T03:39:53.011 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:39:53.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.228+0000 7f58f3dff700 1 Processor -- start 2023-07-18T03:39:53.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.229+0000 7f58f3dff700 1 -- start start 2023-07-18T03:39:53.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.230+0000 7f58f3dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec0b12f0 0x7f58ec0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.230+0000 7f58f3dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ec15ce70 con 0x7f58ec0adea0 2023-07-18T03:39:53.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.230+0000 7f58f3dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ec15d0c0 con 0x7f58ec0c92d0 2023-07-18T03:39:53.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.230+0000 7f58f3dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec15da60 con 0x7f58ec0b12f0 2023-07-18T03:39:53.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.230+0000 7f58f239c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f58ec0adea0 0x7f58ec0cc690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37646/0 (socket says 172.21.15.49:37646) 2023-07-18T03:39:53.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.230+0000 7f58f239c700 1 -- 172.21.15.49:0/1172865573 learned_addr learned my addr 172.21.15.49:0/1172865573 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:53.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.230+0000 7f58f139a700 1 --2- 172.21.15.49:0/1172865573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec0b12f0 0x7f58ec0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.231+0000 7f58f0b99700 1 -- 172.21.15.49:0/1172865573 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099233827 0 0) 0x7f58ec15ce70 con 0x7f58ec0adea0 2023-07-18T03:39:53.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.231+0000 7f58f0b99700 1 -- 172.21.15.49:0/1172865573 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58d80057c0 con 0x7f58ec0adea0 2023-07-18T03:39:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.231+0000 7f58f0b99700 1 -- 172.21.15.49:0/1172865573 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298818361 0 0) 0x7f58ec15d0c0 con 0x7f58ec0c92d0 2023-07-18T03:39:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.231+0000 7f58f0b99700 1 -- 172.21.15.49:0/1172865573 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58ec15ce70 con 0x7f58ec0c92d0 2023-07-18T03:39:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.231+0000 7f58f139a700 1 -- 172.21.15.49:0/1172865573 >> v1:172.21.15.107:6789/0 conn(0x7f58ec0c92d0 legacy=0x7f58ec0c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.231+0000 7f58f139a700 1 -- 172.21.15.49:0/1172865573 >> v1:172.21.15.49:6789/0 conn(0x7f58ec0adea0 legacy=0x7f58ec0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.231+0000 7f58f0b99700 1 -- 172.21.15.49:0/1172865573 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4057306006 0 0) 0x7f58d80057c0 con 0x7f58ec0adea0 2023-07-18T03:39:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.231+0000 7f58f139a700 1 -- 172.21.15.49:0/1172865573 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec15d3a0 con 0x7f58ec0b12f0 2023-07-18T03:39:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.231+0000 7f58f139a700 1 --2- 172.21.15.49:0/1172865573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec0b12f0 0x7f58ec0ad8f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009870 tx=0x7f58dc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fdee8d1a793d995 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:53.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.232+0000 7f58f0b99700 1 -- 172.21.15.49:0/1172865573 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc011830 con 0x7f58ec0b12f0 2023-07-18T03:39:53.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.232+0000 7f58f0b99700 1 -- 172.21.15.49:0/1172865573 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc011990 con 0x7f58ec0b12f0 2023-07-18T03:39:53.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.232+0000 7f58f0b99700 1 -- 172.21.15.49:0/1172865573 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc011d20 con 0x7f58ec0b12f0 2023-07-18T03:39:53.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.232+0000 7f58f3dff700 1 -- 172.21.15.49:0/1172865573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec0b12f0 msgr2=0x7f58ec0ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.232+0000 7f58f3dff700 1 --2- 172.21.15.49:0/1172865573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec0b12f0 0x7f58ec0ad8f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009870 tx=0x7f58dc009b80 comp rx=0 tx=0).stop 2023-07-18T03:39:53.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.232+0000 7f58f3dff700 1 -- 172.21.15.49:0/1172865573 shutdown_connections 2023-07-18T03:39:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.232+0000 7f58f3dff700 1 --2- 172.21.15.49:0/1172865573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec0b12f0 0x7f58ec0ad8f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.232+0000 7f58f3dff700 1 -- 172.21.15.49:0/1172865573 >> 172.21.15.49:0/1172865573 conn(0x7f58ec0b05d0 msgr2=0x7f58ec0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.232+0000 7f58f3dff700 1 -- 172.21.15.49:0/1172865573 shutdown_connections 2023-07-18T03:39:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.232+0000 7f58f3dff700 1 -- 172.21.15.49:0/1172865573 wait complete. 2023-07-18T03:39:53.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.234+0000 7f58f3dff700 1 Processor -- start 2023-07-18T03:39:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.234+0000 7f58f3dff700 1 -- start start 2023-07-18T03:39:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.235+0000 7f58f3dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58ec0adea0 0x7f58ec158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.235+0000 7f58f3dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec0b12f0 0x7f58ec0af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.235+0000 7f58f3dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec0c92d0 0x7f58ec154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.235+0000 7f58f3dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec1f6090 con 0x7f58ec0adea0 2023-07-18T03:39:53.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.235+0000 7f58f3dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec1f57a0 con 0x7f58ec0b12f0 2023-07-18T03:39:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.235+0000 7f58f3dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec1f5a40 con 0x7f58ec0c92d0 2023-07-18T03:39:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f1b9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58ec0adea0 0x7f58ec158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f1b9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58ec0adea0 0x7f58ec158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40034/0 (socket says 172.21.15.49:40034) 2023-07-18T03:39:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f1b9b700 1 -- 172.21.15.49:0/3006693270 learned_addr learned my addr 172.21.15.49:0/3006693270 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f1b9b700 1 -- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec0c92d0 msgr2=0x7f58ec154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f239c700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec0c92d0 0x7f58ec154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f139a700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec0b12f0 0x7f58ec0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f1b9b700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec0c92d0 0x7f58ec154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f1b9b700 1 -- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec0b12f0 msgr2=0x7f58ec0af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f1b9b700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec0b12f0 0x7f58ec0af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f1b9b700 1 -- 172.21.15.49:0/3006693270 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc0094a0 con 0x7f58ec0adea0 2023-07-18T03:39:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f139a700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec0b12f0 0x7f58ec0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:53.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f239c700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec0c92d0 0x7f58ec154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:53.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.236+0000 7f58f1b9b700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58ec0adea0 0x7f58ec158680 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f58e800c520 tx=0x7f58e800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:53.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.237+0000 7f58e27fc700 1 -- 172.21.15.49:0/3006693270 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e80052d0 con 0x7f58ec0adea0 2023-07-18T03:39:53.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.237+0000 7f58e27fc700 1 -- 172.21.15.49:0/3006693270 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e8005430 con 0x7f58ec0adea0 2023-07-18T03:39:53.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.237+0000 7f58f3dff700 1 -- 172.21.15.49:0/3006693270 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ec1f5d20 con 0x7f58ec0adea0 2023-07-18T03:39:53.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.237+0000 7f58e27fc700 1 -- 172.21.15.49:0/3006693270 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e80038c0 con 0x7f58ec0adea0 2023-07-18T03:39:53.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.237+0000 7f58f3dff700 1 -- 172.21.15.49:0/3006693270 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec1f6520 con 0x7f58ec0adea0 2023-07-18T03:39:53.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.238+0000 7f58f3dff700 1 -- 172.21.15.49:0/3006693270 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ec074350 con 0x7f58ec0adea0 2023-07-18T03:39:53.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.239+0000 7f58e27fc700 1 -- 172.21.15.49:0/3006693270 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f58e8003a20 con 0x7f58ec0adea0 2023-07-18T03:39:53.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.240+0000 7f58e27fc700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f58d006ace0 0x7f58d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.240+0000 7f58e27fc700 1 -- 172.21.15.49:0/3006693270 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f58e80886c0 con 0x7f58ec0adea0 2023-07-18T03:39:53.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.240+0000 7f58f139a700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f58d006ace0 0x7f58d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.241+0000 7f58f139a700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f58d006ace0 0x7f58d006d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f58dc000c00 tx=0x7f58dc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.244+0000 7f58e27fc700 1 -- 172.21.15.49:0/3006693270 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f58e8053b50 con 0x7f58ec0adea0 2023-07-18T03:39:53.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.451+0000 7f58f3dff700 1 -- 172.21.15.49:0/3006693270 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58ec0b4a40 con 0x7f58ec0adea0 2023-07-18T03:39:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.452+0000 7f58e27fc700 1 -- 172.21.15.49:0/3006693270 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58e8057170 con 0x7f58ec0adea0 2023-07-18T03:39:53.447 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:39:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.454+0000 7f58f3dff700 1 -- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f58d006ace0 msgr2=0x7f58d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.454+0000 7f58f3dff700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f58d006ace0 0x7f58d006d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f58dc000c00 tx=0x7f58dc01c040 comp rx=0 tx=0).stop 2023-07-18T03:39:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.454+0000 7f58f3dff700 1 -- 172.21.15.49:0/3006693270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58ec0adea0 msgr2=0x7f58ec158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.454+0000 7f58f3dff700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58ec0adea0 0x7f58ec158680 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f58e800c520 tx=0x7f58e800c8e0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.454+0000 7f58f3dff700 1 -- 172.21.15.49:0/3006693270 shutdown_connections 2023-07-18T03:39:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.454+0000 7f58f3dff700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec0c92d0 0x7f58ec154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.454+0000 7f58f3dff700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f58d006ace0 0x7f58d006d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.454+0000 7f58f3dff700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58ec0adea0 0x7f58ec158680 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.454+0000 7f58f3dff700 1 --2- 172.21.15.49:0/3006693270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec0b12f0 0x7f58ec0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.454+0000 7f58f3dff700 1 -- 172.21.15.49:0/3006693270 >> 172.21.15.49:0/3006693270 conn(0x7f58ec0b05d0 msgr2=0x7f58ec06fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.454+0000 7f58f3dff700 1 -- 172.21.15.49:0/3006693270 shutdown_connections 2023-07-18T03:39:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.454+0000 7f58f3dff700 1 -- 172.21.15.49:0/3006693270 wait complete. 2023-07-18T03:39:53.467 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-18T03:39:53.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.678+0000 7fa3effff700 1 Processor -- start 2023-07-18T03:39:53.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.678+0000 7fa3effff700 1 -- start start 2023-07-18T03:39:53.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.679+0000 7fa3effff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3e80b4510 0x7fa3e80b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.679+0000 7fa3effff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e80b7f50 con 0x7fa3e80d0b10 2023-07-18T03:39:53.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.679+0000 7fa3effff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e80cdca0 con 0x7fa3e80b5d90 2023-07-18T03:39:53.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.679+0000 7fa3effff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80b48e0 con 0x7fa3e80b4510 2023-07-18T03:39:53.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.680+0000 7fa3ee59c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa3e80d0b10 0x7fa3e80b83e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37648/0 (socket says 172.21.15.49:37648) 2023-07-18T03:39:53.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.680+0000 7fa3ee59c700 1 -- 172.21.15.49:0/2280166553 learned_addr learned my addr 172.21.15.49:0/2280166553 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:53.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.680+0000 7fa3edd9b700 1 --2- 172.21.15.49:0/2280166553 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3e80b4510 0x7fa3e80b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.680+0000 7fa3ecd99700 1 -- 172.21.15.49:0/2280166553 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585889228 0 0) 0x7fa3e80b7f50 con 0x7fa3e80d0b10 2023-07-18T03:39:53.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.680+0000 7fa3ecd99700 1 -- 172.21.15.49:0/2280166553 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3d40035b0 con 0x7fa3e80d0b10 2023-07-18T03:39:53.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.680+0000 7fa3ecd99700 1 -- 172.21.15.49:0/2280166553 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067194952 0 0) 0x7fa3e80cdca0 con 0x7fa3e80b5d90 2023-07-18T03:39:53.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.680+0000 7fa3ecd99700 1 -- 172.21.15.49:0/2280166553 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3e80b7f50 con 0x7fa3e80b5d90 2023-07-18T03:39:53.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.680+0000 7fa3edd9b700 1 -- 172.21.15.49:0/2280166553 >> v1:172.21.15.107:6789/0 conn(0x7fa3e80b5d90 legacy=0x7fa3e80cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.680+0000 7fa3edd9b700 1 -- 172.21.15.49:0/2280166553 >> v1:172.21.15.49:6789/0 conn(0x7fa3e80d0b10 legacy=0x7fa3e80b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.680+0000 7fa3edd9b700 1 -- 172.21.15.49:0/2280166553 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e80b7340 con 0x7fa3e80b4510 2023-07-18T03:39:53.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.681+0000 7fa3edd9b700 1 --2- 172.21.15.49:0/2280166553 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3e80b4510 0x7fa3e80b6c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009850 tx=0x7fa3e4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11bdb230330465b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:53.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.681+0000 7fa3ecd99700 1 -- 172.21.15.49:0/2280166553 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4005e30 con 0x7fa3e80b4510 2023-07-18T03:39:53.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.681+0000 7fa3ecd99700 1 -- 172.21.15.49:0/2280166553 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e4006030 con 0x7fa3e80b4510 2023-07-18T03:39:53.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.681+0000 7fa3ecd99700 1 -- 172.21.15.49:0/2280166553 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40063e0 con 0x7fa3e80b4510 2023-07-18T03:39:53.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.682+0000 7fa3effff700 1 -- 172.21.15.49:0/2280166553 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3e80b4510 msgr2=0x7fa3e80b6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.682+0000 7fa3effff700 1 --2- 172.21.15.49:0/2280166553 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3e80b4510 0x7fa3e80b6c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009850 tx=0x7fa3e4009b60 comp rx=0 tx=0).stop 2023-07-18T03:39:53.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.682+0000 7fa3effff700 1 -- 172.21.15.49:0/2280166553 shutdown_connections 2023-07-18T03:39:53.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.682+0000 7fa3effff700 1 --2- 172.21.15.49:0/2280166553 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3e80b4510 0x7fa3e80b6c80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.682+0000 7fa3effff700 1 -- 172.21.15.49:0/2280166553 >> 172.21.15.49:0/2280166553 conn(0x7fa3e809c1d0 msgr2=0x7fa3e80d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:53.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.682+0000 7fa3effff700 1 -- 172.21.15.49:0/2280166553 shutdown_connections 2023-07-18T03:39:53.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.682+0000 7fa3effff700 1 -- 172.21.15.49:0/2280166553 wait complete. 2023-07-18T03:39:53.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.684+0000 7fa3effff700 1 Processor -- start 2023-07-18T03:39:53.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.684+0000 7fa3effff700 1 -- start start 2023-07-18T03:39:53.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3effff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3e80b4510 0x7fa3e8159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3effff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e80b5d90 0x7fa3e815b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3effff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3e80d0b10 0x7fa3e815bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3effff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e8155730 con 0x7fa3e80b5d90 2023-07-18T03:39:53.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3effff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e8155a10 con 0x7fa3e80b4510 2023-07-18T03:39:53.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3effff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e8155cf0 con 0x7fa3e80d0b10 2023-07-18T03:39:53.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3ed59a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e80b5d90 0x7fa3e815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3ed59a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e80b5d90 0x7fa3e815b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40042/0 (socket says 172.21.15.49:40042) 2023-07-18T03:39:53.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3ed59a700 1 -- 172.21.15.49:0/3063799165 learned_addr learned my addr 172.21.15.49:0/3063799165 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:53.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3ee59c700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3e80d0b10 0x7fa3e815bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3edd9b700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3e80b4510 0x7fa3e8159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3ed59a700 1 -- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3e80d0b10 msgr2=0x7fa3e815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3ed59a700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3e80d0b10 0x7fa3e815bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3ed59a700 1 -- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3e80b4510 msgr2=0x7fa3e8159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3ed59a700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3e80b4510 0x7fa3e8159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3ed59a700 1 -- 172.21.15.49:0/3063799165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e4009430 con 0x7fa3e80b5d90 2023-07-18T03:39:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.685+0000 7fa3ee59c700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3e80d0b10 0x7fa3e815bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.686+0000 7fa3ed59a700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e80b5d90 0x7fa3e815b340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800c520 tx=0x7fa3d800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.686+0000 7fa3de7fc700 1 -- 172.21.15.49:0/3063799165 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80052d0 con 0x7fa3e80b5d90 2023-07-18T03:39:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.686+0000 7fa3de7fc700 1 -- 172.21.15.49:0/3063799165 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d8005430 con 0x7fa3e80b5d90 2023-07-18T03:39:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.686+0000 7fa3effff700 1 -- 172.21.15.49:0/3063799165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e80c6bf0 con 0x7fa3e80b5d90 2023-07-18T03:39:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.686+0000 7fa3de7fc700 1 -- 172.21.15.49:0/3063799165 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80038a0 con 0x7fa3e80b5d90 2023-07-18T03:39:53.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.686+0000 7fa3effff700 1 -- 172.21.15.49:0/3063799165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e815b880 con 0x7fa3e80b5d90 2023-07-18T03:39:53.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.687+0000 7fa3effff700 1 -- 172.21.15.49:0/3063799165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e80c6e10 con 0x7fa3e80b5d90 2023-07-18T03:39:53.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.688+0000 7fa3de7fc700 1 -- 172.21.15.49:0/3063799165 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3d8003a00 con 0x7fa3e80b5d90 2023-07-18T03:39:53.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.689+0000 7fa3de7fc700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa3cc06ace0 0x7fa3cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.689+0000 7fa3de7fc700 1 -- 172.21.15.49:0/3063799165 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa3d8088710 con 0x7fa3e80b5d90 2023-07-18T03:39:53.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.690+0000 7fa3edd9b700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa3cc06ace0 0x7fa3cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.690+0000 7fa3edd9b700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa3cc06ace0 0x7fa3cc06d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4000c00 tx=0x7fa3e4008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:53.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.693+0000 7fa3de7fc700 1 -- 172.21.15.49:0/3063799165 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa3d8053a70 con 0x7fa3e80b5d90 2023-07-18T03:39:53.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.906+0000 7fa3effff700 1 -- 172.21.15.49:0/3063799165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa3e804e280 con 0x7fa3e80b5d90 2023-07-18T03:39:53.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.906+0000 7fa3de7fc700 1 -- 172.21.15.49:0/3063799165 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa3d8003cb0 con 0x7fa3e80b5d90 2023-07-18T03:39:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:53.901 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2023-07-18T03:39:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2023-07-18T03:39:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": "rbd" 2023-07-18T03:39:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:39:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:39:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 2, 2023-07-18T03:39:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2023-07-18T03:39:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:39:53.903 INFO:teuthology.orchestra.run.smithi049.stdout:] 2023-07-18T03:39:53.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.908+0000 7fa3effff700 1 -- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa3cc06ace0 msgr2=0x7fa3cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.908+0000 7fa3effff700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa3cc06ace0 0x7fa3cc06d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4000c00 tx=0x7fa3e4008070 comp rx=0 tx=0).stop 2023-07-18T03:39:53.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.908+0000 7fa3effff700 1 -- 172.21.15.49:0/3063799165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e80b5d90 msgr2=0x7fa3e815b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.908+0000 7fa3effff700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e80b5d90 0x7fa3e815b340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800c520 tx=0x7fa3d800c830 comp rx=0 tx=0).stop 2023-07-18T03:39:53.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.908+0000 7fa3effff700 1 -- 172.21.15.49:0/3063799165 shutdown_connections 2023-07-18T03:39:53.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.908+0000 7fa3effff700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3e80d0b10 0x7fa3e815bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.908+0000 7fa3effff700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3e80b4510 0x7fa3e8159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.908+0000 7fa3effff700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa3cc06ace0 0x7fa3cc06d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.908+0000 7fa3effff700 1 --2- 172.21.15.49:0/3063799165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e80b5d90 0x7fa3e815b340 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.908+0000 7fa3effff700 1 -- 172.21.15.49:0/3063799165 >> 172.21.15.49:0/3063799165 conn(0x7fa3e809c1d0 msgr2=0x7fa3e80cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:53.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.908+0000 7fa3effff700 1 -- 172.21.15.49:0/3063799165 shutdown_connections 2023-07-18T03:39:53.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:53.909+0000 7fa3effff700 1 -- 172.21.15.49:0/3063799165 wait complete. 2023-07-18T03:39:53.921 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-18T03:39:53.922 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-18T03:39:54.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.136+0000 7f63c7bff700 1 Processor -- start 2023-07-18T03:39:54.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.136+0000 7f63c7bff700 1 -- start start 2023-07-18T03:39:54.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.137+0000 7f63c7bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f63c00d1140 0x7f63c00d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:54.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.137+0000 7f63c7bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f63c00b1e50 0x7f63c00ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:54.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c7bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c00d0190 con 0x7f63c015c7b0 2023-07-18T03:39:54.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c7bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c00d3de0 con 0x7f63c00d1140 2023-07-18T03:39:54.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c7bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c00d0470 con 0x7f63c00b1e50 2023-07-18T03:39:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c599b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f63c015c7b0 0x7f63c00d0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37652/0 (socket says 172.21.15.49:37652) 2023-07-18T03:39:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c599b700 1 -- 172.21.15.49:0/3928793509 learned_addr learned my addr 172.21.15.49:0/3928793509 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c519a700 1 --2- 172.21.15.49:0/3928793509 >> v2:172.21.15.49:3300/0 conn(0x7f63c00d1140 0x7f63c00d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c619c700 1 --2- 172.21.15.49:0/3928793509 >> v2:172.21.15.107:3300/0 conn(0x7f63c00b1e50 0x7f63c00ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c4999700 1 -- 172.21.15.49:0/3928793509 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280115082 0 0) 0x7f63c00d0190 con 0x7f63c015c7b0 2023-07-18T03:39:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c4999700 1 -- 172.21.15.49:0/3928793509 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63ac0035b0 con 0x7f63c015c7b0 2023-07-18T03:39:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c519a700 1 -- 172.21.15.49:0/3928793509 >> v2:172.21.15.107:3300/0 conn(0x7f63c00b1e50 msgr2=0x7f63c00ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c519a700 1 --2- 172.21.15.49:0/3928793509 >> v2:172.21.15.107:3300/0 conn(0x7f63c00b1e50 0x7f63c00ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c519a700 1 -- 172.21.15.49:0/3928793509 >> v1:172.21.15.49:6789/0 conn(0x7f63c015c7b0 legacy=0x7f63c00d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c519a700 1 -- 172.21.15.49:0/3928793509 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c00d0750 con 0x7f63c00d1140 2023-07-18T03:39:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.138+0000 7f63c619c700 1 --2- 172.21.15.49:0/3928793509 >> v2:172.21.15.107:3300/0 conn(0x7f63c00b1e50 0x7f63c00ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.139+0000 7f63c519a700 1 --2- 172.21.15.49:0/3928793509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63c00d1140 0x7f63c00d28a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f63c008d670 tx=0x7f63b0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f1a67e000effc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.139+0000 7f63c4999700 1 -- 172.21.15.49:0/3928793509 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0011830 con 0x7f63c00d1140 2023-07-18T03:39:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.139+0000 7f63c4999700 1 -- 172.21.15.49:0/3928793509 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b0011990 con 0x7f63c00d1140 2023-07-18T03:39:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.139+0000 7f63c4999700 1 -- 172.21.15.49:0/3928793509 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0011d20 con 0x7f63c00d1140 2023-07-18T03:39:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.139+0000 7f63c7bff700 1 -- 172.21.15.49:0/3928793509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63c00d1140 msgr2=0x7f63c00d28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.139+0000 7f63c7bff700 1 --2- 172.21.15.49:0/3928793509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63c00d1140 0x7f63c00d28a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f63c008d670 tx=0x7f63b0009b20 comp rx=0 tx=0).stop 2023-07-18T03:39:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.140+0000 7f63c7bff700 1 -- 172.21.15.49:0/3928793509 shutdown_connections 2023-07-18T03:39:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.140+0000 7f63c7bff700 1 --2- 172.21.15.49:0/3928793509 >> v2:172.21.15.107:3300/0 conn(0x7f63c00b1e50 0x7f63c00ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.140+0000 7f63c7bff700 1 --2- 172.21.15.49:0/3928793509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63c00d1140 0x7f63c00d28a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:54.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.140+0000 7f63c7bff700 1 -- 172.21.15.49:0/3928793509 >> 172.21.15.49:0/3928793509 conn(0x7f63c009c1b0 msgr2=0x7f63c00d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:54.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.140+0000 7f63c7bff700 1 -- 172.21.15.49:0/3928793509 shutdown_connections 2023-07-18T03:39:54.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.140+0000 7f63c7bff700 1 -- 172.21.15.49:0/3928793509 wait complete. 2023-07-18T03:39:54.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.141+0000 7f63c7bff700 1 Processor -- start 2023-07-18T03:39:54.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.142+0000 7f63c7bff700 1 -- start start 2023-07-18T03:39:54.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c7bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63c00b1e50 0x7f63c00b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:54.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c7bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63c00d1140 0x7f63c01f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:54.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c7bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63c015c7b0 0x7f63c00cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:54.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c7bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c01f6440 con 0x7f63c015c7b0 2023-07-18T03:39:54.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c7bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00b26e0 con 0x7f63c00d1140 2023-07-18T03:39:54.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c7bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00b29c0 con 0x7f63c00b1e50 2023-07-18T03:39:54.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c619c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63c015c7b0 0x7f63c00cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:54.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c619c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63c015c7b0 0x7f63c00cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40056/0 (socket says 172.21.15.49:40056) 2023-07-18T03:39:54.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c619c700 1 -- 172.21.15.49:0/1968089139 learned_addr learned my addr 172.21.15.49:0/1968089139 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:54.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c599b700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63c00b1e50 0x7f63c00b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:54.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c519a700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63c00d1140 0x7f63c01f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:54.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c619c700 1 -- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63c00b1e50 msgr2=0x7f63c00b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:54.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c619c700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63c00b1e50 0x7f63c00b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:54.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c619c700 1 -- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63c00d1140 msgr2=0x7f63c01f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:54.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c619c700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63c00d1140 0x7f63c01f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:54.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c619c700 1 -- 172.21.15.49:0/1968089139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b00094a0 con 0x7f63c015c7b0 2023-07-18T03:39:54.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.143+0000 7f63c519a700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63c00d1140 0x7f63c01f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:54.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.144+0000 7f63c619c700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63c015c7b0 0x7f63c00cc3e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009fa0 tx=0x7f63b800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:54.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.144+0000 7f63b67fc700 1 -- 172.21.15.49:0/1968089139 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8003a20 con 0x7f63c015c7b0 2023-07-18T03:39:54.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.144+0000 7f63b67fc700 1 -- 172.21.15.49:0/1968089139 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b800f040 con 0x7f63c015c7b0 2023-07-18T03:39:54.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.144+0000 7f63c7bff700 1 -- 172.21.15.49:0/1968089139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c00b2ca0 con 0x7f63c015c7b0 2023-07-18T03:39:54.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.144+0000 7f63b67fc700 1 -- 172.21.15.49:0/1968089139 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8003de0 con 0x7f63c015c7b0 2023-07-18T03:39:54.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.144+0000 7f63c7bff700 1 -- 172.21.15.49:0/1968089139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c00b3260 con 0x7f63c015c7b0 2023-07-18T03:39:54.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.145+0000 7f63c7bff700 1 -- 172.21.15.49:0/1968089139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c00b3480 con 0x7f63c015c7b0 2023-07-18T03:39:54.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.146+0000 7f63b67fc700 1 -- 172.21.15.49:0/1968089139 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f63b8004030 con 0x7f63c015c7b0 2023-07-18T03:39:54.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.147+0000 7f63b67fc700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f63a406ace0 0x7f63a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:54.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.147+0000 7f63b67fc700 1 -- 172.21.15.49:0/1968089139 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f63b80893f0 con 0x7f63c015c7b0 2023-07-18T03:39:54.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.147+0000 7f63c599b700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f63a406ace0 0x7f63a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:54.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.148+0000 7f63c599b700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f63a406ace0 0x7f63a406d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0011e0 tx=0x7f63bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:54.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.151+0000 7f63b67fc700 1 -- 172.21.15.49:0/1968089139 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f63b8054750 con 0x7f63c015c7b0 2023-07-18T03:39:54.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:54.359+0000 7f63c7bff700 1 -- 172.21.15.49:0/1968089139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f63c00cd520 con 0x7f63c015c7b0 2023-07-18T03:39:55.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.319+0000 7f63b67fc700 1 -- 172.21.15.49:0/1968089139 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7f63b8057d70 con 0x7f63c015c7b0 2023-07-18T03:39:55.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.324+0000 7f63c7bff700 1 -- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f63a406ace0 msgr2=0x7f63a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:55.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.324+0000 7f63c7bff700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f63a406ace0 0x7f63a406d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0011e0 tx=0x7f63bc009040 comp rx=0 tx=0).stop 2023-07-18T03:39:55.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.324+0000 7f63c7bff700 1 -- 172.21.15.49:0/1968089139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63c015c7b0 msgr2=0x7f63c00cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:55.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.324+0000 7f63c7bff700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63c015c7b0 0x7f63c00cc3e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009fa0 tx=0x7f63b800d460 comp rx=0 tx=0).stop 2023-07-18T03:39:55.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.325+0000 7f63c7bff700 1 -- 172.21.15.49:0/1968089139 shutdown_connections 2023-07-18T03:39:55.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.325+0000 7f63c7bff700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f63a406ace0 0x7f63a406d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:55.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.325+0000 7f63c7bff700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63c015c7b0 0x7f63c00cc3e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.325+0000 7f63c7bff700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63c00b1e50 0x7f63c00b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.325+0000 7f63c7bff700 1 --2- 172.21.15.49:0/1968089139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63c00d1140 0x7f63c01f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.325+0000 7f63c7bff700 1 -- 172.21.15.49:0/1968089139 >> 172.21.15.49:0/1968089139 conn(0x7f63c009c1b0 msgr2=0x7f63c00afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.325+0000 7f63c7bff700 1 -- 172.21.15.49:0/1968089139 shutdown_connections 2023-07-18T03:39:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.326+0000 7f63c7bff700 1 -- 172.21.15.49:0/1968089139 wait complete. 2023-07-18T03:39:55.322 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2023-07-18T03:39:55.343 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-18T03:39:55.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.564+0000 7fb1561ff700 1 Processor -- start 2023-07-18T03:39:55.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.565+0000 7fb1561ff700 1 -- start start 2023-07-18T03:39:55.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.566+0000 7fb1561ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb1500b4860 0x7fb1500b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:55.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.566+0000 7fb1561ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1500d12c0 0x7fb1500d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:55.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.566+0000 7fb1561ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1500d02a0 con 0x7fb1500b8660 2023-07-18T03:39:55.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.566+0000 7fb1561ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1500b5240 con 0x7fb1500b4860 2023-07-18T03:39:55.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.566+0000 7fb1561ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1500cfce0 con 0x7fb1500d12c0 2023-07-18T03:39:55.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.566+0000 7fb14effd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb1500b4860 0x7fb1500b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:55.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.566+0000 7fb14f7fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb1500b8660 0x7fb1500cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37660/0 (socket says 172.21.15.49:37660) 2023-07-18T03:39:55.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.566+0000 7fb14effd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb1500b4860 0x7fb1500b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40072/0 (socket says 172.21.15.49:40072) 2023-07-18T03:39:55.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.566+0000 7fb14f7fe700 1 -- 172.21.15.49:0/249753647 learned_addr learned my addr 172.21.15.49:0/249753647 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:55.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.566+0000 7fb14ffff700 1 --2- 172.21.15.49:0/249753647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1500d12c0 0x7fb1500d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:55.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.567+0000 7fb14effd700 1 -- 172.21.15.49:0/249753647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1500d12c0 msgr2=0x7fb1500d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:55.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.567+0000 7fb14effd700 1 --2- 172.21.15.49:0/249753647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1500d12c0 0x7fb1500d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:55.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.567+0000 7fb14effd700 1 -- 172.21.15.49:0/249753647 >> v1:172.21.15.49:6789/0 conn(0x7fb1500b8660 legacy=0x7fb1500cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:55.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.567+0000 7fb14effd700 1 -- 172.21.15.49:0/249753647 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1500cffc0 con 0x7fb1500b4860 2023-07-18T03:39:55.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.567+0000 7fb14e7fc700 1 -- 172.21.15.49:0/249753647 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970045363 0 0) 0x7fb1500d02a0 con 0x7fb1500b8660 2023-07-18T03:39:55.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.567+0000 7fb14effd700 1 --2- 172.21.15.49:0/249753647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1500b4860 0x7fb1500b4d00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb144009840 tx=0x7fb144009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1e37180cf00210 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.567+0000 7fb14e7fc700 1 -- 172.21.15.49:0/249753647 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144011830 con 0x7fb1500b4860 2023-07-18T03:39:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.567+0000 7fb14e7fc700 1 -- 172.21.15.49:0/249753647 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb144011990 con 0x7fb1500b4860 2023-07-18T03:39:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.567+0000 7fb14e7fc700 1 -- 172.21.15.49:0/249753647 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144011d20 con 0x7fb1500b4860 2023-07-18T03:39:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.568+0000 7fb1561ff700 1 -- 172.21.15.49:0/249753647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1500b4860 msgr2=0x7fb1500b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.568+0000 7fb1561ff700 1 --2- 172.21.15.49:0/249753647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1500b4860 0x7fb1500b4d00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb144009840 tx=0x7fb144009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:55.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.568+0000 7fb1561ff700 1 -- 172.21.15.49:0/249753647 shutdown_connections 2023-07-18T03:39:55.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.568+0000 7fb1561ff700 1 --2- 172.21.15.49:0/249753647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1500d12c0 0x7fb1500d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:55.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.568+0000 7fb1561ff700 1 --2- 172.21.15.49:0/249753647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1500b4860 0x7fb1500b4d00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:55.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.568+0000 7fb1561ff700 1 -- 172.21.15.49:0/249753647 >> 172.21.15.49:0/249753647 conn(0x7fb150066860 msgr2=0x7fb15006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:55.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.568+0000 7fb1561ff700 1 -- 172.21.15.49:0/249753647 shutdown_connections 2023-07-18T03:39:55.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.568+0000 7fb1561ff700 1 -- 172.21.15.49:0/249753647 wait complete. 2023-07-18T03:39:55.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.570+0000 7fb1561ff700 1 Processor -- start 2023-07-18T03:39:55.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.570+0000 7fb1561ff700 1 -- start start 2023-07-18T03:39:55.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.571+0000 7fb1561ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1500b4860 0x7fb15015e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:55.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.571+0000 7fb1561ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1500b8660 0x7fb1501e7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:55.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.571+0000 7fb1561ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1500d12c0 0x7fb15014b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:55.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.571+0000 7fb1561ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1501e7a10 con 0x7fb1500b8660 2023-07-18T03:39:55.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.571+0000 7fb1561ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1501e7cf0 con 0x7fb1500b4860 2023-07-18T03:39:55.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.571+0000 7fb1561ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1501e7fd0 con 0x7fb1500d12c0 2023-07-18T03:39:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.571+0000 7fb14effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1500b8660 0x7fb1501e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.571+0000 7fb14effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1500b8660 0x7fb1501e7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40078/0 (socket says 172.21.15.49:40078) 2023-07-18T03:39:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.571+0000 7fb14effd700 1 -- 172.21.15.49:0/761854995 learned_addr learned my addr 172.21.15.49:0/761854995 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.571+0000 7fb14f7fe700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1500b4860 0x7fb15015e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.572+0000 7fb14ffff700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1500d12c0 0x7fb15014b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.572+0000 7fb14effd700 1 -- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1500d12c0 msgr2=0x7fb15014b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.572+0000 7fb14effd700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1500d12c0 0x7fb15014b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.572+0000 7fb14effd700 1 -- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1500b4860 msgr2=0x7fb15015e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:55.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.572+0000 7fb14effd700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1500b4860 0x7fb15015e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:55.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.572+0000 7fb14effd700 1 -- 172.21.15.49:0/761854995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1440094a0 con 0x7fb1500b8660 2023-07-18T03:39:55.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.572+0000 7fb14effd700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1500b8660 0x7fb1501e7010 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb1440090b0 tx=0x7fb144004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:55.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.572+0000 7fb137fff700 1 -- 172.21.15.49:0/761854995 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1440036a0 con 0x7fb1500b8660 2023-07-18T03:39:55.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.572+0000 7fb137fff700 1 -- 172.21.15.49:0/761854995 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14401f070 con 0x7fb1500b8660 2023-07-18T03:39:55.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.572+0000 7fb137fff700 1 -- 172.21.15.49:0/761854995 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144003a20 con 0x7fb1500b8660 2023-07-18T03:39:55.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.572+0000 7fb1561ff700 1 -- 172.21.15.49:0/761854995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1501e82b0 con 0x7fb1500b8660 2023-07-18T03:39:55.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.572+0000 7fb1561ff700 1 -- 172.21.15.49:0/761854995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15015e960 con 0x7fb1500b8660 2023-07-18T03:39:55.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.573+0000 7fb1561ff700 1 -- 172.21.15.49:0/761854995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1501e87d0 con 0x7fb1500b8660 2023-07-18T03:39:55.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.575+0000 7fb137fff700 1 -- 172.21.15.49:0/761854995 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb144003b80 con 0x7fb1500b8660 2023-07-18T03:39:55.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.575+0000 7fb137fff700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb13c06ace0 0x7fb13c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:55.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.576+0000 7fb137fff700 1 -- 172.21.15.49:0/761854995 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fb144021070 con 0x7fb1500b8660 2023-07-18T03:39:55.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.576+0000 7fb14f7fe700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb13c06ace0 0x7fb13c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:55.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.577+0000 7fb14f7fe700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb13c06ace0 0x7fb13c06d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb140007fd0 tx=0x7fb140007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:55.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.581+0000 7fb137fff700 1 -- 172.21.15.49:0/761854995 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb1501e87d0 con 0x7fb1500b8660 2023-07-18T03:39:55.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:55.788+0000 7fb1561ff700 1 -- 172.21.15.49:0/761854995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb1501e87d0 con 0x7fb1500b8660 2023-07-18T03:39:56.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.325+0000 7fb137fff700 1 -- 172.21.15.49:0/761854995 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb1501e87d0 con 0x7fb1500b8660 2023-07-18T03:39:56.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.331+0000 7fb1561ff700 1 -- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb13c06ace0 msgr2=0x7fb13c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:56.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.331+0000 7fb1561ff700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb13c06ace0 0x7fb13c06d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb140007fd0 tx=0x7fb140007de0 comp rx=0 tx=0).stop 2023-07-18T03:39:56.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.331+0000 7fb1561ff700 1 -- 172.21.15.49:0/761854995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1500b8660 msgr2=0x7fb1501e7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:56.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.331+0000 7fb1561ff700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1500b8660 0x7fb1501e7010 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb1440090b0 tx=0x7fb144004750 comp rx=0 tx=0).stop 2023-07-18T03:39:56.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.332+0000 7fb1561ff700 1 -- 172.21.15.49:0/761854995 shutdown_connections 2023-07-18T03:39:56.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.332+0000 7fb1561ff700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1500d12c0 0x7fb15014b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:56.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.332+0000 7fb1561ff700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1500b4860 0x7fb15015e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:56.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.332+0000 7fb1561ff700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb13c06ace0 0x7fb13c06d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:56.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.332+0000 7fb1561ff700 1 --2- 172.21.15.49:0/761854995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1500b8660 0x7fb1501e7010 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:56.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.332+0000 7fb1561ff700 1 -- 172.21.15.49:0/761854995 >> 172.21.15.49:0/761854995 conn(0x7fb150066860 msgr2=0x7fb1500b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:56.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.333+0000 7fb1561ff700 1 -- 172.21.15.49:0/761854995 shutdown_connections 2023-07-18T03:39:56.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.333+0000 7fb1561ff700 1 -- 172.21.15.49:0/761854995 wait complete. 2023-07-18T03:39:56.336 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2023-07-18T03:39:56.356 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-18T03:39:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.570+0000 7f4cd83ff700 1 Processor -- start 2023-07-18T03:39:56.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.571+0000 7f4cd83ff700 1 -- start start 2023-07-18T03:39:56.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.572+0000 7f4cd83ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4cd0158bb0 0x7f4cd00cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:56.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.572+0000 7f4cd83ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4cd00cbe40 0x7f4cd00cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:56.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.572+0000 7f4cd83ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd00ce3a0 con 0x7f4cd00cdf80 2023-07-18T03:39:56.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd83ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd006b5b0 con 0x7f4cd0158bb0 2023-07-18T03:39:56.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd83ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd0071500 con 0x7f4cd00cbe40 2023-07-18T03:39:56.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd619b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4cd0158bb0 0x7f4cd00cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:56.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd699c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4cd00cdf80 0x7f4cd015c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37674/0 (socket says 172.21.15.49:37674) 2023-07-18T03:39:56.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd619b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4cd0158bb0 0x7f4cd00cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40084/0 (socket says 172.21.15.49:40084) 2023-07-18T03:39:56.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd699c700 1 -- 172.21.15.49:0/1970658132 learned_addr learned my addr 172.21.15.49:0/1970658132 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:56.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd599a700 1 --2- 172.21.15.49:0/1970658132 >> v2:172.21.15.107:3300/0 conn(0x7f4cd00cbe40 0x7f4cd00cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:56.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd619b700 1 -- 172.21.15.49:0/1970658132 >> v2:172.21.15.107:3300/0 conn(0x7f4cd00cbe40 msgr2=0x7f4cd00cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:56.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd619b700 1 --2- 172.21.15.49:0/1970658132 >> v2:172.21.15.107:3300/0 conn(0x7f4cd00cbe40 0x7f4cd00cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:56.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd619b700 1 -- 172.21.15.49:0/1970658132 >> v1:172.21.15.49:6789/0 conn(0x7f4cd00cdf80 legacy=0x7f4cd015c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:56.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd619b700 1 -- 172.21.15.49:0/1970658132 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd00ce8c0 con 0x7f4cd0158bb0 2023-07-18T03:39:56.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd599a700 1 --2- 172.21.15.49:0/1970658132 >> v2:172.21.15.107:3300/0 conn(0x7f4cd00cbe40 0x7f4cd00cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:56.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.573+0000 7f4cd5199700 1 -- 172.21.15.49:0/1970658132 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762384067 0 0) 0x7f4cd00ce3a0 con 0x7f4cd00cdf80 2023-07-18T03:39:56.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.574+0000 7f4cd619b700 1 --2- 172.21.15.49:0/1970658132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd0158bb0 0x7f4cd00cd0a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0072040 tx=0x7f4ccc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a43e4ffb606a0a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:56.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.574+0000 7f4cd5199700 1 -- 172.21.15.49:0/1970658132 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc011830 con 0x7f4cd0158bb0 2023-07-18T03:39:56.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.574+0000 7f4cd5199700 1 -- 172.21.15.49:0/1970658132 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc011990 con 0x7f4cd0158bb0 2023-07-18T03:39:56.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.574+0000 7f4cd5199700 1 -- 172.21.15.49:0/1970658132 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc011d20 con 0x7f4cd0158bb0 2023-07-18T03:39:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.574+0000 7f4cd83ff700 1 -- 172.21.15.49:0/1970658132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd0158bb0 msgr2=0x7f4cd00cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.574+0000 7f4cd83ff700 1 --2- 172.21.15.49:0/1970658132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd0158bb0 0x7f4cd00cd0a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0072040 tx=0x7f4ccc009b20 comp rx=0 tx=0).stop 2023-07-18T03:39:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.575+0000 7f4cd83ff700 1 -- 172.21.15.49:0/1970658132 shutdown_connections 2023-07-18T03:39:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.575+0000 7f4cd83ff700 1 --2- 172.21.15.49:0/1970658132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd0158bb0 0x7f4cd00cd0a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.575+0000 7f4cd83ff700 1 --2- 172.21.15.49:0/1970658132 >> v2:172.21.15.107:3300/0 conn(0x7f4cd00cbe40 0x7f4cd00cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.575+0000 7f4cd83ff700 1 -- 172.21.15.49:0/1970658132 >> 172.21.15.49:0/1970658132 conn(0x7f4cd00b05d0 msgr2=0x7f4cd00b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.575+0000 7f4cd83ff700 1 -- 172.21.15.49:0/1970658132 shutdown_connections 2023-07-18T03:39:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.575+0000 7f4cd83ff700 1 -- 172.21.15.49:0/1970658132 wait complete. 2023-07-18T03:39:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.577+0000 7f4cd83ff700 1 Processor -- start 2023-07-18T03:39:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.577+0000 7f4cd83ff700 1 -- start start 2023-07-18T03:39:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd83ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cd00cbe40 0x7f4cd0152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd83ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd00cdf80 0x7f4cd00c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd83ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cd0158bb0 0x7f4cd0156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:56.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd83ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd00c3be0 con 0x7f4cd00cdf80 2023-07-18T03:39:56.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd83ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd0153de0 con 0x7f4cd00cbe40 2023-07-18T03:39:56.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd83ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd01540c0 con 0x7f4cd0158bb0 2023-07-18T03:39:56.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd599a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd00cdf80 0x7f4cd00c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:56.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd599a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd00cdf80 0x7f4cd00c25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40090/0 (socket says 172.21.15.49:40090) 2023-07-18T03:39:56.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd599a700 1 -- 172.21.15.49:0/3075287828 learned_addr learned my addr 172.21.15.49:0/3075287828 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:56.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd619b700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cd00cbe40 0x7f4cd0152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd699c700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cd0158bb0 0x7f4cd0156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd599a700 1 -- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cd0158bb0 msgr2=0x7f4cd0156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd599a700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cd0158bb0 0x7f4cd0156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd599a700 1 -- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cd00cbe40 msgr2=0x7f4cd0152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.579+0000 7f4cd599a700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cd00cbe40 0x7f4cd0152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.580+0000 7f4cd599a700 1 -- 172.21.15.49:0/3075287828 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc0094a0 con 0x7f4cd00cdf80 2023-07-18T03:39:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.580+0000 7f4cd599a700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd00cdf80 0x7f4cd00c25f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000e7a0 tx=0x7f4cc000eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.580+0000 7f4cc6ffd700 1 -- 172.21.15.49:0/3075287828 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0013610 con 0x7f4cd00cdf80 2023-07-18T03:39:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.580+0000 7f4cc6ffd700 1 -- 172.21.15.49:0/3075287828 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc0013770 con 0x7f4cd00cdf80 2023-07-18T03:39:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.580+0000 7f4cd83ff700 1 -- 172.21.15.49:0/3075287828 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd01543a0 con 0x7f4cd00cdf80 2023-07-18T03:39:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.580+0000 7f4cc6ffd700 1 -- 172.21.15.49:0/3075287828 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0013b00 con 0x7f4cd00cdf80 2023-07-18T03:39:56.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.580+0000 7f4cd83ff700 1 -- 172.21.15.49:0/3075287828 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd0154960 con 0x7f4cd00cdf80 2023-07-18T03:39:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.581+0000 7f4cd83ff700 1 -- 172.21.15.49:0/3075287828 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd0074350 con 0x7f4cd00cdf80 2023-07-18T03:39:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.582+0000 7f4cc6ffd700 1 -- 172.21.15.49:0/3075287828 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4cc0013c60 con 0x7f4cd00cdf80 2023-07-18T03:39:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.583+0000 7f4cc6ffd700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4cb406ace0 0x7f4cb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:56.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.583+0000 7f4cc6ffd700 1 -- 172.21.15.49:0/3075287828 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f4cc0088ef0 con 0x7f4cd00cdf80 2023-07-18T03:39:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.584+0000 7f4cd619b700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4cb406ace0 0x7f4cb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:56.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.585+0000 7f4cd619b700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4cb406ace0 0x7f4cb406d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0090b0 tx=0x7f4ccc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:56.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.588+0000 7f4cc6ffd700 1 -- 172.21.15.49:0/3075287828 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4cc0057b20 con 0x7f4cd00cdf80 2023-07-18T03:39:56.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:56.843+0000 7f4cd83ff700 1 -- 172.21.15.49:0/3075287828 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4cd00b4a40 con 0x7f4cd00cdf80 2023-07-18T03:39:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.329+0000 7f4cc6ffd700 1 -- 172.21.15.49:0/3075287828 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4cc00576b0 con 0x7f4cd00cdf80 2023-07-18T03:39:57.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.337+0000 7f4cd83ff700 1 -- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4cb406ace0 msgr2=0x7f4cb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:57.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.337+0000 7f4cd83ff700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4cb406ace0 0x7f4cb406d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0090b0 tx=0x7f4ccc01c040 comp rx=0 tx=0).stop 2023-07-18T03:39:57.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.337+0000 7f4cd83ff700 1 -- 172.21.15.49:0/3075287828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd00cdf80 msgr2=0x7f4cd00c25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:57.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.337+0000 7f4cd83ff700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd00cdf80 0x7f4cd00c25f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000e7a0 tx=0x7f4cc000eab0 comp rx=0 tx=0).stop 2023-07-18T03:39:57.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.338+0000 7f4cd83ff700 1 -- 172.21.15.49:0/3075287828 shutdown_connections 2023-07-18T03:39:57.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.338+0000 7f4cd83ff700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cd0158bb0 0x7f4cd0156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:57.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.338+0000 7f4cd83ff700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cd00cbe40 0x7f4cd0152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:57.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.338+0000 7f4cd83ff700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4cb406ace0 0x7f4cb406d190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:57.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.338+0000 7f4cd83ff700 1 --2- 172.21.15.49:0/3075287828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd00cdf80 0x7f4cd00c25f0 secure :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000e7a0 tx=0x7f4cc000eab0 comp rx=0 tx=0).stop 2023-07-18T03:39:57.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.338+0000 7f4cd83ff700 1 -- 172.21.15.49:0/3075287828 >> 172.21.15.49:0/3075287828 conn(0x7f4cd00b05d0 msgr2=0x7f4cd006f730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:57.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.340+0000 7f4cd83ff700 1 -- 172.21.15.49:0/3075287828 shutdown_connections 2023-07-18T03:39:57.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.340+0000 7f4cd83ff700 1 -- 172.21.15.49:0/3075287828 wait complete. 2023-07-18T03:39:57.339 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 5 and data pool 6 2023-07-18T03:39:57.367 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:57.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.579+0000 7f55e6bff700 1 Processor -- start 2023-07-18T03:39:57.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.579+0000 7f55e6bff700 1 -- start start 2023-07-18T03:39:57.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.580+0000 7f55e6bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f55e00d1220 0x7f55e00b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:57.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.580+0000 7f55e6bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e00d2ce0 con 0x7f55e015c950 2023-07-18T03:39:57.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.580+0000 7f55e6bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e00b3350 con 0x7f55e00b1e00 2023-07-18T03:39:57.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.580+0000 7f55e6bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e00b3630 con 0x7f55e00d1220 2023-07-18T03:39:57.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.580+0000 7f55dffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f55e00d1220 0x7f55e00b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:57.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.580+0000 7f55e499b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f55e015c950 0x7f55e00d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37676/0 (socket says 172.21.15.49:37676) 2023-07-18T03:39:57.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.580+0000 7f55e499b700 1 -- 172.21.15.49:0/2488473286 learned_addr learned my addr 172.21.15.49:0/2488473286 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:39:57.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.581+0000 7f55dffff700 1 -- 172.21.15.49:0/2488473286 >> v1:172.21.15.107:6789/0 conn(0x7f55e00b1e00 legacy=0x7f55e00afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:57.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.581+0000 7f55dffff700 1 -- 172.21.15.49:0/2488473286 >> v1:172.21.15.49:6789/0 conn(0x7f55e015c950 legacy=0x7f55e00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.581+0000 7f55dffff700 1 -- 172.21.15.49:0/2488473286 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e00b3910 con 0x7f55e00d1220 2023-07-18T03:39:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.581+0000 7f55df7fe700 1 -- 172.21.15.49:0/2488473286 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329201428 0 0) 0x7f55e00d2ce0 con 0x7f55e015c950 2023-07-18T03:39:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.581+0000 7f55dffff700 1 --2- 172.21.15.49:0/2488473286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55e00d1220 0x7f55e00b43f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009840 tx=0x7f55cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46c69c601799b15 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.581+0000 7f55df7fe700 1 -- 172.21.15.49:0/2488473286 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc011830 con 0x7f55e00d1220 2023-07-18T03:39:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.582+0000 7f55df7fe700 1 -- 172.21.15.49:0/2488473286 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55cc011990 con 0x7f55e00d1220 2023-07-18T03:39:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.582+0000 7f55df7fe700 1 -- 172.21.15.49:0/2488473286 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc011d20 con 0x7f55e00d1220 2023-07-18T03:39:57.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.582+0000 7f55e6bff700 1 -- 172.21.15.49:0/2488473286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55e00d1220 msgr2=0x7f55e00b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:57.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.582+0000 7f55e6bff700 1 --2- 172.21.15.49:0/2488473286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55e00d1220 0x7f55e00b43f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009840 tx=0x7f55cc009b50 comp rx=0 tx=0).stop 2023-07-18T03:39:57.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.582+0000 7f55e6bff700 1 -- 172.21.15.49:0/2488473286 shutdown_connections 2023-07-18T03:39:57.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.582+0000 7f55e6bff700 1 --2- 172.21.15.49:0/2488473286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55e00d1220 0x7f55e00b43f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:57.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.582+0000 7f55e6bff700 1 -- 172.21.15.49:0/2488473286 >> 172.21.15.49:0/2488473286 conn(0x7f55e009c1d0 msgr2=0x7f55e00d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:57.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.582+0000 7f55e6bff700 1 -- 172.21.15.49:0/2488473286 shutdown_connections 2023-07-18T03:39:57.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.582+0000 7f55e6bff700 1 -- 172.21.15.49:0/2488473286 wait complete. 2023-07-18T03:39:57.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.584+0000 7f55e6bff700 1 Processor -- start 2023-07-18T03:39:57.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.584+0000 7f55e6bff700 1 -- start start 2023-07-18T03:39:57.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.585+0000 7f55e6bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00b1e00 0x7f55e00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:57.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.585+0000 7f55e6bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55e00d1220 0x7f55e01546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:57.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.585+0000 7f55e6bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55e015c950 0x7f55e00ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:57.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.585+0000 7f55e6bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e0154c30 con 0x7f55e00d1220 2023-07-18T03:39:57.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.585+0000 7f55e6bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00cf2d0 con 0x7f55e00b1e00 2023-07-18T03:39:57.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.585+0000 7f55e6bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00cf550 con 0x7f55e015c950 2023-07-18T03:39:57.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55dffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55e00d1220 0x7f55e01546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:57.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55dffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55e00d1220 0x7f55e01546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40118/0 (socket says 172.21.15.49:40118) 2023-07-18T03:39:57.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55dffff700 1 -- 172.21.15.49:0/3811669878 learned_addr learned my addr 172.21.15.49:0/3811669878 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:57.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55e499b700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00b1e00 0x7f55e00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:57.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55e519c700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55e015c950 0x7f55e00ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:57.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55dffff700 1 -- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55e015c950 msgr2=0x7f55e00ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:57.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55dffff700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55e015c950 0x7f55e00ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:57.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55dffff700 1 -- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00b1e00 msgr2=0x7f55e00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:57.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55dffff700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00b1e00 0x7f55e00ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:57.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55dffff700 1 -- 172.21.15.49:0/3811669878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc0094a0 con 0x7f55e00d1220 2023-07-18T03:39:57.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55e499b700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00b1e00 0x7f55e00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:57.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55dffff700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55e00d1220 0x7f55e01546f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f55cc013000 tx=0x7f55cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:57.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.586+0000 7f55dd7fa700 1 -- 172.21.15.49:0/3811669878 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0036a0 con 0x7f55e00d1220 2023-07-18T03:39:57.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.587+0000 7f55dd7fa700 1 -- 172.21.15.49:0/3811669878 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55cc01f070 con 0x7f55e00d1220 2023-07-18T03:39:57.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.587+0000 7f55dd7fa700 1 -- 172.21.15.49:0/3811669878 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc003a60 con 0x7f55e00d1220 2023-07-18T03:39:57.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.587+0000 7f55e6bff700 1 -- 172.21.15.49:0/3811669878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e00cd5f0 con 0x7f55e00d1220 2023-07-18T03:39:57.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.587+0000 7f55e6bff700 1 -- 172.21.15.49:0/3811669878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e00cdbb0 con 0x7f55e00d1220 2023-07-18T03:39:57.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.588+0000 7f55e6bff700 1 -- 172.21.15.49:0/3811669878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e004e280 con 0x7f55e00d1220 2023-07-18T03:39:57.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.589+0000 7f55dd7fa700 1 -- 172.21.15.49:0/3811669878 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f55cc003bc0 con 0x7f55e00d1220 2023-07-18T03:39:57.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.590+0000 7f55dd7fa700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f55c406ace0 0x7f55c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:57.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.590+0000 7f55dd7fa700 1 -- 172.21.15.49:0/3811669878 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f55cc021070 con 0x7f55e00d1220 2023-07-18T03:39:57.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.590+0000 7f55e499b700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f55c406ace0 0x7f55c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:57.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.591+0000 7f55e499b700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f55c406ace0 0x7f55c406d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55d40010e0 tx=0x7f55d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:57.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.594+0000 7f55dd7fa700 1 -- 172.21.15.49:0/3811669878 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f55cc056110 con 0x7f55e00d1220 2023-07-18T03:39:57.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.801+0000 7f55e6bff700 1 -- 172.21.15.49:0/3811669878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55e008c5a0 con 0x7f55e00d1220 2023-07-18T03:39:57.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.802+0000 7f55dd7fa700 1 -- 172.21.15.49:0/3811669878 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18270 (secure 0 0 0) 0x7f55cc059730 con 0x7f55e00d1220 2023-07-18T03:39:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:57.798 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":35,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:56.844490+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:39:54.412334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-18T03:39:55.841218+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:57.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.804+0000 7f55e6bff700 1 -- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f55c406ace0 msgr2=0x7f55c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:57.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.805+0000 7f55e6bff700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f55c406ace0 0x7f55c406d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55d40010e0 tx=0x7f55d4009040 comp rx=0 tx=0).stop 2023-07-18T03:39:57.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.805+0000 7f55e6bff700 1 -- 172.21.15.49:0/3811669878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55e00d1220 msgr2=0x7f55e01546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:57.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.805+0000 7f55e6bff700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55e00d1220 0x7f55e01546f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f55cc013000 tx=0x7f55cc004750 comp rx=0 tx=0).stop 2023-07-18T03:39:57.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.805+0000 7f55e6bff700 1 -- 172.21.15.49:0/3811669878 shutdown_connections 2023-07-18T03:39:57.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.805+0000 7f55e6bff700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55e015c950 0x7f55e00ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:57.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.805+0000 7f55e6bff700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00b1e00 0x7f55e00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:57.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.805+0000 7f55e6bff700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f55c406ace0 0x7f55c406d190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:57.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.805+0000 7f55e6bff700 1 --2- 172.21.15.49:0/3811669878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55e00d1220 0x7f55e01546f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:57.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.805+0000 7f55e6bff700 1 -- 172.21.15.49:0/3811669878 >> 172.21.15.49:0/3811669878 conn(0x7f55e009c1d0 msgr2=0x7f55e00b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:57.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.805+0000 7f55e6bff700 1 -- 172.21.15.49:0/3811669878 shutdown_connections 2023-07-18T03:39:57.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:57.805+0000 7f55e6bff700 1 -- 172.21.15.49:0/3811669878 wait complete. 2023-07-18T03:39:57.818 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:58.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.029+0000 7f2a6d5ff700 1 Processor -- start 2023-07-18T03:39:58.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.029+0000 7f2a6d5ff700 1 -- start start 2023-07-18T03:39:58.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.030+0000 7f2a6d5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a6815c950 0x7f2a680b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.030+0000 7f2a6d5ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2a680d1220 0x7f2a680afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.030+0000 7f2a6d5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a680d2ce0 con 0x7f2a680b1e00 2023-07-18T03:39:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.030+0000 7f2a6d5ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a680b3350 con 0x7f2a680d1220 2023-07-18T03:39:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.030+0000 7f2a6d5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a680b3630 con 0x7f2a6815c950 2023-07-18T03:39:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.030+0000 7f2a677fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2a680d1220 0x7f2a680afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:58.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.030+0000 7f2a677fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2a680d1220 0x7f2a680afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40134/0 (socket says 172.21.15.49:40134) 2023-07-18T03:39:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.030+0000 7f2a677fe700 1 -- 172.21.15.49:0/2157873073 learned_addr learned my addr 172.21.15.49:0/2157873073 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:58.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.031+0000 7f2a66ffd700 1 --2- 172.21.15.49:0/2157873073 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a6815c950 0x7f2a680b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.031+0000 7f2a677fe700 1 -- 172.21.15.49:0/2157873073 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a6815c950 msgr2=0x7f2a680b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.031+0000 7f2a677fe700 1 --2- 172.21.15.49:0/2157873073 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a6815c950 0x7f2a680b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.031+0000 7f2a677fe700 1 -- 172.21.15.49:0/2157873073 >> v1:172.21.15.107:6789/0 conn(0x7f2a680b1e00 legacy=0x7f2a680d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.031+0000 7f2a677fe700 1 -- 172.21.15.49:0/2157873073 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a680b3910 con 0x7f2a680d1220 2023-07-18T03:39:58.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.031+0000 7f2a677fe700 1 --2- 172.21.15.49:0/2157873073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a680d1220 0x7f2a680afac0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009e50 tx=0x7f2a6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac783141dbf5c12e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.031+0000 7f2a65ffb700 1 -- 172.21.15.49:0/2157873073 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000e070 con 0x7f2a680d1220 2023-07-18T03:39:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.031+0000 7f2a65ffb700 1 -- 172.21.15.49:0/2157873073 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6000b910 con 0x7f2a680d1220 2023-07-18T03:39:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.031+0000 7f2a65ffb700 1 -- 172.21.15.49:0/2157873073 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000bcc0 con 0x7f2a680d1220 2023-07-18T03:39:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.032+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2157873073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a680d1220 msgr2=0x7f2a680afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:58.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.032+0000 7f2a6d5ff700 1 --2- 172.21.15.49:0/2157873073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a680d1220 0x7f2a680afac0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009e50 tx=0x7f2a6000b330 comp rx=0 tx=0).stop 2023-07-18T03:39:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.032+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2157873073 shutdown_connections 2023-07-18T03:39:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.032+0000 7f2a6d5ff700 1 --2- 172.21.15.49:0/2157873073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a680d1220 0x7f2a680afac0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.032+0000 7f2a6d5ff700 1 --2- 172.21.15.49:0/2157873073 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a6815c950 0x7f2a680b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.032+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2157873073 >> 172.21.15.49:0/2157873073 conn(0x7f2a6809c1d0 msgr2=0x7f2a680d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.032+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2157873073 shutdown_connections 2023-07-18T03:39:58.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.032+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2157873073 wait complete. 2023-07-18T03:39:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.034+0000 7f2a6d5ff700 1 Processor -- start 2023-07-18T03:39:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.034+0000 7f2a6d5ff700 1 -- start start 2023-07-18T03:39:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.035+0000 7f2a6d5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a680b1e00 0x7f2a680c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.035+0000 7f2a6d5ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a680d1220 0x7f2a6814a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:58.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.035+0000 7f2a6d5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a6815c950 0x7f2a680b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:58.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.035+0000 7f2a6d5ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a680b8110 con 0x7f2a680d1220 2023-07-18T03:39:58.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.035+0000 7f2a6d5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a68157af0 con 0x7f2a680b1e00 2023-07-18T03:39:58.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.035+0000 7f2a6d5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a680b7490 con 0x7f2a6815c950 2023-07-18T03:39:58.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.035+0000 7f2a667fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a680d1220 0x7f2a6814a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:58.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a667fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a680d1220 0x7f2a6814a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40138/0 (socket says 172.21.15.49:40138) 2023-07-18T03:39:58.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a667fc700 1 -- 172.21.15.49:0/2151084873 learned_addr learned my addr 172.21.15.49:0/2151084873 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:58.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a66ffd700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a680b1e00 0x7f2a680c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:58.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a677fe700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a6815c950 0x7f2a680b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:58.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a667fc700 1 -- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a6815c950 msgr2=0x7f2a680b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:58.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a667fc700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a6815c950 0x7f2a680b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a667fc700 1 -- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a680b1e00 msgr2=0x7f2a680c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:58.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a667fc700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a680b1e00 0x7f2a680c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a667fc700 1 -- 172.21.15.49:0/2151084873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a60009b30 con 0x7f2a680d1220 2023-07-18T03:39:58.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a667fc700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a680d1220 0x7f2a6814a470 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00c4e0 tx=0x7f2a5c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:58.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a4f7fe700 1 -- 172.21.15.49:0/2151084873 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0052d0 con 0x7f2a680d1220 2023-07-18T03:39:58.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a4f7fe700 1 -- 172.21.15.49:0/2151084873 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c005430 con 0x7f2a680d1220 2023-07-18T03:39:58.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2151084873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a680b7770 con 0x7f2a680d1220 2023-07-18T03:39:58.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.036+0000 7f2a4f7fe700 1 -- 172.21.15.49:0/2151084873 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0038c0 con 0x7f2a680d1220 2023-07-18T03:39:58.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.037+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2151084873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a680b7d30 con 0x7f2a680d1220 2023-07-18T03:39:58.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.037+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2151084873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a680b7ec0 con 0x7f2a680d1220 2023-07-18T03:39:58.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.039+0000 7f2a4f7fe700 1 -- 172.21.15.49:0/2151084873 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a5c003a20 con 0x7f2a680d1220 2023-07-18T03:39:58.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.040+0000 7f2a4f7fe700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2a5406ace0 0x7f2a5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:58.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.040+0000 7f2a4f7fe700 1 -- 172.21.15.49:0/2151084873 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2a5c0887d0 con 0x7f2a680d1220 2023-07-18T03:39:58.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.040+0000 7f2a66ffd700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2a5406ace0 0x7f2a5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:58.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.041+0000 7f2a66ffd700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2a5406ace0 0x7f2a5406d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009b20 tx=0x7f2a580091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:58.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.044+0000 7f2a4f7fe700 1 -- 172.21.15.49:0/2151084873 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2a5c053910 con 0x7f2a680d1220 2023-07-18T03:39:58.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.253+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2151084873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a680b8250 con 0x7f2a680d1220 2023-07-18T03:39:58.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.254+0000 7f2a4f7fe700 1 -- 172.21.15.49:0/2151084873 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18270 (secure 0 0 0) 0x7f2a680b8250 con 0x7f2a680d1220 2023-07-18T03:39:58.249 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:39:58.250 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":35,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:56.844490+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:39:54.412334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-18T03:39:55.841218+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:58.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.256+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2a5406ace0 msgr2=0x7f2a5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:58.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.256+0000 7f2a6d5ff700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2a5406ace0 0x7f2a5406d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009b20 tx=0x7f2a580091c0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.256+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2151084873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a680d1220 msgr2=0x7f2a6814a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.256+0000 7f2a6d5ff700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a680d1220 0x7f2a6814a470 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00c4e0 tx=0x7f2a5c00c7f0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.256+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2151084873 shutdown_connections 2023-07-18T03:39:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.256+0000 7f2a6d5ff700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a6815c950 0x7f2a680b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.256+0000 7f2a6d5ff700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a680b1e00 0x7f2a680c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.256+0000 7f2a6d5ff700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2a5406ace0 0x7f2a5406d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.256+0000 7f2a6d5ff700 1 --2- 172.21.15.49:0/2151084873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a680d1220 0x7f2a6814a470 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.256+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2151084873 >> 172.21.15.49:0/2151084873 conn(0x7f2a6809c1d0 msgr2=0x7f2a680b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:58.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.257+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2151084873 shutdown_connections 2023-07-18T03:39:58.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.257+0000 7f2a6d5ff700 1 -- 172.21.15.49:0/2151084873 wait complete. 2023-07-18T03:39:58.270 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-18T03:39:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.488+0000 7fe8f37ff700 1 Processor -- start 2023-07-18T03:39:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.489+0000 7fe8f37ff700 1 -- start start 2023-07-18T03:39:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.490+0000 7fe8f37ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8ec0ce030 0x7fe8ec069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.490+0000 7fe8f37ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec0cdba0 con 0x7fe8ec06a470 2023-07-18T03:39:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.490+0000 7fe8f37ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec0cf370 con 0x7fe8ec06f010 2023-07-18T03:39:58.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.490+0000 7fe8f37ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ec06f3e0 con 0x7fe8ec0ce030 2023-07-18T03:39:58.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.490+0000 7fe8f0d9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8ec0ce030 0x7fe8ec069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:58.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.490+0000 7fe8f0d9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8ec0ce030 0x7fe8ec069f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40154/0 (socket says 172.21.15.49:40154) 2023-07-18T03:39:58.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.490+0000 7fe8f0d9a700 1 -- 172.21.15.49:0/103252932 learned_addr learned my addr 172.21.15.49:0/103252932 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:58.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.490+0000 7fe8f1d9c700 1 --1- 172.21.15.49:0/103252932 >> v1:172.21.15.49:6789/0 conn(0x7fe8ec06a470 0x7fe8ec1601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37678/0 (socket says 172.21.15.49:37678) 2023-07-18T03:39:58.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.491+0000 7fe8f0d9a700 1 -- 172.21.15.49:0/103252932 >> v1:172.21.15.107:6789/0 conn(0x7fe8ec06f010 legacy=0x7fe8ec071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:58.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.491+0000 7fe8f0d9a700 1 -- 172.21.15.49:0/103252932 >> v1:172.21.15.49:6789/0 conn(0x7fe8ec06a470 legacy=0x7fe8ec1601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:58.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.491+0000 7fe8f0d9a700 1 -- 172.21.15.49:0/103252932 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec0cf650 con 0x7fe8ec0ce030 2023-07-18T03:39:58.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.491+0000 7fe8e3fff700 1 -- 172.21.15.49:0/103252932 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697109667 0 0) 0x7fe8ec0cdba0 con 0x7fe8ec06a470 2023-07-18T03:39:58.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.491+0000 7fe8f0d9a700 1 --2- 172.21.15.49:0/103252932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8ec0ce030 0x7fe8ec069f30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0098f0 tx=0x7fe8dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f9cc04379f1646 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:58.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.491+0000 7fe8e3fff700 1 -- 172.21.15.49:0/103252932 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00b8a0 con 0x7fe8ec0ce030 2023-07-18T03:39:58.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.491+0000 7fe8e3fff700 1 -- 172.21.15.49:0/103252932 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc00ba00 con 0x7fe8ec0ce030 2023-07-18T03:39:58.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.491+0000 7fe8e3fff700 1 -- 172.21.15.49:0/103252932 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00bd90 con 0x7fe8ec0ce030 2023-07-18T03:39:58.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.492+0000 7fe8f37ff700 1 -- 172.21.15.49:0/103252932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8ec0ce030 msgr2=0x7fe8ec069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:58.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.492+0000 7fe8f37ff700 1 --2- 172.21.15.49:0/103252932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8ec0ce030 0x7fe8ec069f30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0098f0 tx=0x7fe8dc009c00 comp rx=0 tx=0).stop 2023-07-18T03:39:58.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.492+0000 7fe8f37ff700 1 -- 172.21.15.49:0/103252932 shutdown_connections 2023-07-18T03:39:58.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.492+0000 7fe8f37ff700 1 --2- 172.21.15.49:0/103252932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8ec0ce030 0x7fe8ec069f30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.492+0000 7fe8f37ff700 1 -- 172.21.15.49:0/103252932 >> 172.21.15.49:0/103252932 conn(0x7fe8ec0cc640 msgr2=0x7fe8ec0cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:58.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.492+0000 7fe8f37ff700 1 -- 172.21.15.49:0/103252932 shutdown_connections 2023-07-18T03:39:58.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.492+0000 7fe8f37ff700 1 -- 172.21.15.49:0/103252932 wait complete. 2023-07-18T03:39:58.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.494+0000 7fe8f37ff700 1 Processor -- start 2023-07-18T03:39:58.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.495+0000 7fe8f37ff700 1 -- start start 2023-07-18T03:39:58.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.496+0000 7fe8f37ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec06a470 0x7fe8ec1e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:58.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.496+0000 7fe8f37ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8ec06f010 0x7fe8ec14ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:58.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.496+0000 7fe8f37ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8ec0ce030 0x7fe8ec14ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:58.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.496+0000 7fe8f37ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec14f4a0 con 0x7fe8ec06f010 2023-07-18T03:39:58.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.496+0000 7fe8f37ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0b3990 con 0x7fe8ec06a470 2023-07-18T03:39:58.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.496+0000 7fe8f37ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0b3c70 con 0x7fe8ec0ce030 2023-07-18T03:39:58.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.496+0000 7fe8f0d9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8ec06f010 0x7fe8ec14ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:58.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.496+0000 7fe8f0d9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8ec06f010 0x7fe8ec14ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40160/0 (socket says 172.21.15.49:40160) 2023-07-18T03:39:58.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.496+0000 7fe8f0d9a700 1 -- 172.21.15.49:0/945389331 learned_addr learned my addr 172.21.15.49:0/945389331 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:58.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.496+0000 7fe8f159b700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec06a470 0x7fe8ec1e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:58.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.496+0000 7fe8f1d9c700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8ec0ce030 0x7fe8ec14ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:58.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.497+0000 7fe8f0d9a700 1 -- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8ec0ce030 msgr2=0x7fe8ec14ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:58.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.497+0000 7fe8f0d9a700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8ec0ce030 0x7fe8ec14ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.497+0000 7fe8f0d9a700 1 -- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec06a470 msgr2=0x7fe8ec1e7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:58.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.497+0000 7fe8f0d9a700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec06a470 0x7fe8ec1e7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:58.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.497+0000 7fe8f0d9a700 1 -- 172.21.15.49:0/945389331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc009550 con 0x7fe8ec06f010 2023-07-18T03:39:58.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.497+0000 7fe8f159b700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec06a470 0x7fe8ec1e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:58.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.497+0000 7fe8f0d9a700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8ec06f010 0x7fe8ec14ea20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc000c00 tx=0x7fe8dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:58.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.497+0000 7fe8e1ffb700 1 -- 172.21.15.49:0/945389331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc011b60 con 0x7fe8ec06f010 2023-07-18T03:39:58.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.497+0000 7fe8e1ffb700 1 -- 172.21.15.49:0/945389331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc01f070 con 0x7fe8ec06f010 2023-07-18T03:39:58.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.497+0000 7fe8e1ffb700 1 -- 172.21.15.49:0/945389331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc026480 con 0x7fe8ec06f010 2023-07-18T03:39:58.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.497+0000 7fe8f37ff700 1 -- 172.21.15.49:0/945389331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ec0b3f50 con 0x7fe8ec06f010 2023-07-18T03:39:58.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.498+0000 7fe8f37ff700 1 -- 172.21.15.49:0/945389331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec0c4230 con 0x7fe8ec06f010 2023-07-18T03:39:58.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.498+0000 7fe8f37ff700 1 -- 172.21.15.49:0/945389331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec0b1cf0 con 0x7fe8ec06f010 2023-07-18T03:39:58.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.499+0000 7fe8e1ffb700 1 -- 172.21.15.49:0/945389331 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe8dc01a430 con 0x7fe8ec06f010 2023-07-18T03:39:58.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.500+0000 7fe8e1ffb700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe8d006ace0 0x7fe8d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:58.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.500+0000 7fe8e1ffb700 1 -- 172.21.15.49:0/945389331 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe8dc021070 con 0x7fe8ec06f010 2023-07-18T03:39:58.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.501+0000 7fe8f159b700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe8d006ace0 0x7fe8d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:58.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.501+0000 7fe8f159b700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe8d006ace0 0x7fe8d006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000f60 tx=0x7fe8e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:58.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.505+0000 7fe8e1ffb700 1 -- 172.21.15.49:0/945389331 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe8dc08e030 con 0x7fe8ec06f010 2023-07-18T03:39:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:58.763+0000 7fe8f37ff700 1 -- 172.21.15.49:0/945389331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe8ec070e40 con 0x7fe8ec06f010 2023-07-18T03:39:59.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.357+0000 7fe8e1ffb700 1 -- 172.21.15.49:0/945389331 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe8dc056050 con 0x7fe8ec06f010 2023-07-18T03:39:59.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.360+0000 7fe8f37ff700 1 -- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe8d006ace0 msgr2=0x7fe8d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:59.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.360+0000 7fe8f37ff700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe8d006ace0 0x7fe8d006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000f60 tx=0x7fe8e8009040 comp rx=0 tx=0).stop 2023-07-18T03:39:59.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.360+0000 7fe8f37ff700 1 -- 172.21.15.49:0/945389331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8ec06f010 msgr2=0x7fe8ec14ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.360+0000 7fe8f37ff700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8ec06f010 0x7fe8ec14ea20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc000c00 tx=0x7fe8dc004750 comp rx=0 tx=0).stop 2023-07-18T03:39:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.360+0000 7fe8f37ff700 1 -- 172.21.15.49:0/945389331 shutdown_connections 2023-07-18T03:39:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.360+0000 7fe8f37ff700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8ec0ce030 0x7fe8ec14ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.360+0000 7fe8f37ff700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec06a470 0x7fe8ec1e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.360+0000 7fe8f37ff700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe8d006ace0 0x7fe8d006d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.360+0000 7fe8f37ff700 1 --2- 172.21.15.49:0/945389331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8ec06f010 0x7fe8ec14ea20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.360+0000 7fe8f37ff700 1 -- 172.21.15.49:0/945389331 >> 172.21.15.49:0/945389331 conn(0x7fe8ec0cc640 msgr2=0x7fe8ec0cb980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:59.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.360+0000 7fe8f37ff700 1 -- 172.21.15.49:0/945389331 shutdown_connections 2023-07-18T03:39:59.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.360+0000 7fe8f37ff700 1 -- 172.21.15.49:0/945389331 wait complete. 2023-07-18T03:39:59.375 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-18T03:39:59.376 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-18T03:39:59.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.593+0000 7f2af6dff700 1 Processor -- start 2023-07-18T03:39:59.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.593+0000 7f2af6dff700 1 -- start start 2023-07-18T03:39:59.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.594+0000 7f2af6dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2af00b8a60 0x7f2af00b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:59.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.594+0000 7f2af6dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af00b4530 0x7f2af00d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:59.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.594+0000 7f2af6dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af00b8e30 con 0x7f2af00d1050 2023-07-18T03:39:59.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.594+0000 7f2af6dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af00d14d0 con 0x7f2af00b8a60 2023-07-18T03:39:59.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.594+0000 7f2af6dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00b1e30 con 0x7f2af00b4530 2023-07-18T03:39:59.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.595+0000 7f2af4b9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2af00b8a60 0x7f2af00b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:59.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.595+0000 7f2aeffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af00b4530 0x7f2af00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:59.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.595+0000 7f2af4b9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2af00b8a60 0x7f2af00b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:57886/0 (socket says 172.21.15.49:57886) 2023-07-18T03:39:59.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.595+0000 7f2af4b9b700 1 -- 172.21.15.49:0/1563272683 learned_addr learned my addr 172.21.15.49:0/1563272683 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:59.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.595+0000 7f2af4b9b700 1 -- 172.21.15.49:0/1563272683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af00b4530 msgr2=0x7f2af00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:59.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.595+0000 7f2af4b9b700 1 --2- 172.21.15.49:0/1563272683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af00b4530 0x7f2af00d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:59.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.596+0000 7f2af4b9b700 1 -- 172.21.15.49:0/1563272683 >> v1:172.21.15.107:6789/0 conn(0x7f2af00d1050 legacy=0x7f2af00b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:59.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.596+0000 7f2af4b9b700 1 -- 172.21.15.49:0/1563272683 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af00afaa0 con 0x7f2af00b8a60 2023-07-18T03:39:59.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.596+0000 7f2aeffff700 1 --2- 172.21.15.49:0/1563272683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af00b4530 0x7f2af00d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:59.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.596+0000 7f2af4b9b700 1 --2- 172.21.15.49:0/1563272683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af00b8a60 0x7f2af00b3ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40098f0 tx=0x7f2ae4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c4a5cd99b047ed1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:59.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.596+0000 7f2aef7fe700 1 -- 172.21.15.49:0/1563272683 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4011830 con 0x7f2af00b8a60 2023-07-18T03:39:59.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.596+0000 7f2aef7fe700 1 -- 172.21.15.49:0/1563272683 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae4011990 con 0x7f2af00b8a60 2023-07-18T03:39:59.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.596+0000 7f2aef7fe700 1 -- 172.21.15.49:0/1563272683 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4011d20 con 0x7f2af00b8a60 2023-07-18T03:39:59.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.597+0000 7f2af6dff700 1 -- 172.21.15.49:0/1563272683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af00b8a60 msgr2=0x7f2af00b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:59.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.597+0000 7f2af6dff700 1 --2- 172.21.15.49:0/1563272683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af00b8a60 0x7f2af00b3ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40098f0 tx=0x7f2ae4009c00 comp rx=0 tx=0).stop 2023-07-18T03:39:59.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.597+0000 7f2af6dff700 1 -- 172.21.15.49:0/1563272683 shutdown_connections 2023-07-18T03:39:59.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.597+0000 7f2af6dff700 1 --2- 172.21.15.49:0/1563272683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af00b8a60 0x7f2af00b3ff0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:59.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.597+0000 7f2af6dff700 1 --2- 172.21.15.49:0/1563272683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af00b4530 0x7f2af00d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:59.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.597+0000 7f2af6dff700 1 -- 172.21.15.49:0/1563272683 >> 172.21.15.49:0/1563272683 conn(0x7f2af009c1b0 msgr2=0x7f2af00d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:59.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.597+0000 7f2af6dff700 1 -- 172.21.15.49:0/1563272683 shutdown_connections 2023-07-18T03:39:59.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.597+0000 7f2af6dff700 1 -- 172.21.15.49:0/1563272683 wait complete. 2023-07-18T03:39:59.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.599+0000 7f2af6dff700 1 Processor -- start 2023-07-18T03:39:59.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.599+0000 7f2af6dff700 1 -- start start 2023-07-18T03:39:59.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af6dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af00b4530 0x7f2af0152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:59.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af6dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af00b8a60 0x7f2af00cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:59.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af6dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af00d1050 0x7f2af015e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:59.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af6dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00cb6c0 con 0x7f2af00b4530 2023-07-18T03:39:59.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af6dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af01f2ff0 con 0x7f2af00d1050 2023-07-18T03:39:59.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af6dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af015ecc0 con 0x7f2af00b8a60 2023-07-18T03:39:59.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af4b9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af00b4530 0x7f2af0152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:59.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af4b9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af00b4530 0x7f2af0152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38320/0 (socket says 172.21.15.49:38320) 2023-07-18T03:39:59.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af4b9b700 1 -- 172.21.15.49:0/1730224147 learned_addr learned my addr 172.21.15.49:0/1730224147 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:39:59.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af539c700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af00d1050 0x7f2af015e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:59.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2aeffff700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af00b8a60 0x7f2af00cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:59.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af4b9b700 1 -- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af00b8a60 msgr2=0x7f2af00cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:59.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af4b9b700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af00b8a60 0x7f2af00cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:59.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af4b9b700 1 -- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af00d1050 msgr2=0x7f2af015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:59.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.601+0000 7f2af4b9b700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af00d1050 0x7f2af015e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:59.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.602+0000 7f2af4b9b700 1 -- 172.21.15.49:0/1730224147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc0094a0 con 0x7f2af00b4530 2023-07-18T03:39:59.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.602+0000 7f2af539c700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af00d1050 0x7f2af015e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:59.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.602+0000 7f2aeffff700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af00b8a60 0x7f2af00cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:39:59.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.602+0000 7f2af4b9b700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af00b4530 0x7f2af0152560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4000c00 tx=0x7f2ae4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:59.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.602+0000 7f2aed7fa700 1 -- 172.21.15.49:0/1730224147 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40045b0 con 0x7f2af00b4530 2023-07-18T03:39:59.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.602+0000 7f2aed7fa700 1 -- 172.21.15.49:0/1730224147 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae401f070 con 0x7f2af00b4530 2023-07-18T03:39:59.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.602+0000 7f2aed7fa700 1 -- 172.21.15.49:0/1730224147 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40038a0 con 0x7f2af00b4530 2023-07-18T03:39:59.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.602+0000 7f2af6dff700 1 -- 172.21.15.49:0/1730224147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae4009550 con 0x7f2af00b4530 2023-07-18T03:39:59.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.602+0000 7f2af6dff700 1 -- 172.21.15.49:0/1730224147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af01f2750 con 0x7f2af00b4530 2023-07-18T03:39:59.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.603+0000 7f2af6dff700 1 -- 172.21.15.49:0/1730224147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af00d23e0 con 0x7f2af00b4530 2023-07-18T03:39:59.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.604+0000 7f2aed7fa700 1 -- 172.21.15.49:0/1730224147 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2ae4021030 con 0x7f2af00b4530 2023-07-18T03:39:59.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.605+0000 7f2aed7fa700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2ad406ace0 0x7f2ad406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:59.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.606+0000 7f2aed7fa700 1 -- 172.21.15.49:0/1730224147 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2ae408ab60 con 0x7f2af00b4530 2023-07-18T03:39:59.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.606+0000 7f2aeffff700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2ad406ace0 0x7f2ad406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:59.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.607+0000 7f2aeffff700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2ad406ace0 0x7f2ad406d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2af015a980 tx=0x7f2adc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:59.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.610+0000 7f2aed7fa700 1 -- 172.21.15.49:0/1730224147 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2ae4055c90 con 0x7f2af00b4530 2023-07-18T03:39:59.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:39:59.870+0000 7f2af6dff700 1 -- 172.21.15.49:0/1730224147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2af0074350 con 0x7f2af00b4530 2023-07-18T03:40:00.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.359+0000 7f2aed7fa700 1 -- 172.21.15.49:0/1730224147 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2ae40592b0 con 0x7f2af00b4530 2023-07-18T03:40:00.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.362+0000 7f2af6dff700 1 -- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2ad406ace0 msgr2=0x7f2ad406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:00.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.362+0000 7f2af6dff700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2ad406ace0 0x7f2ad406d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2af015a980 tx=0x7f2adc008040 comp rx=0 tx=0).stop 2023-07-18T03:40:00.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.362+0000 7f2af6dff700 1 -- 172.21.15.49:0/1730224147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af00b4530 msgr2=0x7f2af0152560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:00.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.362+0000 7f2af6dff700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af00b4530 0x7f2af0152560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4000c00 tx=0x7f2ae4004750 comp rx=0 tx=0).stop 2023-07-18T03:40:00.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.362+0000 7f2af6dff700 1 -- 172.21.15.49:0/1730224147 shutdown_connections 2023-07-18T03:40:00.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.362+0000 7f2af6dff700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af00d1050 0x7f2af015e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.362+0000 7f2af6dff700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2ad406ace0 0x7f2ad406d190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.362+0000 7f2af6dff700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af00b4530 0x7f2af0152560 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.362+0000 7f2af6dff700 1 --2- 172.21.15.49:0/1730224147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af00b8a60 0x7f2af00cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.362+0000 7f2af6dff700 1 -- 172.21.15.49:0/1730224147 >> 172.21.15.49:0/1730224147 conn(0x7f2af009c1b0 msgr2=0x7f2af00b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:00.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.362+0000 7f2af6dff700 1 -- 172.21.15.49:0/1730224147 shutdown_connections 2023-07-18T03:40:00.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.362+0000 7f2af6dff700 1 -- 172.21.15.49:0/1730224147 wait complete. 2023-07-18T03:40:00.378 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-18T03:40:00.378 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:40:00.531 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:40:00.536 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:40:00.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.594+0000 7f9eec1ff700 1 Processor -- start 2023-07-18T03:40:00.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.595+0000 7f9eec1ff700 1 -- start start 2023-07-18T03:40:00.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.596+0000 7f9eec1ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9ee40cf050 0x7f9ee406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:00.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.596+0000 7f9eec1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9ee40b3fb0 0x7f9ee406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:00.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.596+0000 7f9eec1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee4160640 con 0x7f9ee415c630 2023-07-18T03:40:00.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.596+0000 7f9eec1ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee4160920 con 0x7f9ee40cf050 2023-07-18T03:40:00.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.596+0000 7f9eec1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee4160c00 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.596+0000 7f9ee979a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9ee40cf050 0x7f9ee406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:00.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.596+0000 7f9eea79c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9ee40b3fb0 0x7f9ee406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:00.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.597+0000 7f9ee979a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9ee40cf050 0x7f9ee406bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38326/0 (socket says 172.21.15.49:38326) 2023-07-18T03:40:00.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.597+0000 7f9ee979a700 1 -- 172.21.15.49:0/1251772469 learned_addr learned my addr 172.21.15.49:0/1251772469 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:00.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.597+0000 7f9ee9f9b700 1 --1- 172.21.15.49:0/1251772469 >> v1:172.21.15.107:6789/0 conn(0x7f9ee415c630 0x7f9ee40ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:51440/0 (socket says 172.21.15.49:51440) 2023-07-18T03:40:00.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.597+0000 7f9eea79c700 1 -- 172.21.15.49:0/1251772469 >> v2:172.21.15.49:3300/0 conn(0x7f9ee40cf050 msgr2=0x7f9ee406bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:00.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.597+0000 7f9eea79c700 1 --2- 172.21.15.49:0/1251772469 >> v2:172.21.15.49:3300/0 conn(0x7f9ee40cf050 0x7f9ee406bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.597+0000 7f9eea79c700 1 -- 172.21.15.49:0/1251772469 >> v1:172.21.15.107:6789/0 conn(0x7f9ee415c630 legacy=0x7f9ee40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.597+0000 7f9eea79c700 1 -- 172.21.15.49:0/1251772469 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee406b9c0 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.597+0000 7f9ee979a700 1 --2- 172.21.15.49:0/1251772469 >> v2:172.21.15.49:3300/0 conn(0x7f9ee40cf050 0x7f9ee406bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.597+0000 7f9ee8f99700 1 -- 172.21.15.49:0/1251772469 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754036998 0 0) 0x7f9ee4160640 con 0x7f9ee415c630 2023-07-18T03:40:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.597+0000 7f9eea79c700 1 --2- 172.21.15.49:0/1251772469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee40b3fb0 0x7f9ee406c420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ee406dce0 tx=0x7f9edc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76396fb890c77658 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.598+0000 7f9ee8f99700 1 -- 172.21.15.49:0/1251772469 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00e070 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.598+0000 7f9ee8f99700 1 -- 172.21.15.49:0/1251772469 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc00bb20 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.598+0000 7f9ee8f99700 1 -- 172.21.15.49:0/1251772469 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc012430 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.598+0000 7f9eec1ff700 1 -- 172.21.15.49:0/1251772469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee40b3fb0 msgr2=0x7f9ee406c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:00.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.598+0000 7f9eec1ff700 1 --2- 172.21.15.49:0/1251772469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee40b3fb0 0x7f9ee406c420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ee406dce0 tx=0x7f9edc00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:00.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.598+0000 7f9eec1ff700 1 -- 172.21.15.49:0/1251772469 shutdown_connections 2023-07-18T03:40:00.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.598+0000 7f9eec1ff700 1 --2- 172.21.15.49:0/1251772469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee40b3fb0 0x7f9ee406c420 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.598+0000 7f9eec1ff700 1 --2- 172.21.15.49:0/1251772469 >> v2:172.21.15.49:3300/0 conn(0x7f9ee40cf050 0x7f9ee406bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.598+0000 7f9eec1ff700 1 -- 172.21.15.49:0/1251772469 >> 172.21.15.49:0/1251772469 conn(0x7f9ee4069f30 msgr2=0x7f9ee406a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:00.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.599+0000 7f9eec1ff700 1 -- 172.21.15.49:0/1251772469 shutdown_connections 2023-07-18T03:40:00.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.599+0000 7f9eec1ff700 1 -- 172.21.15.49:0/1251772469 wait complete. 2023-07-18T03:40:00.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.600+0000 7f9eec1ff700 1 Processor -- start 2023-07-18T03:40:00.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.601+0000 7f9eec1ff700 1 -- start start 2023-07-18T03:40:00.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.602+0000 7f9eec1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee40b3fb0 0x7f9ee415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:00.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.602+0000 7f9eec1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ee40cf050 0x7f9ee414b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:00.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.602+0000 7f9eec1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee415c630 0x7f9ee414d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:00.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.602+0000 7f9eec1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee40b0480 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.602+0000 7f9eec1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee414bd90 con 0x7f9ee415c630 2023-07-18T03:40:00.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.602+0000 7f9eec1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee40b01a0 con 0x7f9ee40cf050 2023-07-18T03:40:00.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.602+0000 7f9ee9f9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee40b3fb0 0x7f9ee415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:00.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.602+0000 7f9ee9f9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee40b3fb0 0x7f9ee415b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38338/0 (socket says 172.21.15.49:38338) 2023-07-18T03:40:00.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.602+0000 7f9ee9f9b700 1 -- 172.21.15.49:0/2619898516 learned_addr learned my addr 172.21.15.49:0/2619898516 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:00.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9eea79c700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee415c630 0x7f9ee414d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:00.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9ee979a700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ee40cf050 0x7f9ee414b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:00.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9ee9f9b700 1 -- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ee40cf050 msgr2=0x7f9ee414b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:00.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9ee9f9b700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ee40cf050 0x7f9ee414b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9ee9f9b700 1 -- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee415c630 msgr2=0x7f9ee414d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:00.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9ee9f9b700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee415c630 0x7f9ee414d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9ee9f9b700 1 -- 172.21.15.49:0/2619898516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed4009550 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9eea79c700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee415c630 0x7f9ee414d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:00.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9ee9f9b700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee40b3fb0 0x7f9ee415b880 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00011e0 tx=0x7f9ee000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:00.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9eda7fc700 1 -- 172.21.15.49:0/2619898516 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0004c80 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9eda7fc700 1 -- 172.21.15.49:0/2619898516 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee0004de0 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9eec1ff700 1 -- 172.21.15.49:0/2619898516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9edc009a70 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.603+0000 7f9eda7fc700 1 -- 172.21.15.49:0/2619898516 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0007310 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.604+0000 7f9eec1ff700 1 -- 172.21.15.49:0/2619898516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee415a0a0 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.605+0000 7f9eec1ff700 1 -- 172.21.15.49:0/2619898516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee404e280 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.606+0000 7f9eda7fc700 1 -- 172.21.15.49:0/2619898516 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ee0007470 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.608+0000 7f9eda7fc700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9ec806ace0 0x7f9ec806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:00.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.608+0000 7f9eda7fc700 1 -- 172.21.15.49:0/2619898516 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ee0089150 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.608+0000 7f9ee979a700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9ec806ace0 0x7f9ec806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:00.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.609+0000 7f9ee979a700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9ec806ace0 0x7f9ec806d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40b05e0 tx=0x7f9ed40092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:00.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.611+0000 7f9eda7fc700 1 -- 172.21.15.49:0/2619898516 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9ee0054200 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.866+0000 7f9eec1ff700 1 -- 172.21.15.49:0/2619898516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ee40ad960 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.867+0000 7f9eda7fc700 1 -- 172.21.15.49:0/2619898516 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f9ee0057820 con 0x7f9ee40b3fb0 2023-07-18T03:40:00.862 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:00.862 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1095518881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1095518881},{"type":"v1","addr":"172.21.15.49:6834","nonce":1095518881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/843626338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":843626338},{"type":"v1","addr":"172.21.15.107:6835","nonce":843626338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3238680863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3238680863},{"type":"v1","addr":"172.21.15.107:6837","nonce":3238680863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:39:56.844870+0000","modified":"2023-07-18T03:40:00.356383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5172":{"gid":5172,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6835/3161575771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":3161575771},{"type":"v1","addr":"172.21.15.49:6835","nonce":3161575771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-18T03:40:00.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.869+0000 7f9eec1ff700 1 -- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9ec806ace0 msgr2=0x7f9ec806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:00.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.869+0000 7f9eec1ff700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9ec806ace0 0x7f9ec806d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40b05e0 tx=0x7f9ed40092c0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.869+0000 7f9eec1ff700 1 -- 172.21.15.49:0/2619898516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee40b3fb0 msgr2=0x7f9ee415b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:00.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.869+0000 7f9eec1ff700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee40b3fb0 0x7f9ee415b880 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00011e0 tx=0x7f9ee000ca30 comp rx=0 tx=0).stop 2023-07-18T03:40:00.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.870+0000 7f9eec1ff700 1 -- 172.21.15.49:0/2619898516 shutdown_connections 2023-07-18T03:40:00.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.870+0000 7f9eec1ff700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee415c630 0x7f9ee414d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.870+0000 7f9eec1ff700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9ec806ace0 0x7f9ec806d190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.870+0000 7f9eec1ff700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ee40b3fb0 0x7f9ee415b880 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.870+0000 7f9eec1ff700 1 --2- 172.21.15.49:0/2619898516 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ee40cf050 0x7f9ee414b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:00.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.870+0000 7f9eec1ff700 1 -- 172.21.15.49:0/2619898516 >> 172.21.15.49:0/2619898516 conn(0x7f9ee4069f30 msgr2=0x7f9ee40b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:00.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.870+0000 7f9eec1ff700 1 -- 172.21.15.49:0/2619898516 shutdown_connections 2023-07-18T03:40:00.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:00.870+0000 7f9eec1ff700 1 -- 172.21.15.49:0/2619898516 wait complete. 2023-07-18T03:40:00.866 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 18 2023-07-18T03:40:00.882 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.093+0000 7f7b125ff700 1 Processor -- start 2023-07-18T03:40:01.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.094+0000 7f7b125ff700 1 -- start start 2023-07-18T03:40:01.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.095+0000 7f7b125ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0c15e890 0x7f7b0c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.095+0000 7f7b125ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7b0c0b3780 0x7f7b0c0b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.095+0000 7f7b125ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c0ce110 con 0x7f7b0c15a8c0 2023-07-18T03:40:01.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.095+0000 7f7b125ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0c15b450 con 0x7f7b0c0b3780 2023-07-18T03:40:01.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.095+0000 7f7b125ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c0cf380 con 0x7f7b0c15e890 2023-07-18T03:40:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.095+0000 7f7b0b7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7b0c0b3780 0x7f7b0c0b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.095+0000 7f7b10b9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7b0c15a8c0 0x7f7b0c15ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56816/0 (socket says 172.21.15.49:56816) 2023-07-18T03:40:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.095+0000 7f7b10b9c700 1 -- 172.21.15.49:0/947884725 learned_addr learned my addr 172.21.15.49:0/947884725 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:01.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.096+0000 7f7b0bfff700 1 --2- 172.21.15.49:0/947884725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0c15e890 0x7f7b0c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.096+0000 7f7b0b7fe700 1 -- 172.21.15.49:0/947884725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0c15e890 msgr2=0x7f7b0c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:01.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.096+0000 7f7b0b7fe700 1 --2- 172.21.15.49:0/947884725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0c15e890 0x7f7b0c0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.096+0000 7f7b0b7fe700 1 -- 172.21.15.49:0/947884725 >> v1:172.21.15.49:6789/0 conn(0x7f7b0c15a8c0 legacy=0x7f7b0c15ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.096+0000 7f7b0b7fe700 1 -- 172.21.15.49:0/947884725 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c15b660 con 0x7f7b0c0b3780 2023-07-18T03:40:01.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.096+0000 7f7b0affd700 1 -- 172.21.15.49:0/947884725 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492103318 0 0) 0x7f7b0c0ce110 con 0x7f7b0c15a8c0 2023-07-18T03:40:01.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.096+0000 7f7b0b7fe700 1 --2- 172.21.15.49:0/947884725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b0c0b3780 0x7f7b0c0b3bf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009840 tx=0x7f7b00009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73784838edbbca2a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.096+0000 7f7b0affd700 1 -- 172.21.15.49:0/947884725 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00011830 con 0x7f7b0c0b3780 2023-07-18T03:40:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.096+0000 7f7b0affd700 1 -- 172.21.15.49:0/947884725 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b00011990 con 0x7f7b0c0b3780 2023-07-18T03:40:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.096+0000 7f7b0affd700 1 -- 172.21.15.49:0/947884725 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00011d20 con 0x7f7b0c0b3780 2023-07-18T03:40:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.097+0000 7f7b125ff700 1 -- 172.21.15.49:0/947884725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b0c0b3780 msgr2=0x7f7b0c0b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.097+0000 7f7b125ff700 1 --2- 172.21.15.49:0/947884725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b0c0b3780 0x7f7b0c0b3bf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009840 tx=0x7f7b00009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.097+0000 7f7b125ff700 1 -- 172.21.15.49:0/947884725 shutdown_connections 2023-07-18T03:40:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.097+0000 7f7b125ff700 1 --2- 172.21.15.49:0/947884725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0c15e890 0x7f7b0c0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.097+0000 7f7b125ff700 1 --2- 172.21.15.49:0/947884725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b0c0b3780 0x7f7b0c0b3bf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.097+0000 7f7b125ff700 1 -- 172.21.15.49:0/947884725 >> 172.21.15.49:0/947884725 conn(0x7f7b0c069f30 msgr2=0x7f7b0c06a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:01.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.097+0000 7f7b125ff700 1 -- 172.21.15.49:0/947884725 shutdown_connections 2023-07-18T03:40:01.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.097+0000 7f7b125ff700 1 -- 172.21.15.49:0/947884725 wait complete. 2023-07-18T03:40:01.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.099+0000 7f7b125ff700 1 Processor -- start 2023-07-18T03:40:01.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.099+0000 7f7b125ff700 1 -- start start 2023-07-18T03:40:01.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.100+0000 7f7b125ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b0c0b3780 0x7f7b0c1414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.100+0000 7f7b125ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0c15a8c0 0x7f7b0c0711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.100+0000 7f7b125ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b0c15e890 0x7f7b0c159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.100+0000 7f7b125ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c141a00 con 0x7f7b0c0b3780 2023-07-18T03:40:01.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.100+0000 7f7b125ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c14de10 con 0x7f7b0c15e890 2023-07-18T03:40:01.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.100+0000 7f7b125ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c1509a0 con 0x7f7b0c15a8c0 2023-07-18T03:40:01.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.100+0000 7f7b0bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b0c0b3780 0x7f7b0c1414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.100+0000 7f7b0bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b0c0b3780 0x7f7b0c1414c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38366/0 (socket says 172.21.15.49:38366) 2023-07-18T03:40:01.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.100+0000 7f7b0bfff700 1 -- 172.21.15.49:0/1606614589 learned_addr learned my addr 172.21.15.49:0/1606614589 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:01.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b10b9c700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b0c15e890 0x7f7b0c159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b0b7fe700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0c15a8c0 0x7f7b0c0711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b0bfff700 1 -- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0c15a8c0 msgr2=0x7f7b0c0711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b0bfff700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0c15a8c0 0x7f7b0c0711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b0bfff700 1 -- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b0c15e890 msgr2=0x7f7b0c159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b0bfff700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b0c15e890 0x7f7b0c159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b0bfff700 1 -- 172.21.15.49:0/1606614589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b000094a0 con 0x7f7b0c0b3780 2023-07-18T03:40:01.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b0b7fe700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0c15a8c0 0x7f7b0c0711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:01.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b10b9c700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b0c15e890 0x7f7b0c159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:01.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b0bfff700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b0c0b3780 0x7f7b0c1414c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00e810 tx=0x7f7afc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b08ff9700 1 -- 172.21.15.49:0/1606614589 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc013610 con 0x7f7b0c0b3780 2023-07-18T03:40:01.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b08ff9700 1 -- 172.21.15.49:0/1606614589 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc013770 con 0x7f7b0c0b3780 2023-07-18T03:40:01.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b08ff9700 1 -- 172.21.15.49:0/1606614589 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc013b20 con 0x7f7b0c0b3780 2023-07-18T03:40:01.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b125ff700 1 -- 172.21.15.49:0/1606614589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c14db30 con 0x7f7b0c0b3780 2023-07-18T03:40:01.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.101+0000 7f7b125ff700 1 -- 172.21.15.49:0/1606614589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0c1501a0 con 0x7f7b0c0b3780 2023-07-18T03:40:01.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.103+0000 7f7b125ff700 1 -- 172.21.15.49:0/1606614589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c04e280 con 0x7f7b0c0b3780 2023-07-18T03:40:01.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.104+0000 7f7b08ff9700 1 -- 172.21.15.49:0/1606614589 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7afc013c80 con 0x7f7b0c0b3780 2023-07-18T03:40:01.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.104+0000 7f7b08ff9700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7af806ace0 0x7f7af806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.105+0000 7f7b08ff9700 1 -- 172.21.15.49:0/1606614589 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7afc0891e0 con 0x7f7b0c0b3780 2023-07-18T03:40:01.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.105+0000 7f7b0b7fe700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7af806ace0 0x7f7af806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.106+0000 7f7b0b7fe700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7af806ace0 0x7f7af806d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c070f30 tx=0x7f7b00008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.109+0000 7f7b08ff9700 1 -- 172.21.15.49:0/1606614589 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7afc054210 con 0x7f7b0c0b3780 2023-07-18T03:40:01.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.318+0000 7f7b125ff700 1 -- 172.21.15.49:0/1606614589 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b0c0ce910 con 0x7f7b0c0b3780 2023-07-18T03:40:01.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.319+0000 7f7b08ff9700 1 -- 172.21.15.49:0/1606614589 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18257 (secure 0 0 0) 0x7f7afc057830 con 0x7f7b0c0b3780 2023-07-18T03:40:01.314 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:01.314 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:58.343119+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:39:54.412334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-18T03:39:55.841218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.321+0000 7f7b125ff700 1 -- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7af806ace0 msgr2=0x7f7af806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.321+0000 7f7b125ff700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7af806ace0 0x7f7af806d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c070f30 tx=0x7f7b00008030 comp rx=0 tx=0).stop 2023-07-18T03:40:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.321+0000 7f7b125ff700 1 -- 172.21.15.49:0/1606614589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b0c0b3780 msgr2=0x7f7b0c1414c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.321+0000 7f7b125ff700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b0c0b3780 0x7f7b0c1414c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00e810 tx=0x7f7afc00ebd0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.322+0000 7f7b125ff700 1 -- 172.21.15.49:0/1606614589 shutdown_connections 2023-07-18T03:40:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.322+0000 7f7b125ff700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b0c15e890 0x7f7b0c159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.322+0000 7f7b125ff700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7af806ace0 0x7f7af806d190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.322+0000 7f7b125ff700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b0c0b3780 0x7f7b0c1414c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.322+0000 7f7b125ff700 1 --2- 172.21.15.49:0/1606614589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0c15a8c0 0x7f7b0c0711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.322+0000 7f7b125ff700 1 -- 172.21.15.49:0/1606614589 >> 172.21.15.49:0/1606614589 conn(0x7f7b0c069f30 msgr2=0x7f7b0c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.322+0000 7f7b125ff700 1 -- 172.21.15.49:0/1606614589 shutdown_connections 2023-07-18T03:40:01.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.322+0000 7f7b125ff700 1 -- 172.21.15.49:0/1606614589 wait complete. 2023-07-18T03:40:01.334 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-18T03:40:01.335 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:40:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.546+0000 7fc6c99ff700 1 Processor -- start 2023-07-18T03:40:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.546+0000 7fc6c99ff700 1 -- start start 2023-07-18T03:40:01.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.547+0000 7fc6c99ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc6c415c950 0x7fc6c40b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.548+0000 7fc6c99ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c40d1220 0x7fc6c40afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.548+0000 7fc6c99ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c40d2ce0 con 0x7fc6c40b1e00 2023-07-18T03:40:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.548+0000 7fc6c99ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c40b3350 con 0x7fc6c415c950 2023-07-18T03:40:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.548+0000 7fc6c99ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40b3630 con 0x7fc6c40d1220 2023-07-18T03:40:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.548+0000 7fc6c2ffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc6c415c950 0x7fc6c40b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.548+0000 7fc6c2ffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc6c415c950 0x7fc6c40b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38382/0 (socket says 172.21.15.49:38382) 2023-07-18T03:40:01.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.548+0000 7fc6c2ffd700 1 -- 172.21.15.49:0/3720125117 learned_addr learned my addr 172.21.15.49:0/3720125117 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.548+0000 7fc6c2ffd700 1 -- 172.21.15.49:0/3720125117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c40d1220 msgr2=0x7fc6c40afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.548+0000 7fc6c2ffd700 1 --2- 172.21.15.49:0/3720125117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c40d1220 0x7fc6c40afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.548+0000 7fc6c2ffd700 1 -- 172.21.15.49:0/3720125117 >> v1:172.21.15.107:6789/0 conn(0x7fc6c40b1e00 legacy=0x7fc6c40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.548+0000 7fc6c2ffd700 1 -- 172.21.15.49:0/3720125117 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c40b3910 con 0x7fc6c415c950 2023-07-18T03:40:01.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.549+0000 7fc6c2ffd700 1 --2- 172.21.15.49:0/3720125117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c415c950 0x7fc6c40b43f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4006fd0 tx=0x7fc6b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4dcc17e1ed61eb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.549+0000 7fc6c27fc700 1 -- 172.21.15.49:0/3720125117 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4005e30 con 0x7fc6c415c950 2023-07-18T03:40:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.549+0000 7fc6c27fc700 1 -- 172.21.15.49:0/3720125117 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b4006030 con 0x7fc6c415c950 2023-07-18T03:40:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.549+0000 7fc6c27fc700 1 -- 172.21.15.49:0/3720125117 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40063c0 con 0x7fc6c415c950 2023-07-18T03:40:01.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.549+0000 7fc6c99ff700 1 -- 172.21.15.49:0/3720125117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c415c950 msgr2=0x7fc6c40b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.549+0000 7fc6c99ff700 1 --2- 172.21.15.49:0/3720125117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c415c950 0x7fc6c40b43f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4006fd0 tx=0x7fc6b4009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.549+0000 7fc6c99ff700 1 -- 172.21.15.49:0/3720125117 shutdown_connections 2023-07-18T03:40:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.549+0000 7fc6c99ff700 1 --2- 172.21.15.49:0/3720125117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c40d1220 0x7fc6c40afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.549+0000 7fc6c99ff700 1 --2- 172.21.15.49:0/3720125117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c415c950 0x7fc6c40b43f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.549+0000 7fc6c99ff700 1 -- 172.21.15.49:0/3720125117 >> 172.21.15.49:0/3720125117 conn(0x7fc6c409c1d0 msgr2=0x7fc6c40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.550+0000 7fc6c99ff700 1 -- 172.21.15.49:0/3720125117 shutdown_connections 2023-07-18T03:40:01.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.550+0000 7fc6c99ff700 1 -- 172.21.15.49:0/3720125117 wait complete. 2023-07-18T03:40:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.552+0000 7fc6c99ff700 1 Processor -- start 2023-07-18T03:40:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.552+0000 7fc6c99ff700 1 -- start start 2023-07-18T03:40:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.553+0000 7fc6c99ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6c40b1e00 0x7fc6c4157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.553+0000 7fc6c99ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c40d1220 0x7fc6c40cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.553+0000 7fc6c99ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c415c950 0x7fc6c40ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.553+0000 7fc6c99ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40cab60 con 0x7fc6c40d1220 2023-07-18T03:40:01.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.553+0000 7fc6c99ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40cae40 con 0x7fc6c40b1e00 2023-07-18T03:40:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.553+0000 7fc6c99ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40cb120 con 0x7fc6c415c950 2023-07-18T03:40:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.553+0000 7fc6bbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c40d1220 0x7fc6c40cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.553+0000 7fc6bbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c40d1220 0x7fc6c40cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38396/0 (socket says 172.21.15.49:38396) 2023-07-18T03:40:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.553+0000 7fc6bbfff700 1 -- 172.21.15.49:0/180475439 learned_addr learned my addr 172.21.15.49:0/180475439 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.553+0000 7fc6c2ffd700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6c40b1e00 0x7fc6c4157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.553+0000 7fc6c37fe700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c415c950 0x7fc6c40ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.554+0000 7fc6bbfff700 1 -- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c415c950 msgr2=0x7fc6c40ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.554+0000 7fc6bbfff700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c415c950 0x7fc6c40ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.554+0000 7fc6bbfff700 1 -- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6c40b1e00 msgr2=0x7fc6c4157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.554+0000 7fc6bbfff700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6c40b1e00 0x7fc6c4157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.554+0000 7fc6bbfff700 1 -- 172.21.15.49:0/180475439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b4009500 con 0x7fc6c40d1220 2023-07-18T03:40:01.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.554+0000 7fc6bbfff700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c40d1220 0x7fc6c40cbc00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00c4a0 tx=0x7fc6ac00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.554+0000 7fc6c0ff9700 1 -- 172.21.15.49:0/180475439 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0054e0 con 0x7fc6c40d1220 2023-07-18T03:40:01.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.554+0000 7fc6c0ff9700 1 -- 172.21.15.49:0/180475439 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac003680 con 0x7fc6c40d1220 2023-07-18T03:40:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.554+0000 7fc6c99ff700 1 -- 172.21.15.49:0/180475439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c40cb400 con 0x7fc6c40d1220 2023-07-18T03:40:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.554+0000 7fc6c0ff9700 1 -- 172.21.15.49:0/180475439 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac003a30 con 0x7fc6c40d1220 2023-07-18T03:40:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.554+0000 7fc6c99ff700 1 -- 172.21.15.49:0/180475439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c4157fd0 con 0x7fc6c40d1220 2023-07-18T03:40:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.556+0000 7fc6c99ff700 1 -- 172.21.15.49:0/180475439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c404e280 con 0x7fc6c40d1220 2023-07-18T03:40:01.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.557+0000 7fc6c0ff9700 1 -- 172.21.15.49:0/180475439 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6ac003b90 con 0x7fc6c40d1220 2023-07-18T03:40:01.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.557+0000 7fc6c0ff9700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc6a406ace0 0x7fc6a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.557+0000 7fc6c0ff9700 1 -- 172.21.15.49:0/180475439 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6ac088400 con 0x7fc6c40d1220 2023-07-18T03:40:01.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.558+0000 7fc6c2ffd700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc6a406ace0 0x7fc6a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.559+0000 7fc6c2ffd700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc6a406ace0 0x7fc6a406d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4013040 tx=0x7fc6b4008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.562+0000 7fc6c0ff9700 1 -- 172.21.15.49:0/180475439 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc6ac0534b0 con 0x7fc6c40d1220 2023-07-18T03:40:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.764+0000 7fc6c99ff700 1 -- 172.21.15.49:0/180475439 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6c40b3ed0 con 0x7fc6a406ace0 2023-07-18T03:40:01.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.775+0000 7fc6c0ff9700 1 -- 172.21.15.49:0/180475439 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309001 (secure 0 0 0) 0x7fc6b4004140 con 0x7fc6a406ace0 2023-07-18T03:40:01.774 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:01.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.779+0000 7fc6c99ff700 1 -- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc6a406ace0 msgr2=0x7fc6a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.779+0000 7fc6c99ff700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc6a406ace0 0x7fc6a406d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4013040 tx=0x7fc6b4008070 comp rx=0 tx=0).stop 2023-07-18T03:40:01.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.779+0000 7fc6c99ff700 1 -- 172.21.15.49:0/180475439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c40d1220 msgr2=0x7fc6c40cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.779+0000 7fc6c99ff700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c40d1220 0x7fc6c40cbc00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00c4a0 tx=0x7fc6ac00c7b0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.779+0000 7fc6c99ff700 1 -- 172.21.15.49:0/180475439 shutdown_connections 2023-07-18T03:40:01.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.779+0000 7fc6c99ff700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c415c950 0x7fc6c40ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.779+0000 7fc6c99ff700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6c40b1e00 0x7fc6c4157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.779+0000 7fc6c99ff700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc6a406ace0 0x7fc6a406d190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.779+0000 7fc6c99ff700 1 --2- 172.21.15.49:0/180475439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c40d1220 0x7fc6c40cbc00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.779+0000 7fc6c99ff700 1 -- 172.21.15.49:0/180475439 >> 172.21.15.49:0/180475439 conn(0x7fc6c409c1d0 msgr2=0x7fc6c40d35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:01.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.780+0000 7fc6c99ff700 1 -- 172.21.15.49:0/180475439 shutdown_connections 2023-07-18T03:40:01.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:01.780+0000 7fc6c99ff700 1 -- 172.21.15.49:0/180475439 wait complete. 2023-07-18T03:40:01.778 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:40:01.793 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2023-07-18T03:40:00.116947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591020,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":177,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236180,"kb_used_data":4180,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738540,"statfs":{"total":773094113280,"available":772852264960,"internally_reserved":0,"allocated":4280320,"data_stored":2437232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":652,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.798871"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.337498+0000","last_change":"2023-07-18T03:39:56.337498+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.337498+0000","last_undegraded":"2023-07-18T03:39:56.337498+0000","last_fullsized":"2023-07-18T03:39:56.337498+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335988+0000","last_change":"2023-07-18T03:39:56.335988+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335988+0000","last_undegraded":"2023-07-18T03:39:56.335988+0000","last_fullsized":"2023-07-18T03:39:56.335988+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.352684+0000","last_change":"2023-07-18T03:39:57.349470+0000","last_active":"2023-07-18T03:39:57.352684+0000","last_peered":"2023-07-18T03:39:57.352684+0000","last_clean":"2023-07-18T03:39:57.352684+0000","last_became_active":"2023-07-18T03:39:57.349188+0000","last_became_peered":"2023-07-18T03:39:57.349188+0000","last_unstale":"2023-07-18T03:39:57.352684+0000","last_undegraded":"2023-07-18T03:39:57.352684+0000","last_fullsized":"2023-07-18T03:39:57.352684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:30:54.522092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.360068+0000","last_change":"2023-07-18T03:39:57.356421+0000","last_active":"2023-07-18T03:39:57.360068+0000","last_peered":"2023-07-18T03:39:57.360068+0000","last_clean":"2023-07-18T03:39:57.360068+0000","last_became_active":"2023-07-18T03:39:57.356128+0000","last_became_peered":"2023-07-18T03:39:57.356128+0000","last_unstale":"2023-07-18T03:39:57.360068+0000","last_undegraded":"2023-07-18T03:39:57.360068+0000","last_fullsized":"2023-07-18T03:39:57.360068+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:32:15.921500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359782+0000","last_change":"2023-07-18T03:39:57.356364+0000","last_active":"2023-07-18T03:39:57.359782+0000","last_peered":"2023-07-18T03:39:57.359782+0000","last_clean":"2023-07-18T03:39:57.359782+0000","last_became_active":"2023-07-18T03:39:57.356079+0000","last_became_peered":"2023-07-18T03:39:57.356079+0000","last_unstale":"2023-07-18T03:39:57.359782+0000","last_undegraded":"2023-07-18T03:39:57.359782+0000","last_fullsized":"2023-07-18T03:39:57.359782+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:08:41.904219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328919+0000","last_change":"2023-07-18T03:39:56.328919+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328919+0000","last_undegraded":"2023-07-18T03:39:56.328919+0000","last_fullsized":"2023-07-18T03:39:56.328919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332077+0000","last_change":"2023-07-18T03:39:56.332077+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332077+0000","last_undegraded":"2023-07-18T03:39:56.332077+0000","last_fullsized":"2023-07-18T03:39:56.332077+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335775+0000","last_change":"2023-07-18T03:39:56.335775+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335775+0000","last_undegraded":"2023-07-18T03:39:56.335775+0000","last_fullsized":"2023-07-18T03:39:56.335775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.357950+0000","last_change":"2023-07-18T03:39:57.352415+0000","last_active":"2023-07-18T03:39:57.357950+0000","last_peered":"2023-07-18T03:39:57.357950+0000","last_clean":"2023-07-18T03:39:57.357950+0000","last_became_active":"2023-07-18T03:39:57.352137+0000","last_became_peered":"2023-07-18T03:39:57.352137+0000","last_unstale":"2023-07-18T03:39:57.357950+0000","last_undegraded":"2023-07-18T03:39:57.357950+0000","last_fullsized":"2023-07-18T03:39:57.357950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:36.387384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.338378+0000","last_change":"2023-07-18T03:39:56.338378+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.338378+0000","last_undegraded":"2023-07-18T03:39:56.338378+0000","last_fullsized":"2023-07-18T03:39:56.338378+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.333492+0000","last_change":"2023-07-18T03:39:56.333492+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.333492+0000","last_undegraded":"2023-07-18T03:39:56.333492+0000","last_fullsized":"2023-07-18T03:39:56.333492+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.351146+0000","last_change":"2023-07-18T03:39:57.345772+0000","last_active":"2023-07-18T03:39:57.351146+0000","last_peered":"2023-07-18T03:39:57.351146+0000","last_clean":"2023-07-18T03:39:57.351146+0000","last_became_active":"2023-07-18T03:39:57.345465+0000","last_became_peered":"2023-07-18T03:39:57.345465+0000","last_unstale":"2023-07-18T03:39:57.351146+0000","last_undegraded":"2023-07-18T03:39:57.351146+0000","last_fullsized":"2023-07-18T03:39:57.351146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:48:30.748069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332641+0000","last_change":"2023-07-18T03:39:56.332641+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332641+0000","last_undegraded":"2023-07-18T03:39:56.332641+0000","last_fullsized":"2023-07-18T03:39:56.332641+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.327178+0000","last_change":"2023-07-18T03:39:56.327178+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.327178+0000","last_undegraded":"2023-07-18T03:39:56.327178+0000","last_fullsized":"2023-07-18T03:39:56.327178+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332947+0000","last_change":"2023-07-18T03:39:56.332947+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332947+0000","last_undegraded":"2023-07-18T03:39:56.332947+0000","last_fullsized":"2023-07-18T03:39:56.332947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.348720+0000","last_change":"2023-07-18T03:39:57.337636+0000","last_active":"2023-07-18T03:39:57.348720+0000","last_peered":"2023-07-18T03:39:57.348720+0000","last_clean":"2023-07-18T03:39:57.348720+0000","last_became_active":"2023-07-18T03:39:57.337330+0000","last_became_peered":"2023-07-18T03:39:57.337330+0000","last_unstale":"2023-07-18T03:39:57.348720+0000","last_undegraded":"2023-07-18T03:39:57.348720+0000","last_fullsized":"2023-07-18T03:39:57.348720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:22:11.047392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354699+0000","last_change":"2023-07-18T03:39:57.345108+0000","last_active":"2023-07-18T03:39:57.354699+0000","last_peered":"2023-07-18T03:39:57.354699+0000","last_clean":"2023-07-18T03:39:57.354699+0000","last_became_active":"2023-07-18T03:39:57.344825+0000","last_became_peered":"2023-07-18T03:39:57.344825+0000","last_unstale":"2023-07-18T03:39:57.354699+0000","last_undegraded":"2023-07-18T03:39:57.354699+0000","last_fullsized":"2023-07-18T03:39:57.354699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:07:59.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.365129+0000","last_change":"2023-07-18T03:39:57.361051+0000","last_active":"2023-07-18T03:39:57.365129+0000","last_peered":"2023-07-18T03:39:57.365129+0000","last_clean":"2023-07-18T03:39:57.365129+0000","last_became_active":"2023-07-18T03:39:57.360746+0000","last_became_peered":"2023-07-18T03:39:57.360746+0000","last_unstale":"2023-07-18T03:39:57.365129+0000","last_undegraded":"2023-07-18T03:39:57.365129+0000","last_fullsized":"2023-07-18T03:39:57.365129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:11:30.468969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359427+0000","last_change":"2023-07-18T03:39:57.351301+0000","last_active":"2023-07-18T03:39:57.359427+0000","last_peered":"2023-07-18T03:39:57.359427+0000","last_clean":"2023-07-18T03:39:57.359427+0000","last_became_active":"2023-07-18T03:39:57.351033+0000","last_became_peered":"2023-07-18T03:39:57.351033+0000","last_unstale":"2023-07-18T03:39:57.359427+0000","last_undegraded":"2023-07-18T03:39:57.359427+0000","last_fullsized":"2023-07-18T03:39:57.359427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:50:08.481909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.365031+0000","last_change":"2023-07-18T03:39:57.360471+0000","last_active":"2023-07-18T03:39:57.365031+0000","last_peered":"2023-07-18T03:39:57.365031+0000","last_clean":"2023-07-18T03:39:57.365031+0000","last_became_active":"2023-07-18T03:39:57.360202+0000","last_became_peered":"2023-07-18T03:39:57.360202+0000","last_unstale":"2023-07-18T03:39:57.365031+0000","last_undegraded":"2023-07-18T03:39:57.365031+0000","last_fullsized":"2023-07-18T03:39:57.365031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:32.300586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.331681+0000","last_change":"2023-07-18T03:39:56.331681+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.331681+0000","last_undegraded":"2023-07-18T03:39:56.331681+0000","last_fullsized":"2023-07-18T03:39:56.331681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.329011+0000","last_change":"2023-07-18T03:39:56.329011+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.329011+0000","last_undegraded":"2023-07-18T03:39:56.329011+0000","last_fullsized":"2023-07-18T03:39:56.329011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354754+0000","last_change":"2023-07-18T03:39:57.347725+0000","last_active":"2023-07-18T03:39:57.354754+0000","last_peered":"2023-07-18T03:39:57.354754+0000","last_clean":"2023-07-18T03:39:57.354754+0000","last_became_active":"2023-07-18T03:39:57.346990+0000","last_became_peered":"2023-07-18T03:39:57.346990+0000","last_unstale":"2023-07-18T03:39:57.354754+0000","last_undegraded":"2023-07-18T03:39:57.354754+0000","last_fullsized":"2023-07-18T03:39:57.354754+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:33:51.412210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.361028+0000","last_change":"2023-07-18T03:39:57.355821+0000","last_active":"2023-07-18T03:39:57.361028+0000","last_peered":"2023-07-18T03:39:57.361028+0000","last_clean":"2023-07-18T03:39:57.361028+0000","last_became_active":"2023-07-18T03:39:57.355505+0000","last_became_peered":"2023-07-18T03:39:57.355505+0000","last_unstale":"2023-07-18T03:39:57.361028+0000","last_undegraded":"2023-07-18T03:39:57.361028+0000","last_fullsized":"2023-07-18T03:39:57.361028+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:11:11.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.358089+0000","last_change":"2023-07-18T03:39:57.347320+0000","last_active":"2023-07-18T03:39:57.358089+0000","last_peered":"2023-07-18T03:39:57.358089+0000","last_clean":"2023-07-18T03:39:57.358089+0000","last_became_active":"2023-07-18T03:39:57.347051+0000","last_became_peered":"2023-07-18T03:39:57.347051+0000","last_unstale":"2023-07-18T03:39:57.358089+0000","last_undegraded":"2023-07-18T03:39:57.358089+0000","last_fullsized":"2023-07-18T03:39:57.358089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:45:31.179450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328947+0000","last_change":"2023-07-18T03:39:56.328947+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328947+0000","last_undegraded":"2023-07-18T03:39:56.328947+0000","last_fullsized":"2023-07-18T03:39:56.328947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.362856+0000","last_change":"2023-07-18T03:39:57.349732+0000","last_active":"2023-07-18T03:39:57.362856+0000","last_peered":"2023-07-18T03:39:57.362856+0000","last_clean":"2023-07-18T03:39:57.362856+0000","last_became_active":"2023-07-18T03:39:57.349421+0000","last_became_peered":"2023-07-18T03:39:57.349421+0000","last_unstale":"2023-07-18T03:39:57.362856+0000","last_undegraded":"2023-07-18T03:39:57.362856+0000","last_fullsized":"2023-07-18T03:39:57.362856+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:03:41.731722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.337384+0000","last_change":"2023-07-18T03:39:56.337384+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.337384+0000","last_undegraded":"2023-07-18T03:39:56.337384+0000","last_fullsized":"2023-07-18T03:39:56.337384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.362756+0000","last_change":"2023-07-18T03:39:57.354325+0000","last_active":"2023-07-18T03:39:57.362756+0000","last_peered":"2023-07-18T03:39:57.362756+0000","last_clean":"2023-07-18T03:39:57.362756+0000","last_became_active":"2023-07-18T03:39:57.353910+0000","last_became_peered":"2023-07-18T03:39:57.353910+0000","last_unstale":"2023-07-18T03:39:57.362756+0000","last_undegraded":"2023-07-18T03:39:57.362756+0000","last_fullsized":"2023-07-18T03:39:57.362756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:18:46.390308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.364995+0000","last_change":"2023-07-18T03:39:57.362161+0000","last_active":"2023-07-18T03:39:57.364995+0000","last_peered":"2023-07-18T03:39:57.364995+0000","last_clean":"2023-07-18T03:39:57.364995+0000","last_became_active":"2023-07-18T03:39:57.361882+0000","last_became_peered":"2023-07-18T03:39:57.361882+0000","last_unstale":"2023-07-18T03:39:57.364995+0000","last_undegraded":"2023-07-18T03:39:57.364995+0000","last_fullsized":"2023-07-18T03:39:57.364995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:39:10.257984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354777+0000","last_change":"2023-07-18T03:39:57.347923+0000","last_active":"2023-07-18T03:39:57.354777+0000","last_peered":"2023-07-18T03:39:57.354777+0000","last_clean":"2023-07-18T03:39:57.354777+0000","last_became_active":"2023-07-18T03:39:57.347419+0000","last_became_peered":"2023-07-18T03:39:57.347419+0000","last_unstale":"2023-07-18T03:39:57.354777+0000","last_undegraded":"2023-07-18T03:39:57.354777+0000","last_fullsized":"2023-07-18T03:39:57.354777+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:54:59.548413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328673+0000","last_change":"2023-07-18T03:39:56.328673+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328673+0000","last_undegraded":"2023-07-18T03:39:56.328673+0000","last_fullsized":"2023-07-18T03:39:56.328673+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.348817+0000","last_change":"2023-07-18T03:39:57.338235+0000","last_active":"2023-07-18T03:39:57.348817+0000","last_peered":"2023-07-18T03:39:57.348817+0000","last_clean":"2023-07-18T03:39:57.348817+0000","last_became_active":"2023-07-18T03:39:57.337940+0000","last_became_peered":"2023-07-18T03:39:57.337940+0000","last_unstale":"2023-07-18T03:39:57.348817+0000","last_undegraded":"2023-07-18T03:39:57.348817+0000","last_fullsized":"2023-07-18T03:39:57.348817+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:42:25.160054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.365698+0000","last_change":"2023-07-18T03:39:56.337657+0000","last_active":"2023-07-18T03:39:57.365698+0000","last_peered":"2023-07-18T03:39:57.365698+0000","last_clean":"2023-07-18T03:39:57.365698+0000","last_became_active":"2023-07-18T03:39:56.337379+0000","last_became_peered":"2023-07-18T03:39:56.337379+0000","last_unstale":"2023-07-18T03:39:57.365698+0000","last_undegraded":"2023-07-18T03:39:57.365698+0000","last_fullsized":"2023-07-18T03:39:57.365698+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:30.478922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.330756+0000","last_change":"2023-07-18T03:39:56.330756+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.330756+0000","last_undegraded":"2023-07-18T03:39:56.330756+0000","last_fullsized":"2023-07-18T03:39:56.330756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355305+0000","last_change":"2023-07-18T03:39:56.345284+0000","last_active":"2023-07-18T03:39:57.355305+0000","last_peered":"2023-07-18T03:39:57.355305+0000","last_clean":"2023-07-18T03:39:57.355305+0000","last_became_active":"2023-07-18T03:39:56.344955+0000","last_became_peered":"2023-07-18T03:39:56.344955+0000","last_unstale":"2023-07-18T03:39:57.355305+0000","last_undegraded":"2023-07-18T03:39:57.355305+0000","last_fullsized":"2023-07-18T03:39:57.355305+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:50:11.765103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328243+0000","last_change":"2023-07-18T03:39:56.328243+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328243+0000","last_undegraded":"2023-07-18T03:39:56.328243+0000","last_fullsized":"2023-07-18T03:39:56.328243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.345414+0000","last_change":"2023-07-18T03:39:56.335839+0000","last_active":"2023-07-18T03:39:56.345414+0000","last_peered":"2023-07-18T03:39:56.345414+0000","last_clean":"2023-07-18T03:39:56.345414+0000","last_became_active":"2023-07-18T03:39:56.335506+0000","last_became_peered":"2023-07-18T03:39:56.335506+0000","last_unstale":"2023-07-18T03:39:56.345414+0000","last_undegraded":"2023-07-18T03:39:56.345414+0000","last_fullsized":"2023-07-18T03:39:56.345414+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:55:30.319672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.338278+0000","last_change":"2023-07-18T03:39:56.338278+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.338278+0000","last_undegraded":"2023-07-18T03:39:56.338278+0000","last_fullsized":"2023-07-18T03:39:56.338278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.355959+0000","last_change":"2023-07-18T03:39:56.353462+0000","last_active":"2023-07-18T03:39:56.355959+0000","last_peered":"2023-07-18T03:39:56.355959+0000","last_clean":"2023-07-18T03:39:56.355959+0000","last_became_active":"2023-07-18T03:39:56.353165+0000","last_became_peered":"2023-07-18T03:39:56.353165+0000","last_unstale":"2023-07-18T03:39:56.355959+0000","last_undegraded":"2023-07-18T03:39:56.355959+0000","last_fullsized":"2023-07-18T03:39:56.355959+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:24.138081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.350712+0000","last_change":"2023-07-18T03:39:57.345776+0000","last_active":"2023-07-18T03:39:57.350712+0000","last_peered":"2023-07-18T03:39:57.350712+0000","last_clean":"2023-07-18T03:39:57.350712+0000","last_became_active":"2023-07-18T03:39:57.345450+0000","last_became_peered":"2023-07-18T03:39:57.345450+0000","last_unstale":"2023-07-18T03:39:57.350712+0000","last_undegraded":"2023-07-18T03:39:57.350712+0000","last_fullsized":"2023-07-18T03:39:57.350712+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:39:58.863405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.348456+0000","last_change":"2023-07-18T03:39:56.343054+0000","last_active":"2023-07-18T03:39:56.348456+0000","last_peered":"2023-07-18T03:39:56.348456+0000","last_clean":"2023-07-18T03:39:56.348456+0000","last_became_active":"2023-07-18T03:39:56.342752+0000","last_became_peered":"2023-07-18T03:39:56.342752+0000","last_unstale":"2023-07-18T03:39:56.348456+0000","last_undegraded":"2023-07-18T03:39:56.348456+0000","last_fullsized":"2023-07-18T03:39:56.348456+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:42:03.205955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332552+0000","last_change":"2023-07-18T03:39:56.332552+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332552+0000","last_undegraded":"2023-07-18T03:39:56.332552+0000","last_fullsized":"2023-07-18T03:39:56.332552+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.356389+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:57.356389+0000","last_peered":"2023-07-18T03:39:57.356389+0000","last_clean":"2023-07-18T03:39:57.356389+0000","last_became_active":"2023-07-18T03:39:56.345390+0000","last_became_peered":"2023-07-18T03:39:56.345390+0000","last_unstale":"2023-07-18T03:39:57.356389+0000","last_undegraded":"2023-07-18T03:39:57.356389+0000","last_fullsized":"2023-07-18T03:39:57.356389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:57:57.797956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328578+0000","last_change":"2023-07-18T03:39:56.328578+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328578+0000","last_undegraded":"2023-07-18T03:39:56.328578+0000","last_fullsized":"2023-07-18T03:39:56.328578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.347760+0000","last_change":"2023-07-18T03:39:56.346375+0000","last_active":"2023-07-18T03:39:56.347760+0000","last_peered":"2023-07-18T03:39:56.347760+0000","last_clean":"2023-07-18T03:39:56.347760+0000","last_became_active":"2023-07-18T03:39:56.346098+0000","last_became_peered":"2023-07-18T03:39:56.346098+0000","last_unstale":"2023-07-18T03:39:56.347760+0000","last_undegraded":"2023-07-18T03:39:56.347760+0000","last_fullsized":"2023-07-18T03:39:56.347760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:50:22.604564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.339195+0000","last_change":"2023-07-18T03:39:56.339195+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.339195+0000","last_undegraded":"2023-07-18T03:39:56.339195+0000","last_fullsized":"2023-07-18T03:39:56.339195+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.341115+0000","last_change":"2023-07-18T03:39:56.337739+0000","last_active":"2023-07-18T03:39:56.341115+0000","last_peered":"2023-07-18T03:39:56.341115+0000","last_clean":"2023-07-18T03:39:56.341115+0000","last_became_active":"2023-07-18T03:39:56.337495+0000","last_became_peered":"2023-07-18T03:39:56.337495+0000","last_unstale":"2023-07-18T03:39:56.341115+0000","last_undegraded":"2023-07-18T03:39:56.341115+0000","last_fullsized":"2023-07-18T03:39:56.341115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:01.077387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.360048+0000","last_change":"2023-07-18T03:39:57.356339+0000","last_active":"2023-07-18T03:39:57.360048+0000","last_peered":"2023-07-18T03:39:57.360048+0000","last_clean":"2023-07-18T03:39:57.360048+0000","last_became_active":"2023-07-18T03:39:57.356008+0000","last_became_peered":"2023-07-18T03:39:57.356008+0000","last_unstale":"2023-07-18T03:39:57.360048+0000","last_undegraded":"2023-07-18T03:39:57.360048+0000","last_fullsized":"2023-07-18T03:39:57.360048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:20:26.761581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354507+0000","last_change":"2023-07-18T03:39:56.352591+0000","last_active":"2023-07-18T03:39:56.354507+0000","last_peered":"2023-07-18T03:39:56.354507+0000","last_clean":"2023-07-18T03:39:56.354507+0000","last_became_active":"2023-07-18T03:39:56.352257+0000","last_became_peered":"2023-07-18T03:39:56.352257+0000","last_unstale":"2023-07-18T03:39:56.354507+0000","last_undegraded":"2023-07-18T03:39:56.354507+0000","last_fullsized":"2023-07-18T03:39:56.354507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:40:18.228711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.357301+0000","last_change":"2023-07-18T03:39:57.354324+0000","last_active":"2023-07-18T03:39:57.357301+0000","last_peered":"2023-07-18T03:39:57.357301+0000","last_clean":"2023-07-18T03:39:57.357301+0000","last_became_active":"2023-07-18T03:39:57.354028+0000","last_became_peered":"2023-07-18T03:39:57.354028+0000","last_unstale":"2023-07-18T03:39:57.357301+0000","last_undegraded":"2023-07-18T03:39:57.357301+0000","last_fullsized":"2023-07-18T03:39:57.357301+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:48:08.031962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354264+0000","last_change":"2023-07-18T03:39:56.353133+0000","last_active":"2023-07-18T03:39:56.354264+0000","last_peered":"2023-07-18T03:39:56.354264+0000","last_clean":"2023-07-18T03:39:56.354264+0000","last_became_active":"2023-07-18T03:39:56.352847+0000","last_became_peered":"2023-07-18T03:39:56.352847+0000","last_unstale":"2023-07-18T03:39:56.354264+0000","last_undegraded":"2023-07-18T03:39:56.354264+0000","last_fullsized":"2023-07-18T03:39:56.354264+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:21:01.203032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354872+0000","last_change":"2023-07-18T03:39:57.350961+0000","last_active":"2023-07-18T03:39:57.354872+0000","last_peered":"2023-07-18T03:39:57.354872+0000","last_clean":"2023-07-18T03:39:57.354872+0000","last_became_active":"2023-07-18T03:39:57.350643+0000","last_became_peered":"2023-07-18T03:39:57.350643+0000","last_unstale":"2023-07-18T03:39:57.354872+0000","last_undegraded":"2023-07-18T03:39:57.354872+0000","last_fullsized":"2023-07-18T03:39:57.354872+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:10:28.224008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.349402+0000","last_change":"2023-07-18T03:39:56.348133+0000","last_active":"2023-07-18T03:39:56.349402+0000","last_peered":"2023-07-18T03:39:56.349402+0000","last_clean":"2023-07-18T03:39:56.349402+0000","last_became_active":"2023-07-18T03:39:56.347930+0000","last_became_peered":"2023-07-18T03:39:56.347930+0000","last_unstale":"2023-07-18T03:39:56.349402+0000","last_undegraded":"2023-07-18T03:39:56.349402+0000","last_fullsized":"2023-07-18T03:39:56.349402+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:25:49.425965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.327915+0000","last_change":"2023-07-18T03:39:56.327915+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.327915+0000","last_undegraded":"2023-07-18T03:39:56.327915+0000","last_fullsized":"2023-07-18T03:39:56.327915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354562+0000","last_change":"2023-07-18T03:39:56.352639+0000","last_active":"2023-07-18T03:39:56.354562+0000","last_peered":"2023-07-18T03:39:56.354562+0000","last_clean":"2023-07-18T03:39:56.354562+0000","last_became_active":"2023-07-18T03:39:56.352134+0000","last_became_peered":"2023-07-18T03:39:56.352134+0000","last_unstale":"2023-07-18T03:39:56.354562+0000","last_undegraded":"2023-07-18T03:39:56.354562+0000","last_fullsized":"2023-07-18T03:39:56.354562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:26:35.806559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.352144+0000","last_change":"2023-07-18T03:39:57.349224+0000","last_active":"2023-07-18T03:39:57.352144+0000","last_peered":"2023-07-18T03:39:57.352144+0000","last_clean":"2023-07-18T03:39:57.352144+0000","last_became_active":"2023-07-18T03:39:57.348944+0000","last_became_peered":"2023-07-18T03:39:57.348944+0000","last_unstale":"2023-07-18T03:39:57.352144+0000","last_undegraded":"2023-07-18T03:39:57.352144+0000","last_fullsized":"2023-07-18T03:39:57.352144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:55:01.968006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354442+0000","last_change":"2023-07-18T03:39:56.352532+0000","last_active":"2023-07-18T03:39:56.354442+0000","last_peered":"2023-07-18T03:39:56.354442+0000","last_clean":"2023-07-18T03:39:56.354442+0000","last_became_active":"2023-07-18T03:39:56.352201+0000","last_became_peered":"2023-07-18T03:39:56.352201+0000","last_unstale":"2023-07-18T03:39:56.354442+0000","last_undegraded":"2023-07-18T03:39:56.354442+0000","last_fullsized":"2023-07-18T03:39:56.354442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:57:40.583880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.337060+0000","last_change":"2023-07-18T03:39:56.337060+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.337060+0000","last_undegraded":"2023-07-18T03:39:56.337060+0000","last_fullsized":"2023-07-18T03:39:56.337060+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.347811+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:56.347811+0000","last_peered":"2023-07-18T03:39:56.347811+0000","last_clean":"2023-07-18T03:39:56.347811+0000","last_became_active":"2023-07-18T03:39:56.345735+0000","last_became_peered":"2023-07-18T03:39:56.345735+0000","last_unstale":"2023-07-18T03:39:56.347811+0000","last_undegraded":"2023-07-18T03:39:56.347811+0000","last_fullsized":"2023-07-18T03:39:56.347811+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:49:19.376544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354809+0000","last_change":"2023-07-18T03:39:57.347143+0000","last_active":"2023-07-18T03:39:57.354809+0000","last_peered":"2023-07-18T03:39:57.354809+0000","last_clean":"2023-07-18T03:39:57.354809+0000","last_became_active":"2023-07-18T03:39:57.346833+0000","last_became_peered":"2023-07-18T03:39:57.346833+0000","last_unstale":"2023-07-18T03:39:57.354809+0000","last_undegraded":"2023-07-18T03:39:57.354809+0000","last_fullsized":"2023-07-18T03:39:57.354809+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:26:07.789214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.339763+0000","last_change":"2023-07-18T03:39:56.339148+0000","last_active":"2023-07-18T03:39:57.339763+0000","last_peered":"2023-07-18T03:39:57.339763+0000","last_clean":"2023-07-18T03:39:57.339763+0000","last_became_active":"2023-07-18T03:39:56.338862+0000","last_became_peered":"2023-07-18T03:39:56.338862+0000","last_unstale":"2023-07-18T03:39:57.339763+0000","last_undegraded":"2023-07-18T03:39:57.339763+0000","last_fullsized":"2023-07-18T03:39:57.339763+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:14:22.006899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.340136+0000","last_change":"2023-07-18T03:39:56.340136+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.340136+0000","last_undegraded":"2023-07-18T03:39:56.340136+0000","last_fullsized":"2023-07-18T03:39:56.340136+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.364107+0000","last_change":"2023-07-18T03:39:56.345149+0000","last_active":"2023-07-18T03:39:57.364107+0000","last_peered":"2023-07-18T03:39:57.364107+0000","last_clean":"2023-07-18T03:39:57.364107+0000","last_became_active":"2023-07-18T03:39:56.344848+0000","last_became_peered":"2023-07-18T03:39:56.344848+0000","last_unstale":"2023-07-18T03:39:57.364107+0000","last_undegraded":"2023-07-18T03:39:57.364107+0000","last_fullsized":"2023-07-18T03:39:57.364107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:51:09.697807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335929+0000","last_change":"2023-07-18T03:39:56.335929+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335929+0000","last_undegraded":"2023-07-18T03:39:56.335929+0000","last_fullsized":"2023-07-18T03:39:56.335929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.349330+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:56.349330+0000","last_peered":"2023-07-18T03:39:56.349330+0000","last_clean":"2023-07-18T03:39:56.349330+0000","last_became_active":"2023-07-18T03:39:56.345797+0000","last_became_peered":"2023-07-18T03:39:56.345797+0000","last_unstale":"2023-07-18T03:39:56.349330+0000","last_undegraded":"2023-07-18T03:39:56.349330+0000","last_fullsized":"2023-07-18T03:39:56.349330+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:12.877614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.339214+0000","last_change":"2023-07-18T03:39:56.339214+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.339214+0000","last_undegraded":"2023-07-18T03:39:56.339214+0000","last_fullsized":"2023-07-18T03:39:56.339214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.350939+0000","last_change":"2023-07-18T03:39:56.349196+0000","last_active":"2023-07-18T03:39:56.350939+0000","last_peered":"2023-07-18T03:39:56.350939+0000","last_clean":"2023-07-18T03:39:56.350939+0000","last_became_active":"2023-07-18T03:39:56.347176+0000","last_became_peered":"2023-07-18T03:39:56.347176+0000","last_unstale":"2023-07-18T03:39:56.350939+0000","last_undegraded":"2023-07-18T03:39:56.350939+0000","last_fullsized":"2023-07-18T03:39:56.350939+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:09.700401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.327716+0000","last_change":"2023-07-18T03:39:56.327716+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.327716+0000","last_undegraded":"2023-07-18T03:39:56.327716+0000","last_fullsized":"2023-07-18T03:39:56.327716+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359507+0000","last_change":"2023-07-18T03:39:56.344722+0000","last_active":"2023-07-18T03:39:57.359507+0000","last_peered":"2023-07-18T03:39:57.359507+0000","last_clean":"2023-07-18T03:39:57.359507+0000","last_became_active":"2023-07-18T03:39:56.344439+0000","last_became_peered":"2023-07-18T03:39:56.344439+0000","last_unstale":"2023-07-18T03:39:57.359507+0000","last_undegraded":"2023-07-18T03:39:57.359507+0000","last_fullsized":"2023-07-18T03:39:57.359507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:47:42.656638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.366921+0000","last_change":"2023-07-18T03:39:57.363607+0000","last_active":"2023-07-18T03:39:57.366921+0000","last_peered":"2023-07-18T03:39:57.366921+0000","last_clean":"2023-07-18T03:39:57.366921+0000","last_became_active":"2023-07-18T03:39:57.363312+0000","last_became_peered":"2023-07-18T03:39:57.363312+0000","last_unstale":"2023-07-18T03:39:57.366921+0000","last_undegraded":"2023-07-18T03:39:57.366921+0000","last_fullsized":"2023-07-18T03:39:57.366921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:56:30.355299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.344690+0000","last_change":"2023-07-18T03:39:56.336359+0000","last_active":"2023-07-18T03:39:56.344690+0000","last_peered":"2023-07-18T03:39:56.344690+0000","last_clean":"2023-07-18T03:39:56.344690+0000","last_became_active":"2023-07-18T03:39:56.336094+0000","last_became_peered":"2023-07-18T03:39:56.336094+0000","last_unstale":"2023-07-18T03:39:56.344690+0000","last_undegraded":"2023-07-18T03:39:56.344690+0000","last_fullsized":"2023-07-18T03:39:56.344690+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:30.478922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355019+0000","last_change":"2023-07-18T03:39:57.348592+0000","last_active":"2023-07-18T03:39:57.355019+0000","last_peered":"2023-07-18T03:39:57.355019+0000","last_clean":"2023-07-18T03:39:57.355019+0000","last_became_active":"2023-07-18T03:39:57.348257+0000","last_became_peered":"2023-07-18T03:39:57.348257+0000","last_unstale":"2023-07-18T03:39:57.355019+0000","last_undegraded":"2023-07-18T03:39:57.355019+0000","last_fullsized":"2023-07-18T03:39:57.355019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:36:37.474323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359688+0000","last_change":"2023-07-18T03:39:56.337863+0000","last_active":"2023-07-18T03:39:57.359688+0000","last_peered":"2023-07-18T03:39:57.359688+0000","last_clean":"2023-07-18T03:39:57.359688+0000","last_became_active":"2023-07-18T03:39:56.337561+0000","last_became_peered":"2023-07-18T03:39:56.337561+0000","last_unstale":"2023-07-18T03:39:57.359688+0000","last_undegraded":"2023-07-18T03:39:57.359688+0000","last_fullsized":"2023-07-18T03:39:57.359688+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:10.926722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.353775+0000","last_change":"2023-07-18T03:39:57.351197+0000","last_active":"2023-07-18T03:39:57.353775+0000","last_peered":"2023-07-18T03:39:57.353775+0000","last_clean":"2023-07-18T03:39:57.353775+0000","last_became_active":"2023-07-18T03:39:57.350814+0000","last_became_peered":"2023-07-18T03:39:57.350814+0000","last_unstale":"2023-07-18T03:39:57.353775+0000","last_undegraded":"2023-07-18T03:39:57.353775+0000","last_fullsized":"2023-07-18T03:39:57.353775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:06:35.294279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.339081+0000","last_change":"2023-07-18T03:39:56.336889+0000","last_active":"2023-07-18T03:39:57.339081+0000","last_peered":"2023-07-18T03:39:57.339081+0000","last_clean":"2023-07-18T03:39:57.339081+0000","last_became_active":"2023-07-18T03:39:56.336599+0000","last_became_peered":"2023-07-18T03:39:56.336599+0000","last_unstale":"2023-07-18T03:39:57.339081+0000","last_undegraded":"2023-07-18T03:39:57.339081+0000","last_fullsized":"2023-07-18T03:39:57.339081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:10.544538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.333821+0000","last_change":"2023-07-18T03:39:56.333821+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.333821+0000","last_undegraded":"2023-07-18T03:39:56.333821+0000","last_fullsized":"2023-07-18T03:39:56.333821+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.331084+0000","last_change":"2023-07-18T03:39:56.350386+0000","last_active":"2023-07-18T03:39:57.331084+0000","last_peered":"2023-07-18T03:39:57.331084+0000","last_clean":"2023-07-18T03:39:57.331084+0000","last_became_active":"2023-07-18T03:39:56.350046+0000","last_became_peered":"2023-07-18T03:39:56.350046+0000","last_unstale":"2023-07-18T03:39:57.331084+0000","last_undegraded":"2023-07-18T03:39:57.331084+0000","last_fullsized":"2023-07-18T03:39:57.331084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:57:37.239962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.339618+0000","last_change":"2023-07-18T03:39:56.339618+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.339618+0000","last_undegraded":"2023-07-18T03:39:56.339618+0000","last_fullsized":"2023-07-18T03:39:56.339618+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.338697+0000","last_change":"2023-07-18T03:39:56.338127+0000","last_active":"2023-07-18T03:39:57.338697+0000","last_peered":"2023-07-18T03:39:57.338697+0000","last_clean":"2023-07-18T03:39:57.338697+0000","last_became_active":"2023-07-18T03:39:56.337747+0000","last_became_peered":"2023-07-18T03:39:56.337747+0000","last_unstale":"2023-07-18T03:39:57.338697+0000","last_undegraded":"2023-07-18T03:39:57.338697+0000","last_fullsized":"2023-07-18T03:39:57.338697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:08:17.902329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328126+0000","last_change":"2023-07-18T03:39:56.328126+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328126+0000","last_undegraded":"2023-07-18T03:39:56.328126+0000","last_fullsized":"2023-07-18T03:39:56.328126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.355935+0000","last_change":"2023-07-18T03:39:56.353087+0000","last_active":"2023-07-18T03:39:56.355935+0000","last_peered":"2023-07-18T03:39:56.355935+0000","last_clean":"2023-07-18T03:39:56.355935+0000","last_became_active":"2023-07-18T03:39:56.352738+0000","last_became_peered":"2023-07-18T03:39:56.352738+0000","last_unstale":"2023-07-18T03:39:56.355935+0000","last_undegraded":"2023-07-18T03:39:56.355935+0000","last_fullsized":"2023-07-18T03:39:56.355935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:38:20.581738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.329523+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:39:56.329523+0000","last_peered":"2023-07-18T03:39:56.329523+0000","last_clean":"2023-07-18T03:39:56.329523+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:39:56.329523+0000","last_undegraded":"2023-07-18T03:39:56.329523+0000","last_fullsized":"2023-07-18T03:39:56.329523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:58:20.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355192+0000","last_change":"2023-07-18T03:39:57.348865+0000","last_active":"2023-07-18T03:39:57.355192+0000","last_peered":"2023-07-18T03:39:57.355192+0000","last_clean":"2023-07-18T03:39:57.355192+0000","last_became_active":"2023-07-18T03:39:57.348246+0000","last_became_peered":"2023-07-18T03:39:57.348246+0000","last_unstale":"2023-07-18T03:39:57.355192+0000","last_undegraded":"2023-07-18T03:39:57.355192+0000","last_fullsized":"2023-07-18T03:39:57.355192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:47:29.972115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354659+0000","last_change":"2023-07-18T03:39:56.352671+0000","last_active":"2023-07-18T03:39:56.354659+0000","last_peered":"2023-07-18T03:39:56.354659+0000","last_clean":"2023-07-18T03:39:56.354659+0000","last_became_active":"2023-07-18T03:39:56.351865+0000","last_became_peered":"2023-07-18T03:39:56.351865+0000","last_unstale":"2023-07-18T03:39:56.354659+0000","last_undegraded":"2023-07-18T03:39:56.354659+0000","last_fullsized":"2023-07-18T03:39:56.354659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:01:05.415848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.342292+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:39:56.342292+0000","last_peered":"2023-07-18T03:39:56.342292+0000","last_clean":"2023-07-18T03:39:56.342292+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:39:56.342292+0000","last_undegraded":"2023-07-18T03:39:56.342292+0000","last_fullsized":"2023-07-18T03:39:56.342292+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:40:43.825533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.329925+0000","last_change":"2023-07-18T03:39:56.329925+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.329925+0000","last_undegraded":"2023-07-18T03:39:56.329925+0000","last_fullsized":"2023-07-18T03:39:56.329925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.349349+0000","last_change":"2023-07-18T03:39:56.347782+0000","last_active":"2023-07-18T03:39:56.349349+0000","last_peered":"2023-07-18T03:39:56.349349+0000","last_clean":"2023-07-18T03:39:56.349349+0000","last_became_active":"2023-07-18T03:39:56.347519+0000","last_became_peered":"2023-07-18T03:39:56.347519+0000","last_unstale":"2023-07-18T03:39:56.349349+0000","last_undegraded":"2023-07-18T03:39:56.349349+0000","last_fullsized":"2023-07-18T03:39:56.349349+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:25.713685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.332914+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:39:56.332914+0000","last_peered":"2023-07-18T03:39:56.332914+0000","last_clean":"2023-07-18T03:39:56.332914+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:39:56.332914+0000","last_undegraded":"2023-07-18T03:39:56.332914+0000","last_fullsized":"2023-07-18T03:39:56.332914+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:40:36.216166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.330424+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:39:57.330424+0000","last_peered":"2023-07-18T03:39:57.330424+0000","last_clean":"2023-07-18T03:39:57.330424+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:39:57.330424+0000","last_undegraded":"2023-07-18T03:39:57.330424+0000","last_fullsized":"2023-07-18T03:39:57.330424+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:22:09.535618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355161+0000","last_change":"2023-07-18T03:39:57.350408+0000","last_active":"2023-07-18T03:39:57.355161+0000","last_peered":"2023-07-18T03:39:57.355161+0000","last_clean":"2023-07-18T03:39:57.355161+0000","last_became_active":"2023-07-18T03:39:57.350103+0000","last_became_peered":"2023-07-18T03:39:57.350103+0000","last_unstale":"2023-07-18T03:39:57.355161+0000","last_undegraded":"2023-07-18T03:39:57.355161+0000","last_fullsized":"2023-07-18T03:39:57.355161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:47:42.227277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.338099+0000","last_change":"2023-07-18T03:39:56.346081+0000","last_active":"2023-07-18T03:39:57.338099+0000","last_peered":"2023-07-18T03:39:57.338099+0000","last_clean":"2023-07-18T03:39:57.338099+0000","last_became_active":"2023-07-18T03:39:56.345762+0000","last_became_peered":"2023-07-18T03:39:56.345762+0000","last_unstale":"2023-07-18T03:39:57.338099+0000","last_undegraded":"2023-07-18T03:39:57.338099+0000","last_fullsized":"2023-07-18T03:39:57.338099+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:15:29.121017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":45,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.329508+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:39:57.329508+0000","last_peered":"2023-07-18T03:39:57.329508+0000","last_clean":"2023-07-18T03:39:57.329508+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:39:57.329508+0000","last_undegraded":"2023-07-18T03:39:57.329508+0000","last_fullsized":"2023-07-18T03:39:57.329508+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:13:09.287323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.336448+0000","last_change":"2023-07-18T03:39:56.336448+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.336448+0000","last_undegraded":"2023-07-18T03:39:56.336448+0000","last_fullsized":"2023-07-18T03:39:56.336448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.330231+0000","last_change":"2023-07-18T03:39:56.348881+0000","last_active":"2023-07-18T03:39:57.330231+0000","last_peered":"2023-07-18T03:39:57.330231+0000","last_clean":"2023-07-18T03:39:57.330231+0000","last_became_active":"2023-07-18T03:39:56.348622+0000","last_became_peered":"2023-07-18T03:39:56.348622+0000","last_unstale":"2023-07-18T03:39:57.330231+0000","last_undegraded":"2023-07-18T03:39:57.330231+0000","last_fullsized":"2023-07-18T03:39:57.330231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:53:21.359259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":45,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.330272+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:39:57.330272+0000","last_peered":"2023-07-18T03:39:57.330272+0000","last_clean":"2023-07-18T03:39:57.330272+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:39:57.330272+0000","last_undegraded":"2023-07-18T03:39:57.330272+0000","last_fullsized":"2023-07-18T03:39:57.330272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:39:12.642880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335532+0000","last_change":"2023-07-18T03:39:56.335532+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335532+0000","last_undegraded":"2023-07-18T03:39:56.335532+0000","last_fullsized":"2023-07-18T03:39:56.335532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.336107+0000","last_change":"2023-07-18T03:39:56.337331+0000","last_active":"2023-07-18T03:39:57.336107+0000","last_peered":"2023-07-18T03:39:57.336107+0000","last_clean":"2023-07-18T03:39:57.336107+0000","last_became_active":"2023-07-18T03:39:56.337021+0000","last_became_peered":"2023-07-18T03:39:56.337021+0000","last_unstale":"2023-07-18T03:39:57.336107+0000","last_undegraded":"2023-07-18T03:39:57.336107+0000","last_fullsized":"2023-07-18T03:39:57.336107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:22:50.209633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.340413+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:39:56.340413+0000","last_peered":"2023-07-18T03:39:56.340413+0000","last_clean":"2023-07-18T03:39:56.340413+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:39:56.340413+0000","last_undegraded":"2023-07-18T03:39:56.340413+0000","last_fullsized":"2023-07-18T03:39:56.340413+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:45:16.320741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.329526+0000","last_change":"2023-07-18T03:39:56.329526+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.329526+0000","last_undegraded":"2023-07-18T03:39:56.329526+0000","last_fullsized":"2023-07-18T03:39:56.329526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.340100+0000","last_change":"2023-07-18T03:39:56.341357+0000","last_active":"2023-07-18T03:39:57.340100+0000","last_peered":"2023-07-18T03:39:57.340100+0000","last_clean":"2023-07-18T03:39:57.340100+0000","last_became_active":"2023-07-18T03:39:56.341092+0000","last_became_peered":"2023-07-18T03:39:56.341092+0000","last_unstale":"2023-07-18T03:39:57.340100+0000","last_undegraded":"2023-07-18T03:39:57.340100+0000","last_fullsized":"2023-07-18T03:39:57.340100+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:48:52.596611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.339973+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:39:57.339973+0000","last_peered":"2023-07-18T03:39:57.339973+0000","last_clean":"2023-07-18T03:39:57.339973+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:39:57.339973+0000","last_undegraded":"2023-07-18T03:39:57.339973+0000","last_fullsized":"2023-07-18T03:39:57.339973+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:48:55.614775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.363216+0000","last_change":"2023-07-18T03:39:57.360948+0000","last_active":"2023-07-18T03:39:57.363216+0000","last_peered":"2023-07-18T03:39:57.363216+0000","last_clean":"2023-07-18T03:39:57.363216+0000","last_became_active":"2023-07-18T03:39:57.360601+0000","last_became_peered":"2023-07-18T03:39:57.360601+0000","last_unstale":"2023-07-18T03:39:57.363216+0000","last_undegraded":"2023-07-18T03:39:57.363216+0000","last_fullsized":"2023-07-18T03:39:57.363216+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:49:44.089477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.336599+0000","last_change":"2023-07-18T03:39:56.342369+0000","last_active":"2023-07-18T03:39:57.336599+0000","last_peered":"2023-07-18T03:39:57.336599+0000","last_clean":"2023-07-18T03:39:57.336599+0000","last_became_active":"2023-07-18T03:39:56.342108+0000","last_became_peered":"2023-07-18T03:39:56.342108+0000","last_unstale":"2023-07-18T03:39:57.336599+0000","last_undegraded":"2023-07-18T03:39:57.336599+0000","last_fullsized":"2023-07-18T03:39:57.336599+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:44.715606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.340446+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:39:57.340446+0000","last_peered":"2023-07-18T03:39:57.340446+0000","last_clean":"2023-07-18T03:39:57.340446+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:39:57.340446+0000","last_undegraded":"2023-07-18T03:39:57.340446+0000","last_fullsized":"2023-07-18T03:39:57.340446+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:54:03.329884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":652,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":999424,"data_stored":752338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":1.7729999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":2.2130000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":3.0030000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.2290000000000001,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":2.2570000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.0249999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":4.0010000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":1.2889999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":2.2989999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":3.2029999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":3.1219999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":1.0609999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":393216,"data_stored":160918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.626}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.72899999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.69299999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67800000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.82499999999999996}]}]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":376832,"data_stored":153208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52600000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.8290000000000002,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.747},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.80400000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":970752,"data_stored":743576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32200000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.495}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79000000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.73899999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":376832,"data_stored":153208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.82399999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.78500000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64700000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.75800000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.68899999999999995}]}]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":376832,"data_stored":153208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":4.9260000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":2.1699999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":8.3780000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":5.5060000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":4.8769999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":2.1230000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":2.2370000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":1.6599999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":4.9690000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":3.6829999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":5.3140000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":3.7389999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.71799999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":393216,"data_stored":160370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":8.3040000000000003,"5min":8.3040000000000003,"15min":8.3040000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.502}]},{"osd":3,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.0229999999999997,"5min":8.0229999999999997,"15min":8.0229999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":8.359,"5min":8.359,"15min":8.359},"last":0.48099999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.89900000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.78300000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.88200000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.97399999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":393216,"data_stored":160406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39100000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.438}]},{"osd":3,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.78100000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.79600000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.60499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:40:01.802 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:40:02.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.043+0000 7fb8925ff700 1 Processor -- start 2023-07-18T03:40:02.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.043+0000 7fb8925ff700 1 -- start start 2023-07-18T03:40:02.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.044+0000 7fb8925ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb88c0d1220 0x7fb88c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.044+0000 7fb8925ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb88c0b1e00 0x7fb88c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.044+0000 7fb8925ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88c0d2ce0 con 0x7fb88c15c950 2023-07-18T03:40:02.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.044+0000 7fb8925ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88c0b3350 con 0x7fb88c0b1e00 2023-07-18T03:40:02.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.044+0000 7fb8925ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0b3630 con 0x7fb88c0d1220 2023-07-18T03:40:02.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.045+0000 7fb88bfff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb88c15c950 0x7fb88c0d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56826/0 (socket says 172.21.15.49:56826) 2023-07-18T03:40:02.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.045+0000 7fb88bfff700 1 -- 172.21.15.49:0/2962531601 learned_addr learned my addr 172.21.15.49:0/2962531601 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:02.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.045+0000 7fb890b9c700 1 --2- 172.21.15.49:0/2962531601 >> v2:172.21.15.49:3300/0 conn(0x7fb88c0b1e00 0x7fb88c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.045+0000 7fb88b7fe700 1 --2- 172.21.15.49:0/2962531601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb88c0d1220 0x7fb88c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.045+0000 7fb890b9c700 1 -- 172.21.15.49:0/2962531601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb88c0d1220 msgr2=0x7fb88c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:02.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.045+0000 7fb890b9c700 1 --2- 172.21.15.49:0/2962531601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb88c0d1220 0x7fb88c0b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.045+0000 7fb890b9c700 1 -- 172.21.15.49:0/2962531601 >> v1:172.21.15.49:6789/0 conn(0x7fb88c15c950 legacy=0x7fb88c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.045+0000 7fb890b9c700 1 -- 172.21.15.49:0/2962531601 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c0b3910 con 0x7fb88c0b1e00 2023-07-18T03:40:02.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.045+0000 7fb88affd700 1 -- 172.21.15.49:0/2962531601 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354467657 0 0) 0x7fb88c0d2ce0 con 0x7fb88c15c950 2023-07-18T03:40:02.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.045+0000 7fb890b9c700 1 --2- 172.21.15.49:0/2962531601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88c0b1e00 0x7fb88c0afac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb884009d90 tx=0x7fb88400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8f6eedfd3c97f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:02.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.046+0000 7fb88affd700 1 -- 172.21.15.49:0/2962531601 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400e070 con 0x7fb88c0b1e00 2023-07-18T03:40:02.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.046+0000 7fb88affd700 1 -- 172.21.15.49:0/2962531601 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88400bb20 con 0x7fb88c0b1e00 2023-07-18T03:40:02.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.046+0000 7fb88affd700 1 -- 172.21.15.49:0/2962531601 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884012430 con 0x7fb88c0b1e00 2023-07-18T03:40:02.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.046+0000 7fb8925ff700 1 -- 172.21.15.49:0/2962531601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88c0b1e00 msgr2=0x7fb88c0afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.046+0000 7fb8925ff700 1 --2- 172.21.15.49:0/2962531601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88c0b1e00 0x7fb88c0afac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb884009d90 tx=0x7fb88400b330 comp rx=0 tx=0).stop 2023-07-18T03:40:02.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.046+0000 7fb8925ff700 1 -- 172.21.15.49:0/2962531601 shutdown_connections 2023-07-18T03:40:02.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.046+0000 7fb8925ff700 1 --2- 172.21.15.49:0/2962531601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88c0b1e00 0x7fb88c0afac0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.046+0000 7fb8925ff700 1 --2- 172.21.15.49:0/2962531601 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb88c0d1220 0x7fb88c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.046+0000 7fb8925ff700 1 -- 172.21.15.49:0/2962531601 >> 172.21.15.49:0/2962531601 conn(0x7fb88c09c1d0 msgr2=0x7fb88c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:02.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.046+0000 7fb8925ff700 1 -- 172.21.15.49:0/2962531601 shutdown_connections 2023-07-18T03:40:02.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.046+0000 7fb8925ff700 1 -- 172.21.15.49:0/2962531601 wait complete. 2023-07-18T03:40:02.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.048+0000 7fb8925ff700 1 Processor -- start 2023-07-18T03:40:02.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.048+0000 7fb8925ff700 1 -- start start 2023-07-18T03:40:02.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.049+0000 7fb8925ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb88c0b1e00 0x7fb88c0ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.049+0000 7fb8925ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88c0d1220 0x7fb88c1f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.049+0000 7fb8925ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb88c15c950 0x7fb88c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.049+0000 7fb8925ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c1f6960 con 0x7fb88c0d1220 2023-07-18T03:40:02.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.049+0000 7fb8925ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0cfc80 con 0x7fb88c15c950 2023-07-18T03:40:02.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.049+0000 7fb8925ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0cff60 con 0x7fb88c0b1e00 2023-07-18T03:40:02.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.050+0000 7fb88b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88c0d1220 0x7fb88c1f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.050+0000 7fb88b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88c0d1220 0x7fb88c1f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38422/0 (socket says 172.21.15.49:38422) 2023-07-18T03:40:02.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.050+0000 7fb88b7fe700 1 -- 172.21.15.49:0/2322825947 learned_addr learned my addr 172.21.15.49:0/2322825947 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:02.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.050+0000 7fb88bfff700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb88c0b1e00 0x7fb88c0ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.050+0000 7fb890b9c700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb88c15c950 0x7fb88c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.050+0000 7fb88b7fe700 1 -- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb88c0b1e00 msgr2=0x7fb88c0ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.050+0000 7fb88b7fe700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb88c0b1e00 0x7fb88c0ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.050+0000 7fb88b7fe700 1 -- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb88c15c950 msgr2=0x7fb88c154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.050+0000 7fb88b7fe700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb88c15c950 0x7fb88c154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.050+0000 7fb88b7fe700 1 -- 172.21.15.49:0/2322825947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb884009a70 con 0x7fb88c0d1220 2023-07-18T03:40:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.050+0000 7fb88b7fe700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88c0d1220 0x7fb88c1f6420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e980 tx=0x7fb88000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.051+0000 7fb888ff9700 1 -- 172.21.15.49:0/2322825947 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880009710 con 0x7fb88c0d1220 2023-07-18T03:40:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.051+0000 7fb888ff9700 1 -- 172.21.15.49:0/2322825947 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88000c690 con 0x7fb88c0d1220 2023-07-18T03:40:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.051+0000 7fb8925ff700 1 -- 172.21.15.49:0/2322825947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c0d0240 con 0x7fb88c0d1220 2023-07-18T03:40:02.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.051+0000 7fb888ff9700 1 -- 172.21.15.49:0/2322825947 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88000ca40 con 0x7fb88c0d1220 2023-07-18T03:40:02.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.051+0000 7fb8925ff700 1 -- 172.21.15.49:0/2322825947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c0cf480 con 0x7fb88c0d1220 2023-07-18T03:40:02.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.052+0000 7fb8925ff700 1 -- 172.21.15.49:0/2322825947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88c04e280 con 0x7fb88c0d1220 2023-07-18T03:40:02.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.053+0000 7fb888ff9700 1 -- 172.21.15.49:0/2322825947 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb88000cba0 con 0x7fb88c0d1220 2023-07-18T03:40:02.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.054+0000 7fb888ff9700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb87806ace0 0x7fb87806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.054+0000 7fb888ff9700 1 -- 172.21.15.49:0/2322825947 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb8800893d0 con 0x7fb88c0d1220 2023-07-18T03:40:02.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.055+0000 7fb88bfff700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb87806ace0 0x7fb87806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.055+0000 7fb88bfff700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb87806ace0 0x7fb87806d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb87c007fd0 tx=0x7fb87c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:02.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.059+0000 7fb888ff9700 1 -- 172.21.15.49:0/2322825947 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb880057fa0 con 0x7fb88c0d1220 2023-07-18T03:40:02.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.263+0000 7fb8925ff700 1 -- 172.21.15.49:0/2322825947 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb88c0b3ed0 con 0x7fb87806ace0 2023-07-18T03:40:02.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.272+0000 7fb888ff9700 1 -- 172.21.15.49:0/2322825947 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309001 (secure 0 0 0) 0x7fb87c001d00 con 0x7fb87806ace0 2023-07-18T03:40:02.271 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:02.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.277+0000 7fb8925ff700 1 -- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb87806ace0 msgr2=0x7fb87806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.277+0000 7fb8925ff700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb87806ace0 0x7fb87806d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb87c007fd0 tx=0x7fb87c009040 comp rx=0 tx=0).stop 2023-07-18T03:40:02.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.277+0000 7fb8925ff700 1 -- 172.21.15.49:0/2322825947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88c0d1220 msgr2=0x7fb88c1f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.277+0000 7fb8925ff700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88c0d1220 0x7fb88c1f6420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e980 tx=0x7fb88000ec90 comp rx=0 tx=0).stop 2023-07-18T03:40:02.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.277+0000 7fb8925ff700 1 -- 172.21.15.49:0/2322825947 shutdown_connections 2023-07-18T03:40:02.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.277+0000 7fb8925ff700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb88c15c950 0x7fb88c154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.277+0000 7fb8925ff700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb88c0b1e00 0x7fb88c0ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.277+0000 7fb8925ff700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb87806ace0 0x7fb87806d190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.277+0000 7fb8925ff700 1 --2- 172.21.15.49:0/2322825947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88c0d1220 0x7fb88c1f6420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.277+0000 7fb8925ff700 1 -- 172.21.15.49:0/2322825947 >> 172.21.15.49:0/2322825947 conn(0x7fb88c09c1d0 msgr2=0x7fb88c0d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:02.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.277+0000 7fb8925ff700 1 -- 172.21.15.49:0/2322825947 shutdown_connections 2023-07-18T03:40:02.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.277+0000 7fb8925ff700 1 -- 172.21.15.49:0/2322825947 wait complete. 2023-07-18T03:40:02.275 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:40:02.290 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-07-18T03:40:02.117452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591020,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":177,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236180,"kb_used_data":4180,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738540,"statfs":{"total":773094113280,"available":772852264960,"internally_reserved":0,"allocated":4280320,"data_stored":2437232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":652,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001415"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.337498+0000","last_change":"2023-07-18T03:39:56.337498+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.337498+0000","last_undegraded":"2023-07-18T03:39:56.337498+0000","last_fullsized":"2023-07-18T03:39:56.337498+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335988+0000","last_change":"2023-07-18T03:39:56.335988+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335988+0000","last_undegraded":"2023-07-18T03:39:56.335988+0000","last_fullsized":"2023-07-18T03:39:56.335988+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.352684+0000","last_change":"2023-07-18T03:39:57.349470+0000","last_active":"2023-07-18T03:39:57.352684+0000","last_peered":"2023-07-18T03:39:57.352684+0000","last_clean":"2023-07-18T03:39:57.352684+0000","last_became_active":"2023-07-18T03:39:57.349188+0000","last_became_peered":"2023-07-18T03:39:57.349188+0000","last_unstale":"2023-07-18T03:39:57.352684+0000","last_undegraded":"2023-07-18T03:39:57.352684+0000","last_fullsized":"2023-07-18T03:39:57.352684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:30:54.522092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.360068+0000","last_change":"2023-07-18T03:39:57.356421+0000","last_active":"2023-07-18T03:39:57.360068+0000","last_peered":"2023-07-18T03:39:57.360068+0000","last_clean":"2023-07-18T03:39:57.360068+0000","last_became_active":"2023-07-18T03:39:57.356128+0000","last_became_peered":"2023-07-18T03:39:57.356128+0000","last_unstale":"2023-07-18T03:39:57.360068+0000","last_undegraded":"2023-07-18T03:39:57.360068+0000","last_fullsized":"2023-07-18T03:39:57.360068+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:32:15.921500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359782+0000","last_change":"2023-07-18T03:39:57.356364+0000","last_active":"2023-07-18T03:39:57.359782+0000","last_peered":"2023-07-18T03:39:57.359782+0000","last_clean":"2023-07-18T03:39:57.359782+0000","last_became_active":"2023-07-18T03:39:57.356079+0000","last_became_peered":"2023-07-18T03:39:57.356079+0000","last_unstale":"2023-07-18T03:39:57.359782+0000","last_undegraded":"2023-07-18T03:39:57.359782+0000","last_fullsized":"2023-07-18T03:39:57.359782+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:08:41.904219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328919+0000","last_change":"2023-07-18T03:39:56.328919+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328919+0000","last_undegraded":"2023-07-18T03:39:56.328919+0000","last_fullsized":"2023-07-18T03:39:56.328919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332077+0000","last_change":"2023-07-18T03:39:56.332077+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332077+0000","last_undegraded":"2023-07-18T03:39:56.332077+0000","last_fullsized":"2023-07-18T03:39:56.332077+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335775+0000","last_change":"2023-07-18T03:39:56.335775+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335775+0000","last_undegraded":"2023-07-18T03:39:56.335775+0000","last_fullsized":"2023-07-18T03:39:56.335775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.357950+0000","last_change":"2023-07-18T03:39:57.352415+0000","last_active":"2023-07-18T03:39:57.357950+0000","last_peered":"2023-07-18T03:39:57.357950+0000","last_clean":"2023-07-18T03:39:57.357950+0000","last_became_active":"2023-07-18T03:39:57.352137+0000","last_became_peered":"2023-07-18T03:39:57.352137+0000","last_unstale":"2023-07-18T03:39:57.357950+0000","last_undegraded":"2023-07-18T03:39:57.357950+0000","last_fullsized":"2023-07-18T03:39:57.357950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:36.387384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.338378+0000","last_change":"2023-07-18T03:39:56.338378+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.338378+0000","last_undegraded":"2023-07-18T03:39:56.338378+0000","last_fullsized":"2023-07-18T03:39:56.338378+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.333492+0000","last_change":"2023-07-18T03:39:56.333492+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.333492+0000","last_undegraded":"2023-07-18T03:39:56.333492+0000","last_fullsized":"2023-07-18T03:39:56.333492+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.351146+0000","last_change":"2023-07-18T03:39:57.345772+0000","last_active":"2023-07-18T03:39:57.351146+0000","last_peered":"2023-07-18T03:39:57.351146+0000","last_clean":"2023-07-18T03:39:57.351146+0000","last_became_active":"2023-07-18T03:39:57.345465+0000","last_became_peered":"2023-07-18T03:39:57.345465+0000","last_unstale":"2023-07-18T03:39:57.351146+0000","last_undegraded":"2023-07-18T03:39:57.351146+0000","last_fullsized":"2023-07-18T03:39:57.351146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:48:30.748069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332641+0000","last_change":"2023-07-18T03:39:56.332641+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332641+0000","last_undegraded":"2023-07-18T03:39:56.332641+0000","last_fullsized":"2023-07-18T03:39:56.332641+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.327178+0000","last_change":"2023-07-18T03:39:56.327178+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.327178+0000","last_undegraded":"2023-07-18T03:39:56.327178+0000","last_fullsized":"2023-07-18T03:39:56.327178+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332947+0000","last_change":"2023-07-18T03:39:56.332947+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332947+0000","last_undegraded":"2023-07-18T03:39:56.332947+0000","last_fullsized":"2023-07-18T03:39:56.332947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.348720+0000","last_change":"2023-07-18T03:39:57.337636+0000","last_active":"2023-07-18T03:39:57.348720+0000","last_peered":"2023-07-18T03:39:57.348720+0000","last_clean":"2023-07-18T03:39:57.348720+0000","last_became_active":"2023-07-18T03:39:57.337330+0000","last_became_peered":"2023-07-18T03:39:57.337330+0000","last_unstale":"2023-07-18T03:39:57.348720+0000","last_undegraded":"2023-07-18T03:39:57.348720+0000","last_fullsized":"2023-07-18T03:39:57.348720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:22:11.047392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354699+0000","last_change":"2023-07-18T03:39:57.345108+0000","last_active":"2023-07-18T03:39:57.354699+0000","last_peered":"2023-07-18T03:39:57.354699+0000","last_clean":"2023-07-18T03:39:57.354699+0000","last_became_active":"2023-07-18T03:39:57.344825+0000","last_became_peered":"2023-07-18T03:39:57.344825+0000","last_unstale":"2023-07-18T03:39:57.354699+0000","last_undegraded":"2023-07-18T03:39:57.354699+0000","last_fullsized":"2023-07-18T03:39:57.354699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:07:59.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.365129+0000","last_change":"2023-07-18T03:39:57.361051+0000","last_active":"2023-07-18T03:39:57.365129+0000","last_peered":"2023-07-18T03:39:57.365129+0000","last_clean":"2023-07-18T03:39:57.365129+0000","last_became_active":"2023-07-18T03:39:57.360746+0000","last_became_peered":"2023-07-18T03:39:57.360746+0000","last_unstale":"2023-07-18T03:39:57.365129+0000","last_undegraded":"2023-07-18T03:39:57.365129+0000","last_fullsized":"2023-07-18T03:39:57.365129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:11:30.468969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359427+0000","last_change":"2023-07-18T03:39:57.351301+0000","last_active":"2023-07-18T03:39:57.359427+0000","last_peered":"2023-07-18T03:39:57.359427+0000","last_clean":"2023-07-18T03:39:57.359427+0000","last_became_active":"2023-07-18T03:39:57.351033+0000","last_became_peered":"2023-07-18T03:39:57.351033+0000","last_unstale":"2023-07-18T03:39:57.359427+0000","last_undegraded":"2023-07-18T03:39:57.359427+0000","last_fullsized":"2023-07-18T03:39:57.359427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:50:08.481909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.365031+0000","last_change":"2023-07-18T03:39:57.360471+0000","last_active":"2023-07-18T03:39:57.365031+0000","last_peered":"2023-07-18T03:39:57.365031+0000","last_clean":"2023-07-18T03:39:57.365031+0000","last_became_active":"2023-07-18T03:39:57.360202+0000","last_became_peered":"2023-07-18T03:39:57.360202+0000","last_unstale":"2023-07-18T03:39:57.365031+0000","last_undegraded":"2023-07-18T03:39:57.365031+0000","last_fullsized":"2023-07-18T03:39:57.365031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:32.300586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.331681+0000","last_change":"2023-07-18T03:39:56.331681+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.331681+0000","last_undegraded":"2023-07-18T03:39:56.331681+0000","last_fullsized":"2023-07-18T03:39:56.331681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.329011+0000","last_change":"2023-07-18T03:39:56.329011+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.329011+0000","last_undegraded":"2023-07-18T03:39:56.329011+0000","last_fullsized":"2023-07-18T03:39:56.329011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354754+0000","last_change":"2023-07-18T03:39:57.347725+0000","last_active":"2023-07-18T03:39:57.354754+0000","last_peered":"2023-07-18T03:39:57.354754+0000","last_clean":"2023-07-18T03:39:57.354754+0000","last_became_active":"2023-07-18T03:39:57.346990+0000","last_became_peered":"2023-07-18T03:39:57.346990+0000","last_unstale":"2023-07-18T03:39:57.354754+0000","last_undegraded":"2023-07-18T03:39:57.354754+0000","last_fullsized":"2023-07-18T03:39:57.354754+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:33:51.412210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.361028+0000","last_change":"2023-07-18T03:39:57.355821+0000","last_active":"2023-07-18T03:39:57.361028+0000","last_peered":"2023-07-18T03:39:57.361028+0000","last_clean":"2023-07-18T03:39:57.361028+0000","last_became_active":"2023-07-18T03:39:57.355505+0000","last_became_peered":"2023-07-18T03:39:57.355505+0000","last_unstale":"2023-07-18T03:39:57.361028+0000","last_undegraded":"2023-07-18T03:39:57.361028+0000","last_fullsized":"2023-07-18T03:39:57.361028+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:11:11.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.358089+0000","last_change":"2023-07-18T03:39:57.347320+0000","last_active":"2023-07-18T03:39:57.358089+0000","last_peered":"2023-07-18T03:39:57.358089+0000","last_clean":"2023-07-18T03:39:57.358089+0000","last_became_active":"2023-07-18T03:39:57.347051+0000","last_became_peered":"2023-07-18T03:39:57.347051+0000","last_unstale":"2023-07-18T03:39:57.358089+0000","last_undegraded":"2023-07-18T03:39:57.358089+0000","last_fullsized":"2023-07-18T03:39:57.358089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:45:31.179450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328947+0000","last_change":"2023-07-18T03:39:56.328947+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328947+0000","last_undegraded":"2023-07-18T03:39:56.328947+0000","last_fullsized":"2023-07-18T03:39:56.328947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.362856+0000","last_change":"2023-07-18T03:39:57.349732+0000","last_active":"2023-07-18T03:39:57.362856+0000","last_peered":"2023-07-18T03:39:57.362856+0000","last_clean":"2023-07-18T03:39:57.362856+0000","last_became_active":"2023-07-18T03:39:57.349421+0000","last_became_peered":"2023-07-18T03:39:57.349421+0000","last_unstale":"2023-07-18T03:39:57.362856+0000","last_undegraded":"2023-07-18T03:39:57.362856+0000","last_fullsized":"2023-07-18T03:39:57.362856+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:03:41.731722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.337384+0000","last_change":"2023-07-18T03:39:56.337384+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.337384+0000","last_undegraded":"2023-07-18T03:39:56.337384+0000","last_fullsized":"2023-07-18T03:39:56.337384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.362756+0000","last_change":"2023-07-18T03:39:57.354325+0000","last_active":"2023-07-18T03:39:57.362756+0000","last_peered":"2023-07-18T03:39:57.362756+0000","last_clean":"2023-07-18T03:39:57.362756+0000","last_became_active":"2023-07-18T03:39:57.353910+0000","last_became_peered":"2023-07-18T03:39:57.353910+0000","last_unstale":"2023-07-18T03:39:57.362756+0000","last_undegraded":"2023-07-18T03:39:57.362756+0000","last_fullsized":"2023-07-18T03:39:57.362756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:18:46.390308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.364995+0000","last_change":"2023-07-18T03:39:57.362161+0000","last_active":"2023-07-18T03:39:57.364995+0000","last_peered":"2023-07-18T03:39:57.364995+0000","last_clean":"2023-07-18T03:39:57.364995+0000","last_became_active":"2023-07-18T03:39:57.361882+0000","last_became_peered":"2023-07-18T03:39:57.361882+0000","last_unstale":"2023-07-18T03:39:57.364995+0000","last_undegraded":"2023-07-18T03:39:57.364995+0000","last_fullsized":"2023-07-18T03:39:57.364995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:39:10.257984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354777+0000","last_change":"2023-07-18T03:39:57.347923+0000","last_active":"2023-07-18T03:39:57.354777+0000","last_peered":"2023-07-18T03:39:57.354777+0000","last_clean":"2023-07-18T03:39:57.354777+0000","last_became_active":"2023-07-18T03:39:57.347419+0000","last_became_peered":"2023-07-18T03:39:57.347419+0000","last_unstale":"2023-07-18T03:39:57.354777+0000","last_undegraded":"2023-07-18T03:39:57.354777+0000","last_fullsized":"2023-07-18T03:39:57.354777+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:54:59.548413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328673+0000","last_change":"2023-07-18T03:39:56.328673+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328673+0000","last_undegraded":"2023-07-18T03:39:56.328673+0000","last_fullsized":"2023-07-18T03:39:56.328673+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.348817+0000","last_change":"2023-07-18T03:39:57.338235+0000","last_active":"2023-07-18T03:39:57.348817+0000","last_peered":"2023-07-18T03:39:57.348817+0000","last_clean":"2023-07-18T03:39:57.348817+0000","last_became_active":"2023-07-18T03:39:57.337940+0000","last_became_peered":"2023-07-18T03:39:57.337940+0000","last_unstale":"2023-07-18T03:39:57.348817+0000","last_undegraded":"2023-07-18T03:39:57.348817+0000","last_fullsized":"2023-07-18T03:39:57.348817+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:42:25.160054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.365698+0000","last_change":"2023-07-18T03:39:56.337657+0000","last_active":"2023-07-18T03:39:57.365698+0000","last_peered":"2023-07-18T03:39:57.365698+0000","last_clean":"2023-07-18T03:39:57.365698+0000","last_became_active":"2023-07-18T03:39:56.337379+0000","last_became_peered":"2023-07-18T03:39:56.337379+0000","last_unstale":"2023-07-18T03:39:57.365698+0000","last_undegraded":"2023-07-18T03:39:57.365698+0000","last_fullsized":"2023-07-18T03:39:57.365698+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:30.478922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.330756+0000","last_change":"2023-07-18T03:39:56.330756+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.330756+0000","last_undegraded":"2023-07-18T03:39:56.330756+0000","last_fullsized":"2023-07-18T03:39:56.330756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355305+0000","last_change":"2023-07-18T03:39:56.345284+0000","last_active":"2023-07-18T03:39:57.355305+0000","last_peered":"2023-07-18T03:39:57.355305+0000","last_clean":"2023-07-18T03:39:57.355305+0000","last_became_active":"2023-07-18T03:39:56.344955+0000","last_became_peered":"2023-07-18T03:39:56.344955+0000","last_unstale":"2023-07-18T03:39:57.355305+0000","last_undegraded":"2023-07-18T03:39:57.355305+0000","last_fullsized":"2023-07-18T03:39:57.355305+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:50:11.765103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328243+0000","last_change":"2023-07-18T03:39:56.328243+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328243+0000","last_undegraded":"2023-07-18T03:39:56.328243+0000","last_fullsized":"2023-07-18T03:39:56.328243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.345414+0000","last_change":"2023-07-18T03:39:56.335839+0000","last_active":"2023-07-18T03:39:56.345414+0000","last_peered":"2023-07-18T03:39:56.345414+0000","last_clean":"2023-07-18T03:39:56.345414+0000","last_became_active":"2023-07-18T03:39:56.335506+0000","last_became_peered":"2023-07-18T03:39:56.335506+0000","last_unstale":"2023-07-18T03:39:56.345414+0000","last_undegraded":"2023-07-18T03:39:56.345414+0000","last_fullsized":"2023-07-18T03:39:56.345414+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:55:30.319672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.338278+0000","last_change":"2023-07-18T03:39:56.338278+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.338278+0000","last_undegraded":"2023-07-18T03:39:56.338278+0000","last_fullsized":"2023-07-18T03:39:56.338278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.355959+0000","last_change":"2023-07-18T03:39:56.353462+0000","last_active":"2023-07-18T03:39:56.355959+0000","last_peered":"2023-07-18T03:39:56.355959+0000","last_clean":"2023-07-18T03:39:56.355959+0000","last_became_active":"2023-07-18T03:39:56.353165+0000","last_became_peered":"2023-07-18T03:39:56.353165+0000","last_unstale":"2023-07-18T03:39:56.355959+0000","last_undegraded":"2023-07-18T03:39:56.355959+0000","last_fullsized":"2023-07-18T03:39:56.355959+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:24.138081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.350712+0000","last_change":"2023-07-18T03:39:57.345776+0000","last_active":"2023-07-18T03:39:57.350712+0000","last_peered":"2023-07-18T03:39:57.350712+0000","last_clean":"2023-07-18T03:39:57.350712+0000","last_became_active":"2023-07-18T03:39:57.345450+0000","last_became_peered":"2023-07-18T03:39:57.345450+0000","last_unstale":"2023-07-18T03:39:57.350712+0000","last_undegraded":"2023-07-18T03:39:57.350712+0000","last_fullsized":"2023-07-18T03:39:57.350712+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:39:58.863405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.348456+0000","last_change":"2023-07-18T03:39:56.343054+0000","last_active":"2023-07-18T03:39:56.348456+0000","last_peered":"2023-07-18T03:39:56.348456+0000","last_clean":"2023-07-18T03:39:56.348456+0000","last_became_active":"2023-07-18T03:39:56.342752+0000","last_became_peered":"2023-07-18T03:39:56.342752+0000","last_unstale":"2023-07-18T03:39:56.348456+0000","last_undegraded":"2023-07-18T03:39:56.348456+0000","last_fullsized":"2023-07-18T03:39:56.348456+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:42:03.205955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332552+0000","last_change":"2023-07-18T03:39:56.332552+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332552+0000","last_undegraded":"2023-07-18T03:39:56.332552+0000","last_fullsized":"2023-07-18T03:39:56.332552+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.356389+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:57.356389+0000","last_peered":"2023-07-18T03:39:57.356389+0000","last_clean":"2023-07-18T03:39:57.356389+0000","last_became_active":"2023-07-18T03:39:56.345390+0000","last_became_peered":"2023-07-18T03:39:56.345390+0000","last_unstale":"2023-07-18T03:39:57.356389+0000","last_undegraded":"2023-07-18T03:39:57.356389+0000","last_fullsized":"2023-07-18T03:39:57.356389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:57:57.797956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328578+0000","last_change":"2023-07-18T03:39:56.328578+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328578+0000","last_undegraded":"2023-07-18T03:39:56.328578+0000","last_fullsized":"2023-07-18T03:39:56.328578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.347760+0000","last_change":"2023-07-18T03:39:56.346375+0000","last_active":"2023-07-18T03:39:56.347760+0000","last_peered":"2023-07-18T03:39:56.347760+0000","last_clean":"2023-07-18T03:39:56.347760+0000","last_became_active":"2023-07-18T03:39:56.346098+0000","last_became_peered":"2023-07-18T03:39:56.346098+0000","last_unstale":"2023-07-18T03:39:56.347760+0000","last_undegraded":"2023-07-18T03:39:56.347760+0000","last_fullsized":"2023-07-18T03:39:56.347760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:50:22.604564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.339195+0000","last_change":"2023-07-18T03:39:56.339195+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.339195+0000","last_undegraded":"2023-07-18T03:39:56.339195+0000","last_fullsized":"2023-07-18T03:39:56.339195+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.341115+0000","last_change":"2023-07-18T03:39:56.337739+0000","last_active":"2023-07-18T03:39:56.341115+0000","last_peered":"2023-07-18T03:39:56.341115+0000","last_clean":"2023-07-18T03:39:56.341115+0000","last_became_active":"2023-07-18T03:39:56.337495+0000","last_became_peered":"2023-07-18T03:39:56.337495+0000","last_unstale":"2023-07-18T03:39:56.341115+0000","last_undegraded":"2023-07-18T03:39:56.341115+0000","last_fullsized":"2023-07-18T03:39:56.341115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:01.077387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.360048+0000","last_change":"2023-07-18T03:39:57.356339+0000","last_active":"2023-07-18T03:39:57.360048+0000","last_peered":"2023-07-18T03:39:57.360048+0000","last_clean":"2023-07-18T03:39:57.360048+0000","last_became_active":"2023-07-18T03:39:57.356008+0000","last_became_peered":"2023-07-18T03:39:57.356008+0000","last_unstale":"2023-07-18T03:39:57.360048+0000","last_undegraded":"2023-07-18T03:39:57.360048+0000","last_fullsized":"2023-07-18T03:39:57.360048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:20:26.761581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354507+0000","last_change":"2023-07-18T03:39:56.352591+0000","last_active":"2023-07-18T03:39:56.354507+0000","last_peered":"2023-07-18T03:39:56.354507+0000","last_clean":"2023-07-18T03:39:56.354507+0000","last_became_active":"2023-07-18T03:39:56.352257+0000","last_became_peered":"2023-07-18T03:39:56.352257+0000","last_unstale":"2023-07-18T03:39:56.354507+0000","last_undegraded":"2023-07-18T03:39:56.354507+0000","last_fullsized":"2023-07-18T03:39:56.354507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:40:18.228711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.357301+0000","last_change":"2023-07-18T03:39:57.354324+0000","last_active":"2023-07-18T03:39:57.357301+0000","last_peered":"2023-07-18T03:39:57.357301+0000","last_clean":"2023-07-18T03:39:57.357301+0000","last_became_active":"2023-07-18T03:39:57.354028+0000","last_became_peered":"2023-07-18T03:39:57.354028+0000","last_unstale":"2023-07-18T03:39:57.357301+0000","last_undegraded":"2023-07-18T03:39:57.357301+0000","last_fullsized":"2023-07-18T03:39:57.357301+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:48:08.031962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354264+0000","last_change":"2023-07-18T03:39:56.353133+0000","last_active":"2023-07-18T03:39:56.354264+0000","last_peered":"2023-07-18T03:39:56.354264+0000","last_clean":"2023-07-18T03:39:56.354264+0000","last_became_active":"2023-07-18T03:39:56.352847+0000","last_became_peered":"2023-07-18T03:39:56.352847+0000","last_unstale":"2023-07-18T03:39:56.354264+0000","last_undegraded":"2023-07-18T03:39:56.354264+0000","last_fullsized":"2023-07-18T03:39:56.354264+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:21:01.203032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354872+0000","last_change":"2023-07-18T03:39:57.350961+0000","last_active":"2023-07-18T03:39:57.354872+0000","last_peered":"2023-07-18T03:39:57.354872+0000","last_clean":"2023-07-18T03:39:57.354872+0000","last_became_active":"2023-07-18T03:39:57.350643+0000","last_became_peered":"2023-07-18T03:39:57.350643+0000","last_unstale":"2023-07-18T03:39:57.354872+0000","last_undegraded":"2023-07-18T03:39:57.354872+0000","last_fullsized":"2023-07-18T03:39:57.354872+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:10:28.224008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.349402+0000","last_change":"2023-07-18T03:39:56.348133+0000","last_active":"2023-07-18T03:39:56.349402+0000","last_peered":"2023-07-18T03:39:56.349402+0000","last_clean":"2023-07-18T03:39:56.349402+0000","last_became_active":"2023-07-18T03:39:56.347930+0000","last_became_peered":"2023-07-18T03:39:56.347930+0000","last_unstale":"2023-07-18T03:39:56.349402+0000","last_undegraded":"2023-07-18T03:39:56.349402+0000","last_fullsized":"2023-07-18T03:39:56.349402+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:25:49.425965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.327915+0000","last_change":"2023-07-18T03:39:56.327915+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.327915+0000","last_undegraded":"2023-07-18T03:39:56.327915+0000","last_fullsized":"2023-07-18T03:39:56.327915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354562+0000","last_change":"2023-07-18T03:39:56.352639+0000","last_active":"2023-07-18T03:39:56.354562+0000","last_peered":"2023-07-18T03:39:56.354562+0000","last_clean":"2023-07-18T03:39:56.354562+0000","last_became_active":"2023-07-18T03:39:56.352134+0000","last_became_peered":"2023-07-18T03:39:56.352134+0000","last_unstale":"2023-07-18T03:39:56.354562+0000","last_undegraded":"2023-07-18T03:39:56.354562+0000","last_fullsized":"2023-07-18T03:39:56.354562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:26:35.806559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.352144+0000","last_change":"2023-07-18T03:39:57.349224+0000","last_active":"2023-07-18T03:39:57.352144+0000","last_peered":"2023-07-18T03:39:57.352144+0000","last_clean":"2023-07-18T03:39:57.352144+0000","last_became_active":"2023-07-18T03:39:57.348944+0000","last_became_peered":"2023-07-18T03:39:57.348944+0000","last_unstale":"2023-07-18T03:39:57.352144+0000","last_undegraded":"2023-07-18T03:39:57.352144+0000","last_fullsized":"2023-07-18T03:39:57.352144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:55:01.968006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354442+0000","last_change":"2023-07-18T03:39:56.352532+0000","last_active":"2023-07-18T03:39:56.354442+0000","last_peered":"2023-07-18T03:39:56.354442+0000","last_clean":"2023-07-18T03:39:56.354442+0000","last_became_active":"2023-07-18T03:39:56.352201+0000","last_became_peered":"2023-07-18T03:39:56.352201+0000","last_unstale":"2023-07-18T03:39:56.354442+0000","last_undegraded":"2023-07-18T03:39:56.354442+0000","last_fullsized":"2023-07-18T03:39:56.354442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:57:40.583880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.337060+0000","last_change":"2023-07-18T03:39:56.337060+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.337060+0000","last_undegraded":"2023-07-18T03:39:56.337060+0000","last_fullsized":"2023-07-18T03:39:56.337060+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.347811+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:56.347811+0000","last_peered":"2023-07-18T03:39:56.347811+0000","last_clean":"2023-07-18T03:39:56.347811+0000","last_became_active":"2023-07-18T03:39:56.345735+0000","last_became_peered":"2023-07-18T03:39:56.345735+0000","last_unstale":"2023-07-18T03:39:56.347811+0000","last_undegraded":"2023-07-18T03:39:56.347811+0000","last_fullsized":"2023-07-18T03:39:56.347811+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:49:19.376544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354809+0000","last_change":"2023-07-18T03:39:57.347143+0000","last_active":"2023-07-18T03:39:57.354809+0000","last_peered":"2023-07-18T03:39:57.354809+0000","last_clean":"2023-07-18T03:39:57.354809+0000","last_became_active":"2023-07-18T03:39:57.346833+0000","last_became_peered":"2023-07-18T03:39:57.346833+0000","last_unstale":"2023-07-18T03:39:57.354809+0000","last_undegraded":"2023-07-18T03:39:57.354809+0000","last_fullsized":"2023-07-18T03:39:57.354809+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:26:07.789214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.339763+0000","last_change":"2023-07-18T03:39:56.339148+0000","last_active":"2023-07-18T03:39:57.339763+0000","last_peered":"2023-07-18T03:39:57.339763+0000","last_clean":"2023-07-18T03:39:57.339763+0000","last_became_active":"2023-07-18T03:39:56.338862+0000","last_became_peered":"2023-07-18T03:39:56.338862+0000","last_unstale":"2023-07-18T03:39:57.339763+0000","last_undegraded":"2023-07-18T03:39:57.339763+0000","last_fullsized":"2023-07-18T03:39:57.339763+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:14:22.006899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.340136+0000","last_change":"2023-07-18T03:39:56.340136+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.340136+0000","last_undegraded":"2023-07-18T03:39:56.340136+0000","last_fullsized":"2023-07-18T03:39:56.340136+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.364107+0000","last_change":"2023-07-18T03:39:56.345149+0000","last_active":"2023-07-18T03:39:57.364107+0000","last_peered":"2023-07-18T03:39:57.364107+0000","last_clean":"2023-07-18T03:39:57.364107+0000","last_became_active":"2023-07-18T03:39:56.344848+0000","last_became_peered":"2023-07-18T03:39:56.344848+0000","last_unstale":"2023-07-18T03:39:57.364107+0000","last_undegraded":"2023-07-18T03:39:57.364107+0000","last_fullsized":"2023-07-18T03:39:57.364107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:51:09.697807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335929+0000","last_change":"2023-07-18T03:39:56.335929+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335929+0000","last_undegraded":"2023-07-18T03:39:56.335929+0000","last_fullsized":"2023-07-18T03:39:56.335929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.349330+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:56.349330+0000","last_peered":"2023-07-18T03:39:56.349330+0000","last_clean":"2023-07-18T03:39:56.349330+0000","last_became_active":"2023-07-18T03:39:56.345797+0000","last_became_peered":"2023-07-18T03:39:56.345797+0000","last_unstale":"2023-07-18T03:39:56.349330+0000","last_undegraded":"2023-07-18T03:39:56.349330+0000","last_fullsized":"2023-07-18T03:39:56.349330+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:12.877614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.339214+0000","last_change":"2023-07-18T03:39:56.339214+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.339214+0000","last_undegraded":"2023-07-18T03:39:56.339214+0000","last_fullsized":"2023-07-18T03:39:56.339214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.350939+0000","last_change":"2023-07-18T03:39:56.349196+0000","last_active":"2023-07-18T03:39:56.350939+0000","last_peered":"2023-07-18T03:39:56.350939+0000","last_clean":"2023-07-18T03:39:56.350939+0000","last_became_active":"2023-07-18T03:39:56.347176+0000","last_became_peered":"2023-07-18T03:39:56.347176+0000","last_unstale":"2023-07-18T03:39:56.350939+0000","last_undegraded":"2023-07-18T03:39:56.350939+0000","last_fullsized":"2023-07-18T03:39:56.350939+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:09.700401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.327716+0000","last_change":"2023-07-18T03:39:56.327716+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.327716+0000","last_undegraded":"2023-07-18T03:39:56.327716+0000","last_fullsized":"2023-07-18T03:39:56.327716+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359507+0000","last_change":"2023-07-18T03:39:56.344722+0000","last_active":"2023-07-18T03:39:57.359507+0000","last_peered":"2023-07-18T03:39:57.359507+0000","last_clean":"2023-07-18T03:39:57.359507+0000","last_became_active":"2023-07-18T03:39:56.344439+0000","last_became_peered":"2023-07-18T03:39:56.344439+0000","last_unstale":"2023-07-18T03:39:57.359507+0000","last_undegraded":"2023-07-18T03:39:57.359507+0000","last_fullsized":"2023-07-18T03:39:57.359507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:47:42.656638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.366921+0000","last_change":"2023-07-18T03:39:57.363607+0000","last_active":"2023-07-18T03:39:57.366921+0000","last_peered":"2023-07-18T03:39:57.366921+0000","last_clean":"2023-07-18T03:39:57.366921+0000","last_became_active":"2023-07-18T03:39:57.363312+0000","last_became_peered":"2023-07-18T03:39:57.363312+0000","last_unstale":"2023-07-18T03:39:57.366921+0000","last_undegraded":"2023-07-18T03:39:57.366921+0000","last_fullsized":"2023-07-18T03:39:57.366921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:56:30.355299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.344690+0000","last_change":"2023-07-18T03:39:56.336359+0000","last_active":"2023-07-18T03:39:56.344690+0000","last_peered":"2023-07-18T03:39:56.344690+0000","last_clean":"2023-07-18T03:39:56.344690+0000","last_became_active":"2023-07-18T03:39:56.336094+0000","last_became_peered":"2023-07-18T03:39:56.336094+0000","last_unstale":"2023-07-18T03:39:56.344690+0000","last_undegraded":"2023-07-18T03:39:56.344690+0000","last_fullsized":"2023-07-18T03:39:56.344690+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:30.478922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355019+0000","last_change":"2023-07-18T03:39:57.348592+0000","last_active":"2023-07-18T03:39:57.355019+0000","last_peered":"2023-07-18T03:39:57.355019+0000","last_clean":"2023-07-18T03:39:57.355019+0000","last_became_active":"2023-07-18T03:39:57.348257+0000","last_became_peered":"2023-07-18T03:39:57.348257+0000","last_unstale":"2023-07-18T03:39:57.355019+0000","last_undegraded":"2023-07-18T03:39:57.355019+0000","last_fullsized":"2023-07-18T03:39:57.355019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:36:37.474323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359688+0000","last_change":"2023-07-18T03:39:56.337863+0000","last_active":"2023-07-18T03:39:57.359688+0000","last_peered":"2023-07-18T03:39:57.359688+0000","last_clean":"2023-07-18T03:39:57.359688+0000","last_became_active":"2023-07-18T03:39:56.337561+0000","last_became_peered":"2023-07-18T03:39:56.337561+0000","last_unstale":"2023-07-18T03:39:57.359688+0000","last_undegraded":"2023-07-18T03:39:57.359688+0000","last_fullsized":"2023-07-18T03:39:57.359688+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:10.926722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.353775+0000","last_change":"2023-07-18T03:39:57.351197+0000","last_active":"2023-07-18T03:39:57.353775+0000","last_peered":"2023-07-18T03:39:57.353775+0000","last_clean":"2023-07-18T03:39:57.353775+0000","last_became_active":"2023-07-18T03:39:57.350814+0000","last_became_peered":"2023-07-18T03:39:57.350814+0000","last_unstale":"2023-07-18T03:39:57.353775+0000","last_undegraded":"2023-07-18T03:39:57.353775+0000","last_fullsized":"2023-07-18T03:39:57.353775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:06:35.294279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.339081+0000","last_change":"2023-07-18T03:39:56.336889+0000","last_active":"2023-07-18T03:39:57.339081+0000","last_peered":"2023-07-18T03:39:57.339081+0000","last_clean":"2023-07-18T03:39:57.339081+0000","last_became_active":"2023-07-18T03:39:56.336599+0000","last_became_peered":"2023-07-18T03:39:56.336599+0000","last_unstale":"2023-07-18T03:39:57.339081+0000","last_undegraded":"2023-07-18T03:39:57.339081+0000","last_fullsized":"2023-07-18T03:39:57.339081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:10.544538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.333821+0000","last_change":"2023-07-18T03:39:56.333821+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.333821+0000","last_undegraded":"2023-07-18T03:39:56.333821+0000","last_fullsized":"2023-07-18T03:39:56.333821+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.331084+0000","last_change":"2023-07-18T03:39:56.350386+0000","last_active":"2023-07-18T03:39:57.331084+0000","last_peered":"2023-07-18T03:39:57.331084+0000","last_clean":"2023-07-18T03:39:57.331084+0000","last_became_active":"2023-07-18T03:39:56.350046+0000","last_became_peered":"2023-07-18T03:39:56.350046+0000","last_unstale":"2023-07-18T03:39:57.331084+0000","last_undegraded":"2023-07-18T03:39:57.331084+0000","last_fullsized":"2023-07-18T03:39:57.331084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:57:37.239962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.339618+0000","last_change":"2023-07-18T03:39:56.339618+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.339618+0000","last_undegraded":"2023-07-18T03:39:56.339618+0000","last_fullsized":"2023-07-18T03:39:56.339618+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.338697+0000","last_change":"2023-07-18T03:39:56.338127+0000","last_active":"2023-07-18T03:39:57.338697+0000","last_peered":"2023-07-18T03:39:57.338697+0000","last_clean":"2023-07-18T03:39:57.338697+0000","last_became_active":"2023-07-18T03:39:56.337747+0000","last_became_peered":"2023-07-18T03:39:56.337747+0000","last_unstale":"2023-07-18T03:39:57.338697+0000","last_undegraded":"2023-07-18T03:39:57.338697+0000","last_fullsized":"2023-07-18T03:39:57.338697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:08:17.902329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328126+0000","last_change":"2023-07-18T03:39:56.328126+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328126+0000","last_undegraded":"2023-07-18T03:39:56.328126+0000","last_fullsized":"2023-07-18T03:39:56.328126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.355935+0000","last_change":"2023-07-18T03:39:56.353087+0000","last_active":"2023-07-18T03:39:56.355935+0000","last_peered":"2023-07-18T03:39:56.355935+0000","last_clean":"2023-07-18T03:39:56.355935+0000","last_became_active":"2023-07-18T03:39:56.352738+0000","last_became_peered":"2023-07-18T03:39:56.352738+0000","last_unstale":"2023-07-18T03:39:56.355935+0000","last_undegraded":"2023-07-18T03:39:56.355935+0000","last_fullsized":"2023-07-18T03:39:56.355935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:38:20.581738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.329523+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:39:56.329523+0000","last_peered":"2023-07-18T03:39:56.329523+0000","last_clean":"2023-07-18T03:39:56.329523+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:39:56.329523+0000","last_undegraded":"2023-07-18T03:39:56.329523+0000","last_fullsized":"2023-07-18T03:39:56.329523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:58:20.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355192+0000","last_change":"2023-07-18T03:39:57.348865+0000","last_active":"2023-07-18T03:39:57.355192+0000","last_peered":"2023-07-18T03:39:57.355192+0000","last_clean":"2023-07-18T03:39:57.355192+0000","last_became_active":"2023-07-18T03:39:57.348246+0000","last_became_peered":"2023-07-18T03:39:57.348246+0000","last_unstale":"2023-07-18T03:39:57.355192+0000","last_undegraded":"2023-07-18T03:39:57.355192+0000","last_fullsized":"2023-07-18T03:39:57.355192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:47:29.972115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354659+0000","last_change":"2023-07-18T03:39:56.352671+0000","last_active":"2023-07-18T03:39:56.354659+0000","last_peered":"2023-07-18T03:39:56.354659+0000","last_clean":"2023-07-18T03:39:56.354659+0000","last_became_active":"2023-07-18T03:39:56.351865+0000","last_became_peered":"2023-07-18T03:39:56.351865+0000","last_unstale":"2023-07-18T03:39:56.354659+0000","last_undegraded":"2023-07-18T03:39:56.354659+0000","last_fullsized":"2023-07-18T03:39:56.354659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:01:05.415848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.342292+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:39:56.342292+0000","last_peered":"2023-07-18T03:39:56.342292+0000","last_clean":"2023-07-18T03:39:56.342292+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:39:56.342292+0000","last_undegraded":"2023-07-18T03:39:56.342292+0000","last_fullsized":"2023-07-18T03:39:56.342292+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:40:43.825533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.329925+0000","last_change":"2023-07-18T03:39:56.329925+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.329925+0000","last_undegraded":"2023-07-18T03:39:56.329925+0000","last_fullsized":"2023-07-18T03:39:56.329925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.349349+0000","last_change":"2023-07-18T03:39:56.347782+0000","last_active":"2023-07-18T03:39:56.349349+0000","last_peered":"2023-07-18T03:39:56.349349+0000","last_clean":"2023-07-18T03:39:56.349349+0000","last_became_active":"2023-07-18T03:39:56.347519+0000","last_became_peered":"2023-07-18T03:39:56.347519+0000","last_unstale":"2023-07-18T03:39:56.349349+0000","last_undegraded":"2023-07-18T03:39:56.349349+0000","last_fullsized":"2023-07-18T03:39:56.349349+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:25.713685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.332914+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:39:56.332914+0000","last_peered":"2023-07-18T03:39:56.332914+0000","last_clean":"2023-07-18T03:39:56.332914+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:39:56.332914+0000","last_undegraded":"2023-07-18T03:39:56.332914+0000","last_fullsized":"2023-07-18T03:39:56.332914+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:40:36.216166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.330424+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:39:57.330424+0000","last_peered":"2023-07-18T03:39:57.330424+0000","last_clean":"2023-07-18T03:39:57.330424+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:39:57.330424+0000","last_undegraded":"2023-07-18T03:39:57.330424+0000","last_fullsized":"2023-07-18T03:39:57.330424+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:22:09.535618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355161+0000","last_change":"2023-07-18T03:39:57.350408+0000","last_active":"2023-07-18T03:39:57.355161+0000","last_peered":"2023-07-18T03:39:57.355161+0000","last_clean":"2023-07-18T03:39:57.355161+0000","last_became_active":"2023-07-18T03:39:57.350103+0000","last_became_peered":"2023-07-18T03:39:57.350103+0000","last_unstale":"2023-07-18T03:39:57.355161+0000","last_undegraded":"2023-07-18T03:39:57.355161+0000","last_fullsized":"2023-07-18T03:39:57.355161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:47:42.227277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.338099+0000","last_change":"2023-07-18T03:39:56.346081+0000","last_active":"2023-07-18T03:39:57.338099+0000","last_peered":"2023-07-18T03:39:57.338099+0000","last_clean":"2023-07-18T03:39:57.338099+0000","last_became_active":"2023-07-18T03:39:56.345762+0000","last_became_peered":"2023-07-18T03:39:56.345762+0000","last_unstale":"2023-07-18T03:39:57.338099+0000","last_undegraded":"2023-07-18T03:39:57.338099+0000","last_fullsized":"2023-07-18T03:39:57.338099+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:15:29.121017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":45,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.329508+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:39:57.329508+0000","last_peered":"2023-07-18T03:39:57.329508+0000","last_clean":"2023-07-18T03:39:57.329508+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:39:57.329508+0000","last_undegraded":"2023-07-18T03:39:57.329508+0000","last_fullsized":"2023-07-18T03:39:57.329508+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:13:09.287323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.336448+0000","last_change":"2023-07-18T03:39:56.336448+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.336448+0000","last_undegraded":"2023-07-18T03:39:56.336448+0000","last_fullsized":"2023-07-18T03:39:56.336448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.330231+0000","last_change":"2023-07-18T03:39:56.348881+0000","last_active":"2023-07-18T03:39:57.330231+0000","last_peered":"2023-07-18T03:39:57.330231+0000","last_clean":"2023-07-18T03:39:57.330231+0000","last_became_active":"2023-07-18T03:39:56.348622+0000","last_became_peered":"2023-07-18T03:39:56.348622+0000","last_unstale":"2023-07-18T03:39:57.330231+0000","last_undegraded":"2023-07-18T03:39:57.330231+0000","last_fullsized":"2023-07-18T03:39:57.330231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:53:21.359259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":45,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.330272+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:39:57.330272+0000","last_peered":"2023-07-18T03:39:57.330272+0000","last_clean":"2023-07-18T03:39:57.330272+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:39:57.330272+0000","last_undegraded":"2023-07-18T03:39:57.330272+0000","last_fullsized":"2023-07-18T03:39:57.330272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:39:12.642880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335532+0000","last_change":"2023-07-18T03:39:56.335532+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335532+0000","last_undegraded":"2023-07-18T03:39:56.335532+0000","last_fullsized":"2023-07-18T03:39:56.335532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.336107+0000","last_change":"2023-07-18T03:39:56.337331+0000","last_active":"2023-07-18T03:39:57.336107+0000","last_peered":"2023-07-18T03:39:57.336107+0000","last_clean":"2023-07-18T03:39:57.336107+0000","last_became_active":"2023-07-18T03:39:56.337021+0000","last_became_peered":"2023-07-18T03:39:56.337021+0000","last_unstale":"2023-07-18T03:39:57.336107+0000","last_undegraded":"2023-07-18T03:39:57.336107+0000","last_fullsized":"2023-07-18T03:39:57.336107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:22:50.209633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.340413+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:39:56.340413+0000","last_peered":"2023-07-18T03:39:56.340413+0000","last_clean":"2023-07-18T03:39:56.340413+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:39:56.340413+0000","last_undegraded":"2023-07-18T03:39:56.340413+0000","last_fullsized":"2023-07-18T03:39:56.340413+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:45:16.320741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.329526+0000","last_change":"2023-07-18T03:39:56.329526+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.329526+0000","last_undegraded":"2023-07-18T03:39:56.329526+0000","last_fullsized":"2023-07-18T03:39:56.329526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.340100+0000","last_change":"2023-07-18T03:39:56.341357+0000","last_active":"2023-07-18T03:39:57.340100+0000","last_peered":"2023-07-18T03:39:57.340100+0000","last_clean":"2023-07-18T03:39:57.340100+0000","last_became_active":"2023-07-18T03:39:56.341092+0000","last_became_peered":"2023-07-18T03:39:56.341092+0000","last_unstale":"2023-07-18T03:39:57.340100+0000","last_undegraded":"2023-07-18T03:39:57.340100+0000","last_fullsized":"2023-07-18T03:39:57.340100+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:48:52.596611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.339973+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:39:57.339973+0000","last_peered":"2023-07-18T03:39:57.339973+0000","last_clean":"2023-07-18T03:39:57.339973+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:39:57.339973+0000","last_undegraded":"2023-07-18T03:39:57.339973+0000","last_fullsized":"2023-07-18T03:39:57.339973+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:48:55.614775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.363216+0000","last_change":"2023-07-18T03:39:57.360948+0000","last_active":"2023-07-18T03:39:57.363216+0000","last_peered":"2023-07-18T03:39:57.363216+0000","last_clean":"2023-07-18T03:39:57.363216+0000","last_became_active":"2023-07-18T03:39:57.360601+0000","last_became_peered":"2023-07-18T03:39:57.360601+0000","last_unstale":"2023-07-18T03:39:57.363216+0000","last_undegraded":"2023-07-18T03:39:57.363216+0000","last_fullsized":"2023-07-18T03:39:57.363216+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:49:44.089477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.336599+0000","last_change":"2023-07-18T03:39:56.342369+0000","last_active":"2023-07-18T03:39:57.336599+0000","last_peered":"2023-07-18T03:39:57.336599+0000","last_clean":"2023-07-18T03:39:57.336599+0000","last_became_active":"2023-07-18T03:39:56.342108+0000","last_became_peered":"2023-07-18T03:39:56.342108+0000","last_unstale":"2023-07-18T03:39:57.336599+0000","last_undegraded":"2023-07-18T03:39:57.336599+0000","last_fullsized":"2023-07-18T03:39:57.336599+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:44.715606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.340446+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:39:57.340446+0000","last_peered":"2023-07-18T03:39:57.340446+0000","last_clean":"2023-07-18T03:39:57.340446+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:39:57.340446+0000","last_undegraded":"2023-07-18T03:39:57.340446+0000","last_fullsized":"2023-07-18T03:39:57.340446+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:54:03.329884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":652,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":999424,"data_stored":752338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":1.7729999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":2.2130000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":3.0030000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.2290000000000001,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":2.2570000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.0249999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":4.0010000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":1.2889999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":2.2989999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":3.2029999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":3.1219999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":1.0609999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":393216,"data_stored":160918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.626}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.72899999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.69299999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67800000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.82499999999999996}]}]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":376832,"data_stored":153208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52600000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.8290000000000002,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.747},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.80400000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":970752,"data_stored":743576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32200000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.495}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79000000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.73899999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":376832,"data_stored":153208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.82399999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.78500000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64700000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.75800000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.68899999999999995}]}]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":376832,"data_stored":153208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":4.9260000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":2.1699999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":8.3780000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":5.5060000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":4.8769999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":2.1230000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":2.2370000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":1.6599999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":4.9690000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":3.6829999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":5.3140000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":3.7389999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.71799999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":393216,"data_stored":160370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":8.3040000000000003,"5min":8.3040000000000003,"15min":8.3040000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.502}]},{"osd":3,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.0229999999999997,"5min":8.0229999999999997,"15min":8.0229999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":8.359,"5min":8.359,"15min":8.359},"last":0.48099999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.89900000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.78300000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.88200000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.97399999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":393216,"data_stored":160406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39100000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.438}]},{"osd":3,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.78100000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.79600000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.60499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:40:02.297 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:40:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.540+0000 7f99b03ff700 1 Processor -- start 2023-07-18T03:40:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.541+0000 7f99b03ff700 1 -- start start 2023-07-18T03:40:02.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.542+0000 7f99b03ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f99a80b4510 0x7f99a80b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.542+0000 7f99b03ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f99a80b5d90 0x7f99a80cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.542+0000 7f99b03ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a80b7f50 con 0x7f99a80d0b10 2023-07-18T03:40:02.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.542+0000 7f99b03ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a80b48e0 con 0x7f99a80b5d90 2023-07-18T03:40:02.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.542+0000 7f99b03ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a80b7340 con 0x7f99a80b4510 2023-07-18T03:40:02.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.542+0000 7f99ad99a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f99a80b5d90 0x7f99a80cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.542+0000 7f99ad99a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f99a80b5d90 0x7f99a80cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38438/0 (socket says 172.21.15.49:38438) 2023-07-18T03:40:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.542+0000 7f99ad99a700 1 -- 172.21.15.49:0/3295441728 learned_addr learned my addr 172.21.15.49:0/3295441728 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.542+0000 7f99ae99c700 1 --1- 172.21.15.49:0/3295441728 >> v1:172.21.15.49:6789/0 conn(0x7f99a80d0b10 0x7f99a80b83e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56838/0 (socket says 172.21.15.49:56838) 2023-07-18T03:40:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.542+0000 7f99ae19b700 1 --2- 172.21.15.49:0/3295441728 >> v2:172.21.15.107:3300/0 conn(0x7f99a80b4510 0x7f99a80b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.543+0000 7f99ad99a700 1 -- 172.21.15.49:0/3295441728 >> v2:172.21.15.107:3300/0 conn(0x7f99a80b4510 msgr2=0x7f99a80b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.543+0000 7f99ad99a700 1 --2- 172.21.15.49:0/3295441728 >> v2:172.21.15.107:3300/0 conn(0x7f99a80b4510 0x7f99a80b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.543+0000 7f99ad99a700 1 -- 172.21.15.49:0/3295441728 >> v1:172.21.15.49:6789/0 conn(0x7f99a80d0b10 legacy=0x7f99a80b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.543+0000 7f99ad99a700 1 -- 172.21.15.49:0/3295441728 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a80b7620 con 0x7f99a80b5d90 2023-07-18T03:40:02.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.543+0000 7f99ae19b700 1 --2- 172.21.15.49:0/3295441728 >> v2:172.21.15.107:3300/0 conn(0x7f99a80b4510 0x7f99a80b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:02.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.543+0000 7f99ad199700 1 -- 172.21.15.49:0/3295441728 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739820236 0 0) 0x7f99a80b7f50 con 0x7f99a80d0b10 2023-07-18T03:40:02.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.543+0000 7f99ad99a700 1 --2- 172.21.15.49:0/3295441728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a80b5d90 0x7f99a80cd590 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f99a808bd80 tx=0x7f9998009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac06a0a79e22e490 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:02.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.543+0000 7f99ad199700 1 -- 172.21.15.49:0/3295441728 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999800b870 con 0x7f99a80b5d90 2023-07-18T03:40:02.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.543+0000 7f99ad199700 1 -- 172.21.15.49:0/3295441728 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999800b9d0 con 0x7f99a80b5d90 2023-07-18T03:40:02.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.543+0000 7f99ad199700 1 -- 172.21.15.49:0/3295441728 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999800bd60 con 0x7f99a80b5d90 2023-07-18T03:40:02.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.544+0000 7f99b03ff700 1 -- 172.21.15.49:0/3295441728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a80b5d90 msgr2=0x7f99a80cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.544+0000 7f99b03ff700 1 --2- 172.21.15.49:0/3295441728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a80b5d90 0x7f99a80cd590 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f99a808bd80 tx=0x7f9998009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.544+0000 7f99b03ff700 1 -- 172.21.15.49:0/3295441728 shutdown_connections 2023-07-18T03:40:02.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.544+0000 7f99b03ff700 1 --2- 172.21.15.49:0/3295441728 >> v2:172.21.15.107:3300/0 conn(0x7f99a80b4510 0x7f99a80b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.544+0000 7f99b03ff700 1 --2- 172.21.15.49:0/3295441728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a80b5d90 0x7f99a80cd590 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.544+0000 7f99b03ff700 1 -- 172.21.15.49:0/3295441728 >> 172.21.15.49:0/3295441728 conn(0x7f99a809c1d0 msgr2=0x7f99a80d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:02.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.544+0000 7f99b03ff700 1 -- 172.21.15.49:0/3295441728 shutdown_connections 2023-07-18T03:40:02.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.544+0000 7f99b03ff700 1 -- 172.21.15.49:0/3295441728 wait complete. 2023-07-18T03:40:02.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.546+0000 7f99b03ff700 1 Processor -- start 2023-07-18T03:40:02.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.547+0000 7f99b03ff700 1 -- start start 2023-07-18T03:40:02.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.548+0000 7f99b03ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a80b4510 0x7f99a809f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.548+0000 7f99b03ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99a80b5d90 0x7f99a8156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.548+0000 7f99b03ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a80d0b10 0x7f99a81f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.548+0000 7f99b03ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8156020 con 0x7f99a80d0b10 2023-07-18T03:40:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.548+0000 7f99b03ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a81f2dc0 con 0x7f99a80b4510 2023-07-18T03:40:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.548+0000 7f99b03ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a81f30a0 con 0x7f99a80b5d90 2023-07-18T03:40:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.548+0000 7f99ae99c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a80d0b10 0x7f99a81f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.548+0000 7f99ae99c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a80d0b10 0x7f99a81f2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38450/0 (socket says 172.21.15.49:38450) 2023-07-18T03:40:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.548+0000 7f99ae99c700 1 -- 172.21.15.49:0/2944543241 learned_addr learned my addr 172.21.15.49:0/2944543241 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.548+0000 7f99ad99a700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99a80b5d90 0x7f99a8156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.548+0000 7f99ae19b700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a80b4510 0x7f99a809f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.549+0000 7f99ae99c700 1 -- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99a80b5d90 msgr2=0x7f99a8156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.549+0000 7f99ae99c700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99a80b5d90 0x7f99a8156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.549+0000 7f99ae99c700 1 -- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a80b4510 msgr2=0x7f99a809f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.549+0000 7f99ae99c700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a80b4510 0x7f99a809f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.549+0000 7f99ae99c700 1 -- 172.21.15.49:0/2944543241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9998009550 con 0x7f99a80d0b10 2023-07-18T03:40:02.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.549+0000 7f99ae19b700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a80b4510 0x7f99a809f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:02.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.549+0000 7f99ae99c700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a80d0b10 0x7f99a81f2310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f99a0002230 tx=0x7f99a000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:02.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.549+0000 7f999effd700 1 -- 172.21.15.49:0/2944543241 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a000ab70 con 0x7f99a80d0b10 2023-07-18T03:40:02.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.549+0000 7f999effd700 1 -- 172.21.15.49:0/2944543241 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a000acf0 con 0x7f99a80d0b10 2023-07-18T03:40:02.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.549+0000 7f999effd700 1 -- 172.21.15.49:0/2944543241 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0007310 con 0x7f99a80d0b10 2023-07-18T03:40:02.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.549+0000 7f99b03ff700 1 -- 172.21.15.49:0/2944543241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a8156270 con 0x7f99a80d0b10 2023-07-18T03:40:02.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.549+0000 7f99b03ff700 1 -- 172.21.15.49:0/2944543241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a8157f60 con 0x7f99a80d0b10 2023-07-18T03:40:02.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.550+0000 7f99b03ff700 1 -- 172.21.15.49:0/2944543241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a8156790 con 0x7f99a80d0b10 2023-07-18T03:40:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.551+0000 7f999effd700 1 -- 172.21.15.49:0/2944543241 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f99a0007470 con 0x7f99a80d0b10 2023-07-18T03:40:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.553+0000 7f999effd700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f998c06ace0 0x7f998c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.553+0000 7f999effd700 1 -- 172.21.15.49:0/2944543241 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99a0088ff0 con 0x7f99a80d0b10 2023-07-18T03:40:02.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.553+0000 7f99ae19b700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f998c06ace0 0x7f998c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.554+0000 7f99ae19b700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f998c06ace0 0x7f998c06d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f99a81f5850 tx=0x7f99a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:02.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.558+0000 7f999effd700 1 -- 172.21.15.49:0/2944543241 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f99a8156790 con 0x7f99a80d0b10 2023-07-18T03:40:02.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.761+0000 7f99b03ff700 1 -- 172.21.15.49:0/2944543241 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99a80b7be0 con 0x7f998c06ace0 2023-07-18T03:40:02.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.771+0000 7f999effd700 1 -- 172.21.15.49:0/2944543241 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309001 (secure 0 0 0) 0x7f99a400a700 con 0x7f998c06ace0 2023-07-18T03:40:02.769 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:02.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.775+0000 7f99b03ff700 1 -- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f998c06ace0 msgr2=0x7f998c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.775+0000 7f99b03ff700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f998c06ace0 0x7f998c06d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f99a81f5850 tx=0x7f99a4008040 comp rx=0 tx=0).stop 2023-07-18T03:40:02.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.775+0000 7f99b03ff700 1 -- 172.21.15.49:0/2944543241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a80d0b10 msgr2=0x7f99a81f2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.775+0000 7f99b03ff700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a80d0b10 0x7f99a81f2310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f99a0002230 tx=0x7f99a000cf50 comp rx=0 tx=0).stop 2023-07-18T03:40:02.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.775+0000 7f99b03ff700 1 -- 172.21.15.49:0/2944543241 shutdown_connections 2023-07-18T03:40:02.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.775+0000 7f99b03ff700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f998c06ace0 0x7f998c06d190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.775+0000 7f99b03ff700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99a80d0b10 0x7f99a81f2310 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.775+0000 7f99b03ff700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a80b4510 0x7f99a809f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.775+0000 7f99b03ff700 1 --2- 172.21.15.49:0/2944543241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99a80b5d90 0x7f99a8156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.775+0000 7f99b03ff700 1 -- 172.21.15.49:0/2944543241 >> 172.21.15.49:0/2944543241 conn(0x7f99a809c1d0 msgr2=0x7f99a80d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.776+0000 7f99b03ff700 1 -- 172.21.15.49:0/2944543241 shutdown_connections 2023-07-18T03:40:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:02.776+0000 7f99b03ff700 1 -- 172.21.15.49:0/2944543241 wait complete. 2023-07-18T03:40:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:40:02.789 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-07-18T03:40:02.117452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591020,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":177,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236180,"kb_used_data":4180,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738540,"statfs":{"total":773094113280,"available":772852264960,"internally_reserved":0,"allocated":4280320,"data_stored":2437232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":652,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001415"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.337498+0000","last_change":"2023-07-18T03:39:56.337498+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.337498+0000","last_undegraded":"2023-07-18T03:39:56.337498+0000","last_fullsized":"2023-07-18T03:39:56.337498+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335988+0000","last_change":"2023-07-18T03:39:56.335988+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335988+0000","last_undegraded":"2023-07-18T03:39:56.335988+0000","last_fullsized":"2023-07-18T03:39:56.335988+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.352684+0000","last_change":"2023-07-18T03:39:57.349470+0000","last_active":"2023-07-18T03:39:57.352684+0000","last_peered":"2023-07-18T03:39:57.352684+0000","last_clean":"2023-07-18T03:39:57.352684+0000","last_became_active":"2023-07-18T03:39:57.349188+0000","last_became_peered":"2023-07-18T03:39:57.349188+0000","last_unstale":"2023-07-18T03:39:57.352684+0000","last_undegraded":"2023-07-18T03:39:57.352684+0000","last_fullsized":"2023-07-18T03:39:57.352684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:30:54.522092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.360068+0000","last_change":"2023-07-18T03:39:57.356421+0000","last_active":"2023-07-18T03:39:57.360068+0000","last_peered":"2023-07-18T03:39:57.360068+0000","last_clean":"2023-07-18T03:39:57.360068+0000","last_became_active":"2023-07-18T03:39:57.356128+0000","last_became_peered":"2023-07-18T03:39:57.356128+0000","last_unstale":"2023-07-18T03:39:57.360068+0000","last_undegraded":"2023-07-18T03:39:57.360068+0000","last_fullsized":"2023-07-18T03:39:57.360068+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:32:15.921500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359782+0000","last_change":"2023-07-18T03:39:57.356364+0000","last_active":"2023-07-18T03:39:57.359782+0000","last_peered":"2023-07-18T03:39:57.359782+0000","last_clean":"2023-07-18T03:39:57.359782+0000","last_became_active":"2023-07-18T03:39:57.356079+0000","last_became_peered":"2023-07-18T03:39:57.356079+0000","last_unstale":"2023-07-18T03:39:57.359782+0000","last_undegraded":"2023-07-18T03:39:57.359782+0000","last_fullsized":"2023-07-18T03:39:57.359782+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:08:41.904219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328919+0000","last_change":"2023-07-18T03:39:56.328919+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328919+0000","last_undegraded":"2023-07-18T03:39:56.328919+0000","last_fullsized":"2023-07-18T03:39:56.328919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332077+0000","last_change":"2023-07-18T03:39:56.332077+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332077+0000","last_undegraded":"2023-07-18T03:39:56.332077+0000","last_fullsized":"2023-07-18T03:39:56.332077+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335775+0000","last_change":"2023-07-18T03:39:56.335775+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335775+0000","last_undegraded":"2023-07-18T03:39:56.335775+0000","last_fullsized":"2023-07-18T03:39:56.335775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.357950+0000","last_change":"2023-07-18T03:39:57.352415+0000","last_active":"2023-07-18T03:39:57.357950+0000","last_peered":"2023-07-18T03:39:57.357950+0000","last_clean":"2023-07-18T03:39:57.357950+0000","last_became_active":"2023-07-18T03:39:57.352137+0000","last_became_peered":"2023-07-18T03:39:57.352137+0000","last_unstale":"2023-07-18T03:39:57.357950+0000","last_undegraded":"2023-07-18T03:39:57.357950+0000","last_fullsized":"2023-07-18T03:39:57.357950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:36.387384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.338378+0000","last_change":"2023-07-18T03:39:56.338378+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.338378+0000","last_undegraded":"2023-07-18T03:39:56.338378+0000","last_fullsized":"2023-07-18T03:39:56.338378+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.333492+0000","last_change":"2023-07-18T03:39:56.333492+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.333492+0000","last_undegraded":"2023-07-18T03:39:56.333492+0000","last_fullsized":"2023-07-18T03:39:56.333492+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.351146+0000","last_change":"2023-07-18T03:39:57.345772+0000","last_active":"2023-07-18T03:39:57.351146+0000","last_peered":"2023-07-18T03:39:57.351146+0000","last_clean":"2023-07-18T03:39:57.351146+0000","last_became_active":"2023-07-18T03:39:57.345465+0000","last_became_peered":"2023-07-18T03:39:57.345465+0000","last_unstale":"2023-07-18T03:39:57.351146+0000","last_undegraded":"2023-07-18T03:39:57.351146+0000","last_fullsized":"2023-07-18T03:39:57.351146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:48:30.748069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332641+0000","last_change":"2023-07-18T03:39:56.332641+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332641+0000","last_undegraded":"2023-07-18T03:39:56.332641+0000","last_fullsized":"2023-07-18T03:39:56.332641+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.327178+0000","last_change":"2023-07-18T03:39:56.327178+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.327178+0000","last_undegraded":"2023-07-18T03:39:56.327178+0000","last_fullsized":"2023-07-18T03:39:56.327178+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332947+0000","last_change":"2023-07-18T03:39:56.332947+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332947+0000","last_undegraded":"2023-07-18T03:39:56.332947+0000","last_fullsized":"2023-07-18T03:39:56.332947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.348720+0000","last_change":"2023-07-18T03:39:57.337636+0000","last_active":"2023-07-18T03:39:57.348720+0000","last_peered":"2023-07-18T03:39:57.348720+0000","last_clean":"2023-07-18T03:39:57.348720+0000","last_became_active":"2023-07-18T03:39:57.337330+0000","last_became_peered":"2023-07-18T03:39:57.337330+0000","last_unstale":"2023-07-18T03:39:57.348720+0000","last_undegraded":"2023-07-18T03:39:57.348720+0000","last_fullsized":"2023-07-18T03:39:57.348720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:22:11.047392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354699+0000","last_change":"2023-07-18T03:39:57.345108+0000","last_active":"2023-07-18T03:39:57.354699+0000","last_peered":"2023-07-18T03:39:57.354699+0000","last_clean":"2023-07-18T03:39:57.354699+0000","last_became_active":"2023-07-18T03:39:57.344825+0000","last_became_peered":"2023-07-18T03:39:57.344825+0000","last_unstale":"2023-07-18T03:39:57.354699+0000","last_undegraded":"2023-07-18T03:39:57.354699+0000","last_fullsized":"2023-07-18T03:39:57.354699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:07:59.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.365129+0000","last_change":"2023-07-18T03:39:57.361051+0000","last_active":"2023-07-18T03:39:57.365129+0000","last_peered":"2023-07-18T03:39:57.365129+0000","last_clean":"2023-07-18T03:39:57.365129+0000","last_became_active":"2023-07-18T03:39:57.360746+0000","last_became_peered":"2023-07-18T03:39:57.360746+0000","last_unstale":"2023-07-18T03:39:57.365129+0000","last_undegraded":"2023-07-18T03:39:57.365129+0000","last_fullsized":"2023-07-18T03:39:57.365129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:11:30.468969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359427+0000","last_change":"2023-07-18T03:39:57.351301+0000","last_active":"2023-07-18T03:39:57.359427+0000","last_peered":"2023-07-18T03:39:57.359427+0000","last_clean":"2023-07-18T03:39:57.359427+0000","last_became_active":"2023-07-18T03:39:57.351033+0000","last_became_peered":"2023-07-18T03:39:57.351033+0000","last_unstale":"2023-07-18T03:39:57.359427+0000","last_undegraded":"2023-07-18T03:39:57.359427+0000","last_fullsized":"2023-07-18T03:39:57.359427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:50:08.481909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.365031+0000","last_change":"2023-07-18T03:39:57.360471+0000","last_active":"2023-07-18T03:39:57.365031+0000","last_peered":"2023-07-18T03:39:57.365031+0000","last_clean":"2023-07-18T03:39:57.365031+0000","last_became_active":"2023-07-18T03:39:57.360202+0000","last_became_peered":"2023-07-18T03:39:57.360202+0000","last_unstale":"2023-07-18T03:39:57.365031+0000","last_undegraded":"2023-07-18T03:39:57.365031+0000","last_fullsized":"2023-07-18T03:39:57.365031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:32.300586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.331681+0000","last_change":"2023-07-18T03:39:56.331681+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.331681+0000","last_undegraded":"2023-07-18T03:39:56.331681+0000","last_fullsized":"2023-07-18T03:39:56.331681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.329011+0000","last_change":"2023-07-18T03:39:56.329011+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.329011+0000","last_undegraded":"2023-07-18T03:39:56.329011+0000","last_fullsized":"2023-07-18T03:39:56.329011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354754+0000","last_change":"2023-07-18T03:39:57.347725+0000","last_active":"2023-07-18T03:39:57.354754+0000","last_peered":"2023-07-18T03:39:57.354754+0000","last_clean":"2023-07-18T03:39:57.354754+0000","last_became_active":"2023-07-18T03:39:57.346990+0000","last_became_peered":"2023-07-18T03:39:57.346990+0000","last_unstale":"2023-07-18T03:39:57.354754+0000","last_undegraded":"2023-07-18T03:39:57.354754+0000","last_fullsized":"2023-07-18T03:39:57.354754+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:33:51.412210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.361028+0000","last_change":"2023-07-18T03:39:57.355821+0000","last_active":"2023-07-18T03:39:57.361028+0000","last_peered":"2023-07-18T03:39:57.361028+0000","last_clean":"2023-07-18T03:39:57.361028+0000","last_became_active":"2023-07-18T03:39:57.355505+0000","last_became_peered":"2023-07-18T03:39:57.355505+0000","last_unstale":"2023-07-18T03:39:57.361028+0000","last_undegraded":"2023-07-18T03:39:57.361028+0000","last_fullsized":"2023-07-18T03:39:57.361028+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:11:11.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.358089+0000","last_change":"2023-07-18T03:39:57.347320+0000","last_active":"2023-07-18T03:39:57.358089+0000","last_peered":"2023-07-18T03:39:57.358089+0000","last_clean":"2023-07-18T03:39:57.358089+0000","last_became_active":"2023-07-18T03:39:57.347051+0000","last_became_peered":"2023-07-18T03:39:57.347051+0000","last_unstale":"2023-07-18T03:39:57.358089+0000","last_undegraded":"2023-07-18T03:39:57.358089+0000","last_fullsized":"2023-07-18T03:39:57.358089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:45:31.179450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328947+0000","last_change":"2023-07-18T03:39:56.328947+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328947+0000","last_undegraded":"2023-07-18T03:39:56.328947+0000","last_fullsized":"2023-07-18T03:39:56.328947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.362856+0000","last_change":"2023-07-18T03:39:57.349732+0000","last_active":"2023-07-18T03:39:57.362856+0000","last_peered":"2023-07-18T03:39:57.362856+0000","last_clean":"2023-07-18T03:39:57.362856+0000","last_became_active":"2023-07-18T03:39:57.349421+0000","last_became_peered":"2023-07-18T03:39:57.349421+0000","last_unstale":"2023-07-18T03:39:57.362856+0000","last_undegraded":"2023-07-18T03:39:57.362856+0000","last_fullsized":"2023-07-18T03:39:57.362856+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:03:41.731722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.337384+0000","last_change":"2023-07-18T03:39:56.337384+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.337384+0000","last_undegraded":"2023-07-18T03:39:56.337384+0000","last_fullsized":"2023-07-18T03:39:56.337384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.362756+0000","last_change":"2023-07-18T03:39:57.354325+0000","last_active":"2023-07-18T03:39:57.362756+0000","last_peered":"2023-07-18T03:39:57.362756+0000","last_clean":"2023-07-18T03:39:57.362756+0000","last_became_active":"2023-07-18T03:39:57.353910+0000","last_became_peered":"2023-07-18T03:39:57.353910+0000","last_unstale":"2023-07-18T03:39:57.362756+0000","last_undegraded":"2023-07-18T03:39:57.362756+0000","last_fullsized":"2023-07-18T03:39:57.362756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:18:46.390308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.364995+0000","last_change":"2023-07-18T03:39:57.362161+0000","last_active":"2023-07-18T03:39:57.364995+0000","last_peered":"2023-07-18T03:39:57.364995+0000","last_clean":"2023-07-18T03:39:57.364995+0000","last_became_active":"2023-07-18T03:39:57.361882+0000","last_became_peered":"2023-07-18T03:39:57.361882+0000","last_unstale":"2023-07-18T03:39:57.364995+0000","last_undegraded":"2023-07-18T03:39:57.364995+0000","last_fullsized":"2023-07-18T03:39:57.364995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:39:10.257984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354777+0000","last_change":"2023-07-18T03:39:57.347923+0000","last_active":"2023-07-18T03:39:57.354777+0000","last_peered":"2023-07-18T03:39:57.354777+0000","last_clean":"2023-07-18T03:39:57.354777+0000","last_became_active":"2023-07-18T03:39:57.347419+0000","last_became_peered":"2023-07-18T03:39:57.347419+0000","last_unstale":"2023-07-18T03:39:57.354777+0000","last_undegraded":"2023-07-18T03:39:57.354777+0000","last_fullsized":"2023-07-18T03:39:57.354777+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:54:59.548413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328673+0000","last_change":"2023-07-18T03:39:56.328673+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328673+0000","last_undegraded":"2023-07-18T03:39:56.328673+0000","last_fullsized":"2023-07-18T03:39:56.328673+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.348817+0000","last_change":"2023-07-18T03:39:57.338235+0000","last_active":"2023-07-18T03:39:57.348817+0000","last_peered":"2023-07-18T03:39:57.348817+0000","last_clean":"2023-07-18T03:39:57.348817+0000","last_became_active":"2023-07-18T03:39:57.337940+0000","last_became_peered":"2023-07-18T03:39:57.337940+0000","last_unstale":"2023-07-18T03:39:57.348817+0000","last_undegraded":"2023-07-18T03:39:57.348817+0000","last_fullsized":"2023-07-18T03:39:57.348817+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:42:25.160054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.365698+0000","last_change":"2023-07-18T03:39:56.337657+0000","last_active":"2023-07-18T03:39:57.365698+0000","last_peered":"2023-07-18T03:39:57.365698+0000","last_clean":"2023-07-18T03:39:57.365698+0000","last_became_active":"2023-07-18T03:39:56.337379+0000","last_became_peered":"2023-07-18T03:39:56.337379+0000","last_unstale":"2023-07-18T03:39:57.365698+0000","last_undegraded":"2023-07-18T03:39:57.365698+0000","last_fullsized":"2023-07-18T03:39:57.365698+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:30.478922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.330756+0000","last_change":"2023-07-18T03:39:56.330756+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.330756+0000","last_undegraded":"2023-07-18T03:39:56.330756+0000","last_fullsized":"2023-07-18T03:39:56.330756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355305+0000","last_change":"2023-07-18T03:39:56.345284+0000","last_active":"2023-07-18T03:39:57.355305+0000","last_peered":"2023-07-18T03:39:57.355305+0000","last_clean":"2023-07-18T03:39:57.355305+0000","last_became_active":"2023-07-18T03:39:56.344955+0000","last_became_peered":"2023-07-18T03:39:56.344955+0000","last_unstale":"2023-07-18T03:39:57.355305+0000","last_undegraded":"2023-07-18T03:39:57.355305+0000","last_fullsized":"2023-07-18T03:39:57.355305+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:50:11.765103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328243+0000","last_change":"2023-07-18T03:39:56.328243+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328243+0000","last_undegraded":"2023-07-18T03:39:56.328243+0000","last_fullsized":"2023-07-18T03:39:56.328243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.345414+0000","last_change":"2023-07-18T03:39:56.335839+0000","last_active":"2023-07-18T03:39:56.345414+0000","last_peered":"2023-07-18T03:39:56.345414+0000","last_clean":"2023-07-18T03:39:56.345414+0000","last_became_active":"2023-07-18T03:39:56.335506+0000","last_became_peered":"2023-07-18T03:39:56.335506+0000","last_unstale":"2023-07-18T03:39:56.345414+0000","last_undegraded":"2023-07-18T03:39:56.345414+0000","last_fullsized":"2023-07-18T03:39:56.345414+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:55:30.319672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.338278+0000","last_change":"2023-07-18T03:39:56.338278+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.338278+0000","last_undegraded":"2023-07-18T03:39:56.338278+0000","last_fullsized":"2023-07-18T03:39:56.338278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.355959+0000","last_change":"2023-07-18T03:39:56.353462+0000","last_active":"2023-07-18T03:39:56.355959+0000","last_peered":"2023-07-18T03:39:56.355959+0000","last_clean":"2023-07-18T03:39:56.355959+0000","last_became_active":"2023-07-18T03:39:56.353165+0000","last_became_peered":"2023-07-18T03:39:56.353165+0000","last_unstale":"2023-07-18T03:39:56.355959+0000","last_undegraded":"2023-07-18T03:39:56.355959+0000","last_fullsized":"2023-07-18T03:39:56.355959+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:24.138081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.350712+0000","last_change":"2023-07-18T03:39:57.345776+0000","last_active":"2023-07-18T03:39:57.350712+0000","last_peered":"2023-07-18T03:39:57.350712+0000","last_clean":"2023-07-18T03:39:57.350712+0000","last_became_active":"2023-07-18T03:39:57.345450+0000","last_became_peered":"2023-07-18T03:39:57.345450+0000","last_unstale":"2023-07-18T03:39:57.350712+0000","last_undegraded":"2023-07-18T03:39:57.350712+0000","last_fullsized":"2023-07-18T03:39:57.350712+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:39:58.863405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.348456+0000","last_change":"2023-07-18T03:39:56.343054+0000","last_active":"2023-07-18T03:39:56.348456+0000","last_peered":"2023-07-18T03:39:56.348456+0000","last_clean":"2023-07-18T03:39:56.348456+0000","last_became_active":"2023-07-18T03:39:56.342752+0000","last_became_peered":"2023-07-18T03:39:56.342752+0000","last_unstale":"2023-07-18T03:39:56.348456+0000","last_undegraded":"2023-07-18T03:39:56.348456+0000","last_fullsized":"2023-07-18T03:39:56.348456+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:42:03.205955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332552+0000","last_change":"2023-07-18T03:39:56.332552+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332552+0000","last_undegraded":"2023-07-18T03:39:56.332552+0000","last_fullsized":"2023-07-18T03:39:56.332552+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.356389+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:57.356389+0000","last_peered":"2023-07-18T03:39:57.356389+0000","last_clean":"2023-07-18T03:39:57.356389+0000","last_became_active":"2023-07-18T03:39:56.345390+0000","last_became_peered":"2023-07-18T03:39:56.345390+0000","last_unstale":"2023-07-18T03:39:57.356389+0000","last_undegraded":"2023-07-18T03:39:57.356389+0000","last_fullsized":"2023-07-18T03:39:57.356389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:57:57.797956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328578+0000","last_change":"2023-07-18T03:39:56.328578+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328578+0000","last_undegraded":"2023-07-18T03:39:56.328578+0000","last_fullsized":"2023-07-18T03:39:56.328578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.347760+0000","last_change":"2023-07-18T03:39:56.346375+0000","last_active":"2023-07-18T03:39:56.347760+0000","last_peered":"2023-07-18T03:39:56.347760+0000","last_clean":"2023-07-18T03:39:56.347760+0000","last_became_active":"2023-07-18T03:39:56.346098+0000","last_became_peered":"2023-07-18T03:39:56.346098+0000","last_unstale":"2023-07-18T03:39:56.347760+0000","last_undegraded":"2023-07-18T03:39:56.347760+0000","last_fullsized":"2023-07-18T03:39:56.347760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:50:22.604564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.339195+0000","last_change":"2023-07-18T03:39:56.339195+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.339195+0000","last_undegraded":"2023-07-18T03:39:56.339195+0000","last_fullsized":"2023-07-18T03:39:56.339195+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.341115+0000","last_change":"2023-07-18T03:39:56.337739+0000","last_active":"2023-07-18T03:39:56.341115+0000","last_peered":"2023-07-18T03:39:56.341115+0000","last_clean":"2023-07-18T03:39:56.341115+0000","last_became_active":"2023-07-18T03:39:56.337495+0000","last_became_peered":"2023-07-18T03:39:56.337495+0000","last_unstale":"2023-07-18T03:39:56.341115+0000","last_undegraded":"2023-07-18T03:39:56.341115+0000","last_fullsized":"2023-07-18T03:39:56.341115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:01.077387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.360048+0000","last_change":"2023-07-18T03:39:57.356339+0000","last_active":"2023-07-18T03:39:57.360048+0000","last_peered":"2023-07-18T03:39:57.360048+0000","last_clean":"2023-07-18T03:39:57.360048+0000","last_became_active":"2023-07-18T03:39:57.356008+0000","last_became_peered":"2023-07-18T03:39:57.356008+0000","last_unstale":"2023-07-18T03:39:57.360048+0000","last_undegraded":"2023-07-18T03:39:57.360048+0000","last_fullsized":"2023-07-18T03:39:57.360048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:20:26.761581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354507+0000","last_change":"2023-07-18T03:39:56.352591+0000","last_active":"2023-07-18T03:39:56.354507+0000","last_peered":"2023-07-18T03:39:56.354507+0000","last_clean":"2023-07-18T03:39:56.354507+0000","last_became_active":"2023-07-18T03:39:56.352257+0000","last_became_peered":"2023-07-18T03:39:56.352257+0000","last_unstale":"2023-07-18T03:39:56.354507+0000","last_undegraded":"2023-07-18T03:39:56.354507+0000","last_fullsized":"2023-07-18T03:39:56.354507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:40:18.228711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.357301+0000","last_change":"2023-07-18T03:39:57.354324+0000","last_active":"2023-07-18T03:39:57.357301+0000","last_peered":"2023-07-18T03:39:57.357301+0000","last_clean":"2023-07-18T03:39:57.357301+0000","last_became_active":"2023-07-18T03:39:57.354028+0000","last_became_peered":"2023-07-18T03:39:57.354028+0000","last_unstale":"2023-07-18T03:39:57.357301+0000","last_undegraded":"2023-07-18T03:39:57.357301+0000","last_fullsized":"2023-07-18T03:39:57.357301+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:48:08.031962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354264+0000","last_change":"2023-07-18T03:39:56.353133+0000","last_active":"2023-07-18T03:39:56.354264+0000","last_peered":"2023-07-18T03:39:56.354264+0000","last_clean":"2023-07-18T03:39:56.354264+0000","last_became_active":"2023-07-18T03:39:56.352847+0000","last_became_peered":"2023-07-18T03:39:56.352847+0000","last_unstale":"2023-07-18T03:39:56.354264+0000","last_undegraded":"2023-07-18T03:39:56.354264+0000","last_fullsized":"2023-07-18T03:39:56.354264+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:21:01.203032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354872+0000","last_change":"2023-07-18T03:39:57.350961+0000","last_active":"2023-07-18T03:39:57.354872+0000","last_peered":"2023-07-18T03:39:57.354872+0000","last_clean":"2023-07-18T03:39:57.354872+0000","last_became_active":"2023-07-18T03:39:57.350643+0000","last_became_peered":"2023-07-18T03:39:57.350643+0000","last_unstale":"2023-07-18T03:39:57.354872+0000","last_undegraded":"2023-07-18T03:39:57.354872+0000","last_fullsized":"2023-07-18T03:39:57.354872+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:10:28.224008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.349402+0000","last_change":"2023-07-18T03:39:56.348133+0000","last_active":"2023-07-18T03:39:56.349402+0000","last_peered":"2023-07-18T03:39:56.349402+0000","last_clean":"2023-07-18T03:39:56.349402+0000","last_became_active":"2023-07-18T03:39:56.347930+0000","last_became_peered":"2023-07-18T03:39:56.347930+0000","last_unstale":"2023-07-18T03:39:56.349402+0000","last_undegraded":"2023-07-18T03:39:56.349402+0000","last_fullsized":"2023-07-18T03:39:56.349402+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:25:49.425965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.327915+0000","last_change":"2023-07-18T03:39:56.327915+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.327915+0000","last_undegraded":"2023-07-18T03:39:56.327915+0000","last_fullsized":"2023-07-18T03:39:56.327915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354562+0000","last_change":"2023-07-18T03:39:56.352639+0000","last_active":"2023-07-18T03:39:56.354562+0000","last_peered":"2023-07-18T03:39:56.354562+0000","last_clean":"2023-07-18T03:39:56.354562+0000","last_became_active":"2023-07-18T03:39:56.352134+0000","last_became_peered":"2023-07-18T03:39:56.352134+0000","last_unstale":"2023-07-18T03:39:56.354562+0000","last_undegraded":"2023-07-18T03:39:56.354562+0000","last_fullsized":"2023-07-18T03:39:56.354562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:26:35.806559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.352144+0000","last_change":"2023-07-18T03:39:57.349224+0000","last_active":"2023-07-18T03:39:57.352144+0000","last_peered":"2023-07-18T03:39:57.352144+0000","last_clean":"2023-07-18T03:39:57.352144+0000","last_became_active":"2023-07-18T03:39:57.348944+0000","last_became_peered":"2023-07-18T03:39:57.348944+0000","last_unstale":"2023-07-18T03:39:57.352144+0000","last_undegraded":"2023-07-18T03:39:57.352144+0000","last_fullsized":"2023-07-18T03:39:57.352144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:55:01.968006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354442+0000","last_change":"2023-07-18T03:39:56.352532+0000","last_active":"2023-07-18T03:39:56.354442+0000","last_peered":"2023-07-18T03:39:56.354442+0000","last_clean":"2023-07-18T03:39:56.354442+0000","last_became_active":"2023-07-18T03:39:56.352201+0000","last_became_peered":"2023-07-18T03:39:56.352201+0000","last_unstale":"2023-07-18T03:39:56.354442+0000","last_undegraded":"2023-07-18T03:39:56.354442+0000","last_fullsized":"2023-07-18T03:39:56.354442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:57:40.583880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.337060+0000","last_change":"2023-07-18T03:39:56.337060+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.337060+0000","last_undegraded":"2023-07-18T03:39:56.337060+0000","last_fullsized":"2023-07-18T03:39:56.337060+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.347811+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:56.347811+0000","last_peered":"2023-07-18T03:39:56.347811+0000","last_clean":"2023-07-18T03:39:56.347811+0000","last_became_active":"2023-07-18T03:39:56.345735+0000","last_became_peered":"2023-07-18T03:39:56.345735+0000","last_unstale":"2023-07-18T03:39:56.347811+0000","last_undegraded":"2023-07-18T03:39:56.347811+0000","last_fullsized":"2023-07-18T03:39:56.347811+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:49:19.376544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354809+0000","last_change":"2023-07-18T03:39:57.347143+0000","last_active":"2023-07-18T03:39:57.354809+0000","last_peered":"2023-07-18T03:39:57.354809+0000","last_clean":"2023-07-18T03:39:57.354809+0000","last_became_active":"2023-07-18T03:39:57.346833+0000","last_became_peered":"2023-07-18T03:39:57.346833+0000","last_unstale":"2023-07-18T03:39:57.354809+0000","last_undegraded":"2023-07-18T03:39:57.354809+0000","last_fullsized":"2023-07-18T03:39:57.354809+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:26:07.789214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.339763+0000","last_change":"2023-07-18T03:39:56.339148+0000","last_active":"2023-07-18T03:39:57.339763+0000","last_peered":"2023-07-18T03:39:57.339763+0000","last_clean":"2023-07-18T03:39:57.339763+0000","last_became_active":"2023-07-18T03:39:56.338862+0000","last_became_peered":"2023-07-18T03:39:56.338862+0000","last_unstale":"2023-07-18T03:39:57.339763+0000","last_undegraded":"2023-07-18T03:39:57.339763+0000","last_fullsized":"2023-07-18T03:39:57.339763+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:14:22.006899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.340136+0000","last_change":"2023-07-18T03:39:56.340136+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.340136+0000","last_undegraded":"2023-07-18T03:39:56.340136+0000","last_fullsized":"2023-07-18T03:39:56.340136+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.364107+0000","last_change":"2023-07-18T03:39:56.345149+0000","last_active":"2023-07-18T03:39:57.364107+0000","last_peered":"2023-07-18T03:39:57.364107+0000","last_clean":"2023-07-18T03:39:57.364107+0000","last_became_active":"2023-07-18T03:39:56.344848+0000","last_became_peered":"2023-07-18T03:39:56.344848+0000","last_unstale":"2023-07-18T03:39:57.364107+0000","last_undegraded":"2023-07-18T03:39:57.364107+0000","last_fullsized":"2023-07-18T03:39:57.364107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:51:09.697807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335929+0000","last_change":"2023-07-18T03:39:56.335929+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335929+0000","last_undegraded":"2023-07-18T03:39:56.335929+0000","last_fullsized":"2023-07-18T03:39:56.335929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.349330+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:56.349330+0000","last_peered":"2023-07-18T03:39:56.349330+0000","last_clean":"2023-07-18T03:39:56.349330+0000","last_became_active":"2023-07-18T03:39:56.345797+0000","last_became_peered":"2023-07-18T03:39:56.345797+0000","last_unstale":"2023-07-18T03:39:56.349330+0000","last_undegraded":"2023-07-18T03:39:56.349330+0000","last_fullsized":"2023-07-18T03:39:56.349330+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:12.877614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.339214+0000","last_change":"2023-07-18T03:39:56.339214+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.339214+0000","last_undegraded":"2023-07-18T03:39:56.339214+0000","last_fullsized":"2023-07-18T03:39:56.339214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.350939+0000","last_change":"2023-07-18T03:39:56.349196+0000","last_active":"2023-07-18T03:39:56.350939+0000","last_peered":"2023-07-18T03:39:56.350939+0000","last_clean":"2023-07-18T03:39:56.350939+0000","last_became_active":"2023-07-18T03:39:56.347176+0000","last_became_peered":"2023-07-18T03:39:56.347176+0000","last_unstale":"2023-07-18T03:39:56.350939+0000","last_undegraded":"2023-07-18T03:39:56.350939+0000","last_fullsized":"2023-07-18T03:39:56.350939+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:09.700401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.327716+0000","last_change":"2023-07-18T03:39:56.327716+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.327716+0000","last_undegraded":"2023-07-18T03:39:56.327716+0000","last_fullsized":"2023-07-18T03:39:56.327716+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359507+0000","last_change":"2023-07-18T03:39:56.344722+0000","last_active":"2023-07-18T03:39:57.359507+0000","last_peered":"2023-07-18T03:39:57.359507+0000","last_clean":"2023-07-18T03:39:57.359507+0000","last_became_active":"2023-07-18T03:39:56.344439+0000","last_became_peered":"2023-07-18T03:39:56.344439+0000","last_unstale":"2023-07-18T03:39:57.359507+0000","last_undegraded":"2023-07-18T03:39:57.359507+0000","last_fullsized":"2023-07-18T03:39:57.359507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:47:42.656638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.366921+0000","last_change":"2023-07-18T03:39:57.363607+0000","last_active":"2023-07-18T03:39:57.366921+0000","last_peered":"2023-07-18T03:39:57.366921+0000","last_clean":"2023-07-18T03:39:57.366921+0000","last_became_active":"2023-07-18T03:39:57.363312+0000","last_became_peered":"2023-07-18T03:39:57.363312+0000","last_unstale":"2023-07-18T03:39:57.366921+0000","last_undegraded":"2023-07-18T03:39:57.366921+0000","last_fullsized":"2023-07-18T03:39:57.366921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:56:30.355299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.344690+0000","last_change":"2023-07-18T03:39:56.336359+0000","last_active":"2023-07-18T03:39:56.344690+0000","last_peered":"2023-07-18T03:39:56.344690+0000","last_clean":"2023-07-18T03:39:56.344690+0000","last_became_active":"2023-07-18T03:39:56.336094+0000","last_became_peered":"2023-07-18T03:39:56.336094+0000","last_unstale":"2023-07-18T03:39:56.344690+0000","last_undegraded":"2023-07-18T03:39:56.344690+0000","last_fullsized":"2023-07-18T03:39:56.344690+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:30.478922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355019+0000","last_change":"2023-07-18T03:39:57.348592+0000","last_active":"2023-07-18T03:39:57.355019+0000","last_peered":"2023-07-18T03:39:57.355019+0000","last_clean":"2023-07-18T03:39:57.355019+0000","last_became_active":"2023-07-18T03:39:57.348257+0000","last_became_peered":"2023-07-18T03:39:57.348257+0000","last_unstale":"2023-07-18T03:39:57.355019+0000","last_undegraded":"2023-07-18T03:39:57.355019+0000","last_fullsized":"2023-07-18T03:39:57.355019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:36:37.474323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359688+0000","last_change":"2023-07-18T03:39:56.337863+0000","last_active":"2023-07-18T03:39:57.359688+0000","last_peered":"2023-07-18T03:39:57.359688+0000","last_clean":"2023-07-18T03:39:57.359688+0000","last_became_active":"2023-07-18T03:39:56.337561+0000","last_became_peered":"2023-07-18T03:39:56.337561+0000","last_unstale":"2023-07-18T03:39:57.359688+0000","last_undegraded":"2023-07-18T03:39:57.359688+0000","last_fullsized":"2023-07-18T03:39:57.359688+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:10.926722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.353775+0000","last_change":"2023-07-18T03:39:57.351197+0000","last_active":"2023-07-18T03:39:57.353775+0000","last_peered":"2023-07-18T03:39:57.353775+0000","last_clean":"2023-07-18T03:39:57.353775+0000","last_became_active":"2023-07-18T03:39:57.350814+0000","last_became_peered":"2023-07-18T03:39:57.350814+0000","last_unstale":"2023-07-18T03:39:57.353775+0000","last_undegraded":"2023-07-18T03:39:57.353775+0000","last_fullsized":"2023-07-18T03:39:57.353775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:06:35.294279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.339081+0000","last_change":"2023-07-18T03:39:56.336889+0000","last_active":"2023-07-18T03:39:57.339081+0000","last_peered":"2023-07-18T03:39:57.339081+0000","last_clean":"2023-07-18T03:39:57.339081+0000","last_became_active":"2023-07-18T03:39:56.336599+0000","last_became_peered":"2023-07-18T03:39:56.336599+0000","last_unstale":"2023-07-18T03:39:57.339081+0000","last_undegraded":"2023-07-18T03:39:57.339081+0000","last_fullsized":"2023-07-18T03:39:57.339081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:10.544538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.333821+0000","last_change":"2023-07-18T03:39:56.333821+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.333821+0000","last_undegraded":"2023-07-18T03:39:56.333821+0000","last_fullsized":"2023-07-18T03:39:56.333821+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.331084+0000","last_change":"2023-07-18T03:39:56.350386+0000","last_active":"2023-07-18T03:39:57.331084+0000","last_peered":"2023-07-18T03:39:57.331084+0000","last_clean":"2023-07-18T03:39:57.331084+0000","last_became_active":"2023-07-18T03:39:56.350046+0000","last_became_peered":"2023-07-18T03:39:56.350046+0000","last_unstale":"2023-07-18T03:39:57.331084+0000","last_undegraded":"2023-07-18T03:39:57.331084+0000","last_fullsized":"2023-07-18T03:39:57.331084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:57:37.239962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.339618+0000","last_change":"2023-07-18T03:39:56.339618+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.339618+0000","last_undegraded":"2023-07-18T03:39:56.339618+0000","last_fullsized":"2023-07-18T03:39:56.339618+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.338697+0000","last_change":"2023-07-18T03:39:56.338127+0000","last_active":"2023-07-18T03:39:57.338697+0000","last_peered":"2023-07-18T03:39:57.338697+0000","last_clean":"2023-07-18T03:39:57.338697+0000","last_became_active":"2023-07-18T03:39:56.337747+0000","last_became_peered":"2023-07-18T03:39:56.337747+0000","last_unstale":"2023-07-18T03:39:57.338697+0000","last_undegraded":"2023-07-18T03:39:57.338697+0000","last_fullsized":"2023-07-18T03:39:57.338697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:08:17.902329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328126+0000","last_change":"2023-07-18T03:39:56.328126+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328126+0000","last_undegraded":"2023-07-18T03:39:56.328126+0000","last_fullsized":"2023-07-18T03:39:56.328126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.355935+0000","last_change":"2023-07-18T03:39:56.353087+0000","last_active":"2023-07-18T03:39:56.355935+0000","last_peered":"2023-07-18T03:39:56.355935+0000","last_clean":"2023-07-18T03:39:56.355935+0000","last_became_active":"2023-07-18T03:39:56.352738+0000","last_became_peered":"2023-07-18T03:39:56.352738+0000","last_unstale":"2023-07-18T03:39:56.355935+0000","last_undegraded":"2023-07-18T03:39:56.355935+0000","last_fullsized":"2023-07-18T03:39:56.355935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:38:20.581738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.329523+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:39:56.329523+0000","last_peered":"2023-07-18T03:39:56.329523+0000","last_clean":"2023-07-18T03:39:56.329523+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:39:56.329523+0000","last_undegraded":"2023-07-18T03:39:56.329523+0000","last_fullsized":"2023-07-18T03:39:56.329523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:58:20.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355192+0000","last_change":"2023-07-18T03:39:57.348865+0000","last_active":"2023-07-18T03:39:57.355192+0000","last_peered":"2023-07-18T03:39:57.355192+0000","last_clean":"2023-07-18T03:39:57.355192+0000","last_became_active":"2023-07-18T03:39:57.348246+0000","last_became_peered":"2023-07-18T03:39:57.348246+0000","last_unstale":"2023-07-18T03:39:57.355192+0000","last_undegraded":"2023-07-18T03:39:57.355192+0000","last_fullsized":"2023-07-18T03:39:57.355192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:47:29.972115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354659+0000","last_change":"2023-07-18T03:39:56.352671+0000","last_active":"2023-07-18T03:39:56.354659+0000","last_peered":"2023-07-18T03:39:56.354659+0000","last_clean":"2023-07-18T03:39:56.354659+0000","last_became_active":"2023-07-18T03:39:56.351865+0000","last_became_peered":"2023-07-18T03:39:56.351865+0000","last_unstale":"2023-07-18T03:39:56.354659+0000","last_undegraded":"2023-07-18T03:39:56.354659+0000","last_fullsized":"2023-07-18T03:39:56.354659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:01:05.415848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.342292+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:39:56.342292+0000","last_peered":"2023-07-18T03:39:56.342292+0000","last_clean":"2023-07-18T03:39:56.342292+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:39:56.342292+0000","last_undegraded":"2023-07-18T03:39:56.342292+0000","last_fullsized":"2023-07-18T03:39:56.342292+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:40:43.825533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.329925+0000","last_change":"2023-07-18T03:39:56.329925+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.329925+0000","last_undegraded":"2023-07-18T03:39:56.329925+0000","last_fullsized":"2023-07-18T03:39:56.329925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.349349+0000","last_change":"2023-07-18T03:39:56.347782+0000","last_active":"2023-07-18T03:39:56.349349+0000","last_peered":"2023-07-18T03:39:56.349349+0000","last_clean":"2023-07-18T03:39:56.349349+0000","last_became_active":"2023-07-18T03:39:56.347519+0000","last_became_peered":"2023-07-18T03:39:56.347519+0000","last_unstale":"2023-07-18T03:39:56.349349+0000","last_undegraded":"2023-07-18T03:39:56.349349+0000","last_fullsized":"2023-07-18T03:39:56.349349+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:25.713685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.332914+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:39:56.332914+0000","last_peered":"2023-07-18T03:39:56.332914+0000","last_clean":"2023-07-18T03:39:56.332914+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:39:56.332914+0000","last_undegraded":"2023-07-18T03:39:56.332914+0000","last_fullsized":"2023-07-18T03:39:56.332914+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:40:36.216166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.330424+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:39:57.330424+0000","last_peered":"2023-07-18T03:39:57.330424+0000","last_clean":"2023-07-18T03:39:57.330424+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:39:57.330424+0000","last_undegraded":"2023-07-18T03:39:57.330424+0000","last_fullsized":"2023-07-18T03:39:57.330424+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:22:09.535618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355161+0000","last_change":"2023-07-18T03:39:57.350408+0000","last_active":"2023-07-18T03:39:57.355161+0000","last_peered":"2023-07-18T03:39:57.355161+0000","last_clean":"2023-07-18T03:39:57.355161+0000","last_became_active":"2023-07-18T03:39:57.350103+0000","last_became_peered":"2023-07-18T03:39:57.350103+0000","last_unstale":"2023-07-18T03:39:57.355161+0000","last_undegraded":"2023-07-18T03:39:57.355161+0000","last_fullsized":"2023-07-18T03:39:57.355161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:47:42.227277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.338099+0000","last_change":"2023-07-18T03:39:56.346081+0000","last_active":"2023-07-18T03:39:57.338099+0000","last_peered":"2023-07-18T03:39:57.338099+0000","last_clean":"2023-07-18T03:39:57.338099+0000","last_became_active":"2023-07-18T03:39:56.345762+0000","last_became_peered":"2023-07-18T03:39:56.345762+0000","last_unstale":"2023-07-18T03:39:57.338099+0000","last_undegraded":"2023-07-18T03:39:57.338099+0000","last_fullsized":"2023-07-18T03:39:57.338099+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:15:29.121017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":45,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.329508+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:39:57.329508+0000","last_peered":"2023-07-18T03:39:57.329508+0000","last_clean":"2023-07-18T03:39:57.329508+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:39:57.329508+0000","last_undegraded":"2023-07-18T03:39:57.329508+0000","last_fullsized":"2023-07-18T03:39:57.329508+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:13:09.287323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.336448+0000","last_change":"2023-07-18T03:39:56.336448+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.336448+0000","last_undegraded":"2023-07-18T03:39:56.336448+0000","last_fullsized":"2023-07-18T03:39:56.336448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.330231+0000","last_change":"2023-07-18T03:39:56.348881+0000","last_active":"2023-07-18T03:39:57.330231+0000","last_peered":"2023-07-18T03:39:57.330231+0000","last_clean":"2023-07-18T03:39:57.330231+0000","last_became_active":"2023-07-18T03:39:56.348622+0000","last_became_peered":"2023-07-18T03:39:56.348622+0000","last_unstale":"2023-07-18T03:39:57.330231+0000","last_undegraded":"2023-07-18T03:39:57.330231+0000","last_fullsized":"2023-07-18T03:39:57.330231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:53:21.359259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":45,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.330272+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:39:57.330272+0000","last_peered":"2023-07-18T03:39:57.330272+0000","last_clean":"2023-07-18T03:39:57.330272+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:39:57.330272+0000","last_undegraded":"2023-07-18T03:39:57.330272+0000","last_fullsized":"2023-07-18T03:39:57.330272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:39:12.642880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335532+0000","last_change":"2023-07-18T03:39:56.335532+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335532+0000","last_undegraded":"2023-07-18T03:39:56.335532+0000","last_fullsized":"2023-07-18T03:39:56.335532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.336107+0000","last_change":"2023-07-18T03:39:56.337331+0000","last_active":"2023-07-18T03:39:57.336107+0000","last_peered":"2023-07-18T03:39:57.336107+0000","last_clean":"2023-07-18T03:39:57.336107+0000","last_became_active":"2023-07-18T03:39:56.337021+0000","last_became_peered":"2023-07-18T03:39:56.337021+0000","last_unstale":"2023-07-18T03:39:57.336107+0000","last_undegraded":"2023-07-18T03:39:57.336107+0000","last_fullsized":"2023-07-18T03:39:57.336107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:22:50.209633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.340413+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:39:56.340413+0000","last_peered":"2023-07-18T03:39:56.340413+0000","last_clean":"2023-07-18T03:39:56.340413+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:39:56.340413+0000","last_undegraded":"2023-07-18T03:39:56.340413+0000","last_fullsized":"2023-07-18T03:39:56.340413+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:45:16.320741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.329526+0000","last_change":"2023-07-18T03:39:56.329526+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.329526+0000","last_undegraded":"2023-07-18T03:39:56.329526+0000","last_fullsized":"2023-07-18T03:39:56.329526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.340100+0000","last_change":"2023-07-18T03:39:56.341357+0000","last_active":"2023-07-18T03:39:57.340100+0000","last_peered":"2023-07-18T03:39:57.340100+0000","last_clean":"2023-07-18T03:39:57.340100+0000","last_became_active":"2023-07-18T03:39:56.341092+0000","last_became_peered":"2023-07-18T03:39:56.341092+0000","last_unstale":"2023-07-18T03:39:57.340100+0000","last_undegraded":"2023-07-18T03:39:57.340100+0000","last_fullsized":"2023-07-18T03:39:57.340100+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:48:52.596611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.339973+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:39:57.339973+0000","last_peered":"2023-07-18T03:39:57.339973+0000","last_clean":"2023-07-18T03:39:57.339973+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:39:57.339973+0000","last_undegraded":"2023-07-18T03:39:57.339973+0000","last_fullsized":"2023-07-18T03:39:57.339973+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:48:55.614775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.363216+0000","last_change":"2023-07-18T03:39:57.360948+0000","last_active":"2023-07-18T03:39:57.363216+0000","last_peered":"2023-07-18T03:39:57.363216+0000","last_clean":"2023-07-18T03:39:57.363216+0000","last_became_active":"2023-07-18T03:39:57.360601+0000","last_became_peered":"2023-07-18T03:39:57.360601+0000","last_unstale":"2023-07-18T03:39:57.363216+0000","last_undegraded":"2023-07-18T03:39:57.363216+0000","last_fullsized":"2023-07-18T03:39:57.363216+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:49:44.089477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.336599+0000","last_change":"2023-07-18T03:39:56.342369+0000","last_active":"2023-07-18T03:39:57.336599+0000","last_peered":"2023-07-18T03:39:57.336599+0000","last_clean":"2023-07-18T03:39:57.336599+0000","last_became_active":"2023-07-18T03:39:56.342108+0000","last_became_peered":"2023-07-18T03:39:56.342108+0000","last_unstale":"2023-07-18T03:39:57.336599+0000","last_undegraded":"2023-07-18T03:39:57.336599+0000","last_fullsized":"2023-07-18T03:39:57.336599+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:44.715606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.340446+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:39:57.340446+0000","last_peered":"2023-07-18T03:39:57.340446+0000","last_clean":"2023-07-18T03:39:57.340446+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:39:57.340446+0000","last_undegraded":"2023-07-18T03:39:57.340446+0000","last_fullsized":"2023-07-18T03:39:57.340446+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:54:03.329884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":652,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":999424,"data_stored":752338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":1.7729999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":2.2130000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":3.0030000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.2290000000000001,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":2.2570000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.0249999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":4.0010000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":1.2889999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":2.2989999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":3.2029999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":3.1219999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":1.0609999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":393216,"data_stored":160918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.626}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.72899999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.69299999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67800000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.82499999999999996}]}]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":376832,"data_stored":153208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52600000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.8290000000000002,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.747},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.80400000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":970752,"data_stored":743576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32200000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.495}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79000000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.73899999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":376832,"data_stored":153208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.82399999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.78500000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64700000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.75800000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.68899999999999995}]}]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":376832,"data_stored":153208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":4.9260000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":2.1699999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":8.3780000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":5.5060000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":4.8769999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":2.1230000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":2.2370000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":1.6599999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":4.9690000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":3.6829999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":5.3140000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":3.7389999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.71799999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":393216,"data_stored":160370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":8.3040000000000003,"5min":8.3040000000000003,"15min":8.3040000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.502}]},{"osd":3,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.0229999999999997,"5min":8.0229999999999997,"15min":8.0229999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":8.359,"5min":8.359,"15min":8.359},"last":0.48099999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.89900000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.78300000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.88200000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.97399999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":393216,"data_stored":160406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39100000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.438}]},{"osd":3,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.78100000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.79600000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.60499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:40:02.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2023-07-18T03:40:02.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.337498+0000', 'last_change': '2023-07-18T03:39:56.337498+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.337498+0000', 'last_undegraded': '2023-07-18T03:39:56.337498+0000', 'last_fullsized': '2023-07-18T03:39:56.337498+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-18T03:40:02.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2023-07-18T03:40:02.800 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.335988+0000', 'last_change': '2023-07-18T03:39:56.335988+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.335988+0000', 'last_undegraded': '2023-07-18T03:39:56.335988+0000', 'last_fullsized': '2023-07-18T03:39:56.335988+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-18T03:40:02.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2023-07-18T03:40:02.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.328919+0000', 'last_change': '2023-07-18T03:39:56.328919+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.328919+0000', 'last_undegraded': '2023-07-18T03:39:56.328919+0000', 'last_fullsized': '2023-07-18T03:39:56.328919+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-18T03:40:02.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2023-07-18T03:40:02.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.332077+0000', 'last_change': '2023-07-18T03:39:56.332077+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.332077+0000', 'last_undegraded': '2023-07-18T03:39:56.332077+0000', 'last_fullsized': '2023-07-18T03:39:56.332077+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-18T03:40:02.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2023-07-18T03:40:02.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.335775+0000', 'last_change': '2023-07-18T03:39:56.335775+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.335775+0000', 'last_undegraded': '2023-07-18T03:39:56.335775+0000', 'last_fullsized': '2023-07-18T03:39:56.335775+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-18T03:40:02.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2023-07-18T03:40:02.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.338378+0000', 'last_change': '2023-07-18T03:39:56.338378+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.338378+0000', 'last_undegraded': '2023-07-18T03:39:56.338378+0000', 'last_fullsized': '2023-07-18T03:39:56.338378+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-18T03:40:02.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2023-07-18T03:40:02.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.333492+0000', 'last_change': '2023-07-18T03:39:56.333492+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.333492+0000', 'last_undegraded': '2023-07-18T03:39:56.333492+0000', 'last_fullsized': '2023-07-18T03:39:56.333492+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-18T03:40:02.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2023-07-18T03:40:02.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.332641+0000', 'last_change': '2023-07-18T03:39:56.332641+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.332641+0000', 'last_undegraded': '2023-07-18T03:39:56.332641+0000', 'last_fullsized': '2023-07-18T03:39:56.332641+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-18T03:40:02.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2023-07-18T03:40:02.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.327178+0000', 'last_change': '2023-07-18T03:39:56.327178+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.327178+0000', 'last_undegraded': '2023-07-18T03:39:56.327178+0000', 'last_fullsized': '2023-07-18T03:39:56.327178+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-18T03:40:02.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2023-07-18T03:40:02.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.332947+0000', 'last_change': '2023-07-18T03:39:56.332947+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.332947+0000', 'last_undegraded': '2023-07-18T03:39:56.332947+0000', 'last_fullsized': '2023-07-18T03:39:56.332947+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-18T03:40:02.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2023-07-18T03:40:02.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.331681+0000', 'last_change': '2023-07-18T03:39:56.331681+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.331681+0000', 'last_undegraded': '2023-07-18T03:39:56.331681+0000', 'last_fullsized': '2023-07-18T03:39:56.331681+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-18T03:40:02.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2023-07-18T03:40:02.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.329011+0000', 'last_change': '2023-07-18T03:39:56.329011+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.329011+0000', 'last_undegraded': '2023-07-18T03:39:56.329011+0000', 'last_fullsized': '2023-07-18T03:39:56.329011+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-18T03:40:02.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2023-07-18T03:40:02.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.328947+0000', 'last_change': '2023-07-18T03:39:56.328947+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.328947+0000', 'last_undegraded': '2023-07-18T03:39:56.328947+0000', 'last_fullsized': '2023-07-18T03:39:56.328947+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-18T03:40:02.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2023-07-18T03:40:02.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.337384+0000', 'last_change': '2023-07-18T03:39:56.337384+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.337384+0000', 'last_undegraded': '2023-07-18T03:39:56.337384+0000', 'last_fullsized': '2023-07-18T03:39:56.337384+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-18T03:40:02.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2023-07-18T03:40:02.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.328673+0000', 'last_change': '2023-07-18T03:39:56.328673+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.328673+0000', 'last_undegraded': '2023-07-18T03:39:56.328673+0000', 'last_fullsized': '2023-07-18T03:39:56.328673+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-18T03:40:02.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-07-18T03:40:02.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.330756+0000', 'last_change': '2023-07-18T03:39:56.330756+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.330756+0000', 'last_undegraded': '2023-07-18T03:39:56.330756+0000', 'last_fullsized': '2023-07-18T03:39:56.330756+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-18T03:40:02.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-07-18T03:40:02.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.328243+0000', 'last_change': '2023-07-18T03:39:56.328243+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.328243+0000', 'last_undegraded': '2023-07-18T03:39:56.328243+0000', 'last_fullsized': '2023-07-18T03:39:56.328243+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-18T03:40:02.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-07-18T03:40:02.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.338278+0000', 'last_change': '2023-07-18T03:39:56.338278+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.338278+0000', 'last_undegraded': '2023-07-18T03:39:56.338278+0000', 'last_fullsized': '2023-07-18T03:39:56.338278+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-18T03:40:02.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-07-18T03:40:02.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.332552+0000', 'last_change': '2023-07-18T03:39:56.332552+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.332552+0000', 'last_undegraded': '2023-07-18T03:39:56.332552+0000', 'last_fullsized': '2023-07-18T03:39:56.332552+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-18T03:40:02.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-07-18T03:40:02.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.328578+0000', 'last_change': '2023-07-18T03:39:56.328578+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.328578+0000', 'last_undegraded': '2023-07-18T03:39:56.328578+0000', 'last_fullsized': '2023-07-18T03:39:56.328578+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-18T03:40:02.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-07-18T03:40:02.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.339195+0000', 'last_change': '2023-07-18T03:39:56.339195+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.339195+0000', 'last_undegraded': '2023-07-18T03:39:56.339195+0000', 'last_fullsized': '2023-07-18T03:39:56.339195+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-18T03:40:02.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-07-18T03:40:02.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.327915+0000', 'last_change': '2023-07-18T03:39:56.327915+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.327915+0000', 'last_undegraded': '2023-07-18T03:39:56.327915+0000', 'last_fullsized': '2023-07-18T03:39:56.327915+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-18T03:40:02.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-07-18T03:40:02.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.337060+0000', 'last_change': '2023-07-18T03:39:56.337060+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.337060+0000', 'last_undegraded': '2023-07-18T03:39:56.337060+0000', 'last_fullsized': '2023-07-18T03:39:56.337060+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-18T03:40:02.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2023-07-18T03:40:02.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.340136+0000', 'last_change': '2023-07-18T03:39:56.340136+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.340136+0000', 'last_undegraded': '2023-07-18T03:39:56.340136+0000', 'last_fullsized': '2023-07-18T03:39:56.340136+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-18T03:40:02.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-07-18T03:40:02.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.335929+0000', 'last_change': '2023-07-18T03:39:56.335929+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.335929+0000', 'last_undegraded': '2023-07-18T03:39:56.335929+0000', 'last_fullsized': '2023-07-18T03:39:56.335929+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-18T03:40:02.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-07-18T03:40:02.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.339214+0000', 'last_change': '2023-07-18T03:39:56.339214+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.339214+0000', 'last_undegraded': '2023-07-18T03:39:56.339214+0000', 'last_fullsized': '2023-07-18T03:39:56.339214+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-18T03:40:02.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-07-18T03:40:02.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.327716+0000', 'last_change': '2023-07-18T03:39:56.327716+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.327716+0000', 'last_undegraded': '2023-07-18T03:39:56.327716+0000', 'last_fullsized': '2023-07-18T03:39:56.327716+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-18T03:40:02.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-07-18T03:40:02.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.333821+0000', 'last_change': '2023-07-18T03:39:56.333821+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.333821+0000', 'last_undegraded': '2023-07-18T03:39:56.333821+0000', 'last_fullsized': '2023-07-18T03:39:56.333821+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-18T03:40:02.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2023-07-18T03:40:02.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.339618+0000', 'last_change': '2023-07-18T03:39:56.339618+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.339618+0000', 'last_undegraded': '2023-07-18T03:39:56.339618+0000', 'last_fullsized': '2023-07-18T03:39:56.339618+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-18T03:40:02.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-07-18T03:40:02.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.328126+0000', 'last_change': '2023-07-18T03:39:56.328126+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.328126+0000', 'last_undegraded': '2023-07-18T03:39:56.328126+0000', 'last_fullsized': '2023-07-18T03:39:56.328126+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-18T03:40:02.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-07-18T03:40:02.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.329925+0000', 'last_change': '2023-07-18T03:39:56.329925+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.329925+0000', 'last_undegraded': '2023-07-18T03:39:56.329925+0000', 'last_fullsized': '2023-07-18T03:39:56.329925+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-18T03:40:02.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-07-18T03:40:02.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.336448+0000', 'last_change': '2023-07-18T03:39:56.336448+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.336448+0000', 'last_undegraded': '2023-07-18T03:39:56.336448+0000', 'last_fullsized': '2023-07-18T03:39:56.336448+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-18T03:40:02.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-07-18T03:40:02.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.335532+0000', 'last_change': '2023-07-18T03:39:56.335532+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.335532+0000', 'last_undegraded': '2023-07-18T03:39:56.335532+0000', 'last_fullsized': '2023-07-18T03:39:56.335532+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-18T03:40:02.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-07-18T03:40:02.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-07-18T03:39:56.329526+0000', 'last_change': '2023-07-18T03:39:56.329526+0000', 'last_active': '2023-07-18T03:39:56.318026+0000', 'last_peered': '2023-07-18T03:39:56.318026+0000', 'last_clean': '2023-07-18T03:39:56.318026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-18T03:39:56.329526+0000', 'last_undegraded': '2023-07-18T03:39:56.329526+0000', 'last_fullsized': '2023-07-18T03:39:56.329526+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'last_clean_scrub_stamp': '2023-07-18T03:39:56.318026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-18T03:40:02.829 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-07-18T03:40:03.027 INFO:tasks.ceph.mds.c.smithi049.stderr:2023-07-18T03:40:03.032+0000 7f60b99f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-18T03:40:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.038+0000 7f1a61bff700 1 Processor -- start 2023-07-18T03:40:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.038+0000 7f1a61bff700 1 -- start start 2023-07-18T03:40:03.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.039+0000 7f1a61bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1a5c15c950 0x7f1a5c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.039+0000 7f1a61bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a5c0b1e00 0x7f1a5c0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.039+0000 7f1a61bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c0d2ce0 con 0x7f1a5c0b2350 2023-07-18T03:40:03.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.039+0000 7f1a61bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5c0b3350 con 0x7f1a5c15c950 2023-07-18T03:40:03.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.039+0000 7f1a61bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0b3630 con 0x7f1a5c0b1e00 2023-07-18T03:40:03.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.040+0000 7f1a5affd700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a5c0b1e00 0x7f1a5c0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.040+0000 7f1a5b7fe700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1a5c15c950 0x7f1a5c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.040+0000 7f1a5b7fe700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1a5c15c950 0x7f1a5c0b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:57968/0 (socket says 172.21.15.49:57968) 2023-07-18T03:40:03.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.040+0000 7f1a5b7fe700 1 -- 172.21.15.49:0/1808416483 learned_addr learned my addr 172.21.15.49:0/1808416483 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:03.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.040+0000 7f1a5b7fe700 1 -- 172.21.15.49:0/1808416483 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a5c0b1e00 msgr2=0x7f1a5c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:03.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.040+0000 7f1a5b7fe700 1 --2- 172.21.15.49:0/1808416483 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a5c0b1e00 0x7f1a5c0d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.040+0000 7f1a5b7fe700 1 -- 172.21.15.49:0/1808416483 >> v1:172.21.15.107:6789/0 conn(0x7f1a5c0b2350 legacy=0x7f1a5c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.040+0000 7f1a5b7fe700 1 -- 172.21.15.49:0/1808416483 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c0b3910 con 0x7f1a5c15c950 2023-07-18T03:40:03.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.041+0000 7f1a5b7fe700 1 --2- 172.21.15.49:0/1808416483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a5c15c950 0x7f1a5c0b43f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c006fd0 tx=0x7f1a4c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df701d9df593df5a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:03.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.041+0000 7f1a5a7fc700 1 -- 172.21.15.49:0/1808416483 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c005e30 con 0x7f1a5c15c950 2023-07-18T03:40:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.041+0000 7f1a5a7fc700 1 -- 172.21.15.49:0/1808416483 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4c004030 con 0x7f1a5c15c950 2023-07-18T03:40:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.041+0000 7f1a5a7fc700 1 -- 172.21.15.49:0/1808416483 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0043e0 con 0x7f1a5c15c950 2023-07-18T03:40:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.041+0000 7f1a61bff700 1 -- 172.21.15.49:0/1808416483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a5c15c950 msgr2=0x7f1a5c0b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.041+0000 7f1a61bff700 1 --2- 172.21.15.49:0/1808416483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a5c15c950 0x7f1a5c0b43f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c006fd0 tx=0x7f1a4c009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.041+0000 7f1a61bff700 1 -- 172.21.15.49:0/1808416483 shutdown_connections 2023-07-18T03:40:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.041+0000 7f1a61bff700 1 --2- 172.21.15.49:0/1808416483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a5c15c950 0x7f1a5c0b43f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.041+0000 7f1a61bff700 1 --2- 172.21.15.49:0/1808416483 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a5c0b1e00 0x7f1a5c0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.041+0000 7f1a61bff700 1 -- 172.21.15.49:0/1808416483 >> 172.21.15.49:0/1808416483 conn(0x7f1a5c09c1d0 msgr2=0x7f1a5c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:03.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.042+0000 7f1a61bff700 1 -- 172.21.15.49:0/1808416483 shutdown_connections 2023-07-18T03:40:03.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.042+0000 7f1a61bff700 1 -- 172.21.15.49:0/1808416483 wait complete. 2023-07-18T03:40:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.043+0000 7f1a61bff700 1 Processor -- start 2023-07-18T03:40:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.044+0000 7f1a61bff700 1 -- start start 2023-07-18T03:40:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.044+0000 7f1a61bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a5c0b1e00 0x7f1a5c0c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.044+0000 7f1a61bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a5c0b2350 0x7f1a5c147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.044+0000 7f1a61bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a5c15c950 0x7f1a5c0c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a61bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0c8480 con 0x7f1a5c0b2350 2023-07-18T03:40:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a61bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0cec10 con 0x7f1a5c15c950 2023-07-18T03:40:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a61bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0ceef0 con 0x7f1a5c0b1e00 2023-07-18T03:40:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a5affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a5c0b2350 0x7f1a5c147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a5affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a5c0b2350 0x7f1a5c147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38464/0 (socket says 172.21.15.49:38464) 2023-07-18T03:40:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a5affd700 1 -- 172.21.15.49:0/2170647647 learned_addr learned my addr 172.21.15.49:0/2170647647 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:03.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a5b7fe700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a5c0b1e00 0x7f1a5c0c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a5bfff700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a5c15c950 0x7f1a5c0c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a5affd700 1 -- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a5c0b1e00 msgr2=0x7f1a5c0c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a5affd700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a5c0b1e00 0x7f1a5c0c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a5affd700 1 -- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a5c15c950 msgr2=0x7f1a5c0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a5affd700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a5c15c950 0x7f1a5c0c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.045+0000 7f1a5affd700 1 -- 172.21.15.49:0/2170647647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c009500 con 0x7f1a5c0b2350 2023-07-18T03:40:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.046+0000 7f1a5affd700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a5c0b2350 0x7f1a5c147c50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000e9b0 tx=0x7f1a5000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.046+0000 7f1a43fff700 1 -- 172.21.15.49:0/2170647647 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50009710 con 0x7f1a5c0b2350 2023-07-18T03:40:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.046+0000 7f1a43fff700 1 -- 172.21.15.49:0/2170647647 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5000ee50 con 0x7f1a5c0b2350 2023-07-18T03:40:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.046+0000 7f1a61bff700 1 -- 172.21.15.49:0/2170647647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5c0cf1d0 con 0x7f1a5c0b2350 2023-07-18T03:40:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.046+0000 7f1a43fff700 1 -- 172.21.15.49:0/2170647647 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5000ca50 con 0x7f1a5c0b2350 2023-07-18T03:40:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.046+0000 7f1a61bff700 1 -- 172.21.15.49:0/2170647647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c158020 con 0x7f1a5c0b2350 2023-07-18T03:40:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.047+0000 7f1a61bff700 1 -- 172.21.15.49:0/2170647647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5c04e280 con 0x7f1a5c0b2350 2023-07-18T03:40:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.048+0000 7f1a43fff700 1 -- 172.21.15.49:0/2170647647 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a5000cbb0 con 0x7f1a5c0b2350 2023-07-18T03:40:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.049+0000 7f1a43fff700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1a4806ace0 0x7f1a4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.049+0000 7f1a43fff700 1 -- 172.21.15.49:0/2170647647 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a500893e0 con 0x7f1a5c0b2350 2023-07-18T03:40:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.049+0000 7f1a5b7fe700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1a4806ace0 0x7f1a4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.050+0000 7f1a5b7fe700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1a4806ace0 0x7f1a4806d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c006fd0 tx=0x7f1a4c008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.053+0000 7f1a43fff700 1 -- 172.21.15.49:0/2170647647 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1a50057fb0 con 0x7f1a5c0b2350 2023-07-18T03:40:03.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.342+0000 7f1a61bff700 1 -- 172.21.15.49:0/2170647647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1a5c08c5a0 con 0x7f1a5c0b2350 2023-07-18T03:40:03.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.342+0000 7f1a43fff700 1 -- 172.21.15.49:0/2170647647 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1098 (secure 0 0 0) 0x7f1a5000ce60 con 0x7f1a5c0b2350 2023-07-18T03:40:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:03.338 INFO:teuthology.orchestra.run.smithi049.stdout:{"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":108,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":36,"num_osds":8,"num_up_osds":8,"osd_up_since":1689651507,"num_in_osds":8,"osd_in_since":1689651498,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":71},{"state_name":"creating+peering","count":34}],"num_pgs":105,"num_pools":4,"num_objects":10,"data_bytes":591020,"bytes_used":241848320,"bytes_avail":772852264960,"bytes_total":773094113280,"inactive_pgs_ratio":0.32380953431129456,"write_bytes_sec":341,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5172}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-07-18T03:39:38.111901+0000","services":{}},"progress_events":{}} 2023-07-18T03:40:03.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.344+0000 7f1a61bff700 1 -- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1a4806ace0 msgr2=0x7f1a4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.344+0000 7f1a61bff700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1a4806ace0 0x7f1a4806d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c006fd0 tx=0x7f1a4c008070 comp rx=0 tx=0).stop 2023-07-18T03:40:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.344+0000 7f1a61bff700 1 -- 172.21.15.49:0/2170647647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a5c0b2350 msgr2=0x7f1a5c147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.344+0000 7f1a61bff700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a5c0b2350 0x7f1a5c147c50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000e9b0 tx=0x7f1a5000ecc0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.344+0000 7f1a61bff700 1 -- 172.21.15.49:0/2170647647 shutdown_connections 2023-07-18T03:40:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.344+0000 7f1a61bff700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a5c15c950 0x7f1a5c0c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.344+0000 7f1a61bff700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a5c0b1e00 0x7f1a5c0c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.344+0000 7f1a61bff700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1a4806ace0 0x7f1a4806d190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.344+0000 7f1a61bff700 1 --2- 172.21.15.49:0/2170647647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a5c0b2350 0x7f1a5c147c50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.344+0000 7f1a61bff700 1 -- 172.21.15.49:0/2170647647 >> 172.21.15.49:0/2170647647 conn(0x7f1a5c09c1d0 msgr2=0x7f1a5c0d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:03.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.344+0000 7f1a61bff700 1 -- 172.21.15.49:0/2170647647 shutdown_connections 2023-07-18T03:40:03.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.345+0000 7f1a61bff700 1 -- 172.21.15.49:0/2170647647 wait complete. 2023-07-18T03:40:03.358 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-07-18T03:40:03.359 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:40:03.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.570+0000 7fdf331ff700 1 Processor -- start 2023-07-18T03:40:03.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.570+0000 7fdf331ff700 1 -- start start 2023-07-18T03:40:03.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.571+0000 7fdf331ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdf2c0d1220 0x7fdf2c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.571+0000 7fdf331ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c0b1e00 0x7fdf2c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.571+0000 7fdf331ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c0d2ce0 con 0x7fdf2c15c950 2023-07-18T03:40:03.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.571+0000 7fdf331ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c0b3350 con 0x7fdf2c0d1220 2023-07-18T03:40:03.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.571+0000 7fdf331ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0b3630 con 0x7fdf2c0b1e00 2023-07-18T03:40:03.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.571+0000 7fdf30f9b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdf2c15c950 0x7fdf2c0d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56854/0 (socket says 172.21.15.49:56854) 2023-07-18T03:40:03.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.571+0000 7fdf30f9b700 1 -- 172.21.15.49:0/2774443856 learned_addr learned my addr 172.21.15.49:0/2774443856 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:03.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf3179c700 1 --2- 172.21.15.49:0/2774443856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c0b1e00 0x7fdf2c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2bfff700 1 --2- 172.21.15.49:0/2774443856 >> v2:172.21.15.107:3300/0 conn(0x7fdf2c0d1220 0x7fdf2c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410826992 0 0) 0x7fdf2c0d2ce0 con 0x7fdf2c15c950 2023-07-18T03:40:03.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf1c0035b0 con 0x7fdf2c15c950 2023-07-18T03:40:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1475219091 0 0) 0x7fdf1c0035b0 con 0x7fdf2c15c950 2023-07-18T03:40:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c0b1e00 msgr2=0x7fdf2c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 --2- 172.21.15.49:0/2774443856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c0b1e00 0x7fdf2c0afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 >> v2:172.21.15.107:3300/0 conn(0x7fdf2c0d1220 msgr2=0x7fdf2c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:03.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 --2- 172.21.15.49:0/2774443856 >> v2:172.21.15.107:3300/0 conn(0x7fdf2c0d1220 0x7fdf2c0b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c0b3910 con 0x7fdf2c15c950 2023-07-18T03:40:03.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7fdf200017f0 con 0x7fdf2c15c950 2023-07-18T03:40:03.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 >> v1:172.21.15.49:6789/0 conn(0x7fdf2c15c950 legacy=0x7fdf2c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 --2- 172.21.15.49:0/2774443856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c0b1e00 0x7fdf1c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 --2- 172.21.15.49:0/2774443856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf1c003fd0 0x7fdf1c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.572+0000 7fdf2b7fe700 1 --2- 172.21.15.49:0/2774443856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c004d90 0x7fdf1c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.573+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0b3350 con 0x7fdf1c004d90 2023-07-18T03:40:03.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.573+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0b3630 con 0x7fdf1c003fd0 2023-07-18T03:40:03.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.573+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c00ba90 con 0x7fdf2c0b1e00 2023-07-18T03:40:03.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.573+0000 7fdf30f9b700 1 --2- 172.21.15.49:0/2774443856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c0b1e00 0x7fdf1c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.573+0000 7fdf3179c700 1 --2- 172.21.15.49:0/2774443856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c004d90 0x7fdf1c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.573+0000 7fdf2bfff700 1 --2- 172.21.15.49:0/2774443856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf1c003fd0 0x7fdf1c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.573+0000 7fdf3179c700 1 -- 172.21.15.49:0/2774443856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c0b1e00 msgr2=0x7fdf1c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:03.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.573+0000 7fdf3179c700 1 --2- 172.21.15.49:0/2774443856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c0b1e00 0x7fdf1c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.573+0000 7fdf3179c700 1 -- 172.21.15.49:0/2774443856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf1c003fd0 msgr2=0x7fdf1c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:03.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.573+0000 7fdf3179c700 1 --2- 172.21.15.49:0/2774443856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf1c003fd0 0x7fdf1c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.573+0000 7fdf3179c700 1 -- 172.21.15.49:0/2774443856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c0035b0 con 0x7fdf1c004d90 2023-07-18T03:40:03.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.574+0000 7fdf3179c700 1 --2- 172.21.15.49:0/2774443856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c004d90 0x7fdf1c005200 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009fc0 tx=0x7fdf1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8083e553a47c5401 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:03.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.574+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400ec30 con 0x7fdf1c004d90 2023-07-18T03:40:03.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.574+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2400f040 con 0x7fdf1c004d90 2023-07-18T03:40:03.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.574+0000 7fdf2b7fe700 1 -- 172.21.15.49:0/2774443856 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24014480 con 0x7fdf1c004d90 2023-07-18T03:40:03.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.574+0000 7fdf331ff700 1 -- 172.21.15.49:0/2774443856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c004d90 msgr2=0x7fdf1c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.574+0000 7fdf331ff700 1 --2- 172.21.15.49:0/2774443856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf1c004d90 0x7fdf1c005200 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009fc0 tx=0x7fdf1c005880 comp rx=0 tx=0).stop 2023-07-18T03:40:03.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.574+0000 7fdf3179c700 1 -- 172.21.15.49:0/2774443856 reap_dead start 2023-07-18T03:40:03.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.574+0000 7fdf331ff700 1 -- 172.21.15.49:0/2774443856 shutdown_connections 2023-07-18T03:40:03.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.574+0000 7fdf331ff700 1 -- 172.21.15.49:0/2774443856 >> 172.21.15.49:0/2774443856 conn(0x7fdf2c09c1d0 msgr2=0x7fdf2c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:03.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.574+0000 7fdf331ff700 1 -- 172.21.15.49:0/2774443856 shutdown_connections 2023-07-18T03:40:03.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.575+0000 7fdf331ff700 1 -- 172.21.15.49:0/2774443856 wait complete. 2023-07-18T03:40:03.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.576+0000 7fdf331ff700 1 Processor -- start 2023-07-18T03:40:03.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.576+0000 7fdf331ff700 1 -- start start 2023-07-18T03:40:03.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.577+0000 7fdf331ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf1c004d90 0x7fdf2c14fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.577+0000 7fdf331ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c158b10 0x7fdf2c0b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.577+0000 7fdf331ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf2c0b5d70 0x7fdf2c0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.577+0000 7fdf331ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0cf050 con 0x7fdf2c0b5d70 2023-07-18T03:40:03.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.577+0000 7fdf331ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0b6750 con 0x7fdf1c004d90 2023-07-18T03:40:03.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.577+0000 7fdf331ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c150270 con 0x7fdf2c158b10 2023-07-18T03:40:03.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.578+0000 7fdf3179c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf2c0b5d70 0x7fdf2c0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.578+0000 7fdf3179c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf2c0b5d70 0x7fdf2c0b6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38494/0 (socket says 172.21.15.49:38494) 2023-07-18T03:40:03.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.578+0000 7fdf3179c700 1 -- 172.21.15.49:0/887451550 learned_addr learned my addr 172.21.15.49:0/887451550 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:03.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.578+0000 7fdf30f9b700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf1c004d90 0x7fdf2c14fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.578+0000 7fdf2bfff700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c158b10 0x7fdf2c0b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.578+0000 7fdf3179c700 1 -- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c158b10 msgr2=0x7fdf2c0b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.578+0000 7fdf3179c700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c158b10 0x7fdf2c0b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.578+0000 7fdf3179c700 1 -- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf1c004d90 msgr2=0x7fdf2c14fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.578+0000 7fdf3179c700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf1c004d90 0x7fdf2c14fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.578+0000 7fdf3179c700 1 -- 172.21.15.49:0/887451550 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c0b8c70 con 0x7fdf2c0b5d70 2023-07-18T03:40:03.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.578+0000 7fdf3179c700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf2c0b5d70 0x7fdf2c0b6210 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009f60 tx=0x7fdf24009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:03.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.579+0000 7fdf297fa700 1 -- 172.21.15.49:0/887451550 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24007000 con 0x7fdf2c0b5d70 2023-07-18T03:40:03.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.579+0000 7fdf297fa700 1 -- 172.21.15.49:0/887451550 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf24004470 con 0x7fdf2c0b5d70 2023-07-18T03:40:03.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.579+0000 7fdf297fa700 1 -- 172.21.15.49:0/887451550 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24004820 con 0x7fdf2c0b5d70 2023-07-18T03:40:03.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.579+0000 7fdf331ff700 1 -- 172.21.15.49:0/887451550 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c0c92e0 con 0x7fdf2c0b5d70 2023-07-18T03:40:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.579+0000 7fdf331ff700 1 -- 172.21.15.49:0/887451550 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c0c9000 con 0x7fdf2c0b5d70 2023-07-18T03:40:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.580+0000 7fdf331ff700 1 -- 172.21.15.49:0/887451550 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c0b3bd0 con 0x7fdf2c0b5d70 2023-07-18T03:40:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.581+0000 7fdf297fa700 1 -- 172.21.15.49:0/887451550 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf24023070 con 0x7fdf2c0b5d70 2023-07-18T03:40:03.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.582+0000 7fdf297fa700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdf1006ace0 0x7fdf1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:03.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.582+0000 7fdf297fa700 1 -- 172.21.15.49:0/887451550 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdf2408ab80 con 0x7fdf2c0b5d70 2023-07-18T03:40:03.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.582+0000 7fdf30f9b700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdf1006ace0 0x7fdf1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:03.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.583+0000 7fdf30f9b700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdf1006ace0 0x7fdf1006d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf20001260 tx=0x7fdf20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.586+0000 7fdf297fa700 1 -- 172.21.15.49:0/887451550 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdf2c0b3bd0 con 0x7fdf2c0b5d70 2023-07-18T03:40:03.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.790+0000 7fdf331ff700 1 -- 172.21.15.49:0/887451550 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf2c0b3ed0 con 0x7fdf1006ace0 2023-07-18T03:40:03.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.799+0000 7fdf297fa700 1 -- 172.21.15.49:0/887451550 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309001 (secure 0 0 0) 0x7fdf20012890 con 0x7fdf1006ace0 2023-07-18T03:40:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:03.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.804+0000 7fdf331ff700 1 -- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdf1006ace0 msgr2=0x7fdf1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.804+0000 7fdf331ff700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdf1006ace0 0x7fdf1006d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf20001260 tx=0x7fdf20009040 comp rx=0 tx=0).stop 2023-07-18T03:40:03.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.804+0000 7fdf331ff700 1 -- 172.21.15.49:0/887451550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf2c0b5d70 msgr2=0x7fdf2c0b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.804+0000 7fdf331ff700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf2c0b5d70 0x7fdf2c0b6210 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009f60 tx=0x7fdf24009750 comp rx=0 tx=0).stop 2023-07-18T03:40:03.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.804+0000 7fdf331ff700 1 -- 172.21.15.49:0/887451550 shutdown_connections 2023-07-18T03:40:03.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.804+0000 7fdf331ff700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdf1006ace0 0x7fdf1006d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.804+0000 7fdf331ff700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf2c0b5d70 0x7fdf2c0b6210 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.804+0000 7fdf331ff700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf1c004d90 0x7fdf2c14fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.804+0000 7fdf331ff700 1 --2- 172.21.15.49:0/887451550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf2c158b10 0x7fdf2c0b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.804+0000 7fdf331ff700 1 -- 172.21.15.49:0/887451550 >> 172.21.15.49:0/887451550 conn(0x7fdf2c09c1d0 msgr2=0x7fdf2c155b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:03.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.804+0000 7fdf331ff700 1 -- 172.21.15.49:0/887451550 shutdown_connections 2023-07-18T03:40:03.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:03.804+0000 7fdf331ff700 1 -- 172.21.15.49:0/887451550 wait complete. 2023-07-18T03:40:03.805 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:40:03.817 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-07-18T03:40:02.117452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591020,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":177,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236180,"kb_used_data":4180,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738540,"statfs":{"total":773094113280,"available":772852264960,"internally_reserved":0,"allocated":4280320,"data_stored":2437232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":652,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001415"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.337498+0000","last_change":"2023-07-18T03:39:56.337498+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.337498+0000","last_undegraded":"2023-07-18T03:39:56.337498+0000","last_fullsized":"2023-07-18T03:39:56.337498+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335988+0000","last_change":"2023-07-18T03:39:56.335988+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335988+0000","last_undegraded":"2023-07-18T03:39:56.335988+0000","last_fullsized":"2023-07-18T03:39:56.335988+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.352684+0000","last_change":"2023-07-18T03:39:57.349470+0000","last_active":"2023-07-18T03:39:57.352684+0000","last_peered":"2023-07-18T03:39:57.352684+0000","last_clean":"2023-07-18T03:39:57.352684+0000","last_became_active":"2023-07-18T03:39:57.349188+0000","last_became_peered":"2023-07-18T03:39:57.349188+0000","last_unstale":"2023-07-18T03:39:57.352684+0000","last_undegraded":"2023-07-18T03:39:57.352684+0000","last_fullsized":"2023-07-18T03:39:57.352684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:30:54.522092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.360068+0000","last_change":"2023-07-18T03:39:57.356421+0000","last_active":"2023-07-18T03:39:57.360068+0000","last_peered":"2023-07-18T03:39:57.360068+0000","last_clean":"2023-07-18T03:39:57.360068+0000","last_became_active":"2023-07-18T03:39:57.356128+0000","last_became_peered":"2023-07-18T03:39:57.356128+0000","last_unstale":"2023-07-18T03:39:57.360068+0000","last_undegraded":"2023-07-18T03:39:57.360068+0000","last_fullsized":"2023-07-18T03:39:57.360068+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:32:15.921500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359782+0000","last_change":"2023-07-18T03:39:57.356364+0000","last_active":"2023-07-18T03:39:57.359782+0000","last_peered":"2023-07-18T03:39:57.359782+0000","last_clean":"2023-07-18T03:39:57.359782+0000","last_became_active":"2023-07-18T03:39:57.356079+0000","last_became_peered":"2023-07-18T03:39:57.356079+0000","last_unstale":"2023-07-18T03:39:57.359782+0000","last_undegraded":"2023-07-18T03:39:57.359782+0000","last_fullsized":"2023-07-18T03:39:57.359782+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:08:41.904219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328919+0000","last_change":"2023-07-18T03:39:56.328919+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328919+0000","last_undegraded":"2023-07-18T03:39:56.328919+0000","last_fullsized":"2023-07-18T03:39:56.328919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332077+0000","last_change":"2023-07-18T03:39:56.332077+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332077+0000","last_undegraded":"2023-07-18T03:39:56.332077+0000","last_fullsized":"2023-07-18T03:39:56.332077+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335775+0000","last_change":"2023-07-18T03:39:56.335775+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335775+0000","last_undegraded":"2023-07-18T03:39:56.335775+0000","last_fullsized":"2023-07-18T03:39:56.335775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.357950+0000","last_change":"2023-07-18T03:39:57.352415+0000","last_active":"2023-07-18T03:39:57.357950+0000","last_peered":"2023-07-18T03:39:57.357950+0000","last_clean":"2023-07-18T03:39:57.357950+0000","last_became_active":"2023-07-18T03:39:57.352137+0000","last_became_peered":"2023-07-18T03:39:57.352137+0000","last_unstale":"2023-07-18T03:39:57.357950+0000","last_undegraded":"2023-07-18T03:39:57.357950+0000","last_fullsized":"2023-07-18T03:39:57.357950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:36.387384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.338378+0000","last_change":"2023-07-18T03:39:56.338378+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.338378+0000","last_undegraded":"2023-07-18T03:39:56.338378+0000","last_fullsized":"2023-07-18T03:39:56.338378+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.333492+0000","last_change":"2023-07-18T03:39:56.333492+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.333492+0000","last_undegraded":"2023-07-18T03:39:56.333492+0000","last_fullsized":"2023-07-18T03:39:56.333492+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.351146+0000","last_change":"2023-07-18T03:39:57.345772+0000","last_active":"2023-07-18T03:39:57.351146+0000","last_peered":"2023-07-18T03:39:57.351146+0000","last_clean":"2023-07-18T03:39:57.351146+0000","last_became_active":"2023-07-18T03:39:57.345465+0000","last_became_peered":"2023-07-18T03:39:57.345465+0000","last_unstale":"2023-07-18T03:39:57.351146+0000","last_undegraded":"2023-07-18T03:39:57.351146+0000","last_fullsized":"2023-07-18T03:39:57.351146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:48:30.748069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332641+0000","last_change":"2023-07-18T03:39:56.332641+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332641+0000","last_undegraded":"2023-07-18T03:39:56.332641+0000","last_fullsized":"2023-07-18T03:39:56.332641+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.327178+0000","last_change":"2023-07-18T03:39:56.327178+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.327178+0000","last_undegraded":"2023-07-18T03:39:56.327178+0000","last_fullsized":"2023-07-18T03:39:56.327178+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332947+0000","last_change":"2023-07-18T03:39:56.332947+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332947+0000","last_undegraded":"2023-07-18T03:39:56.332947+0000","last_fullsized":"2023-07-18T03:39:56.332947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.348720+0000","last_change":"2023-07-18T03:39:57.337636+0000","last_active":"2023-07-18T03:39:57.348720+0000","last_peered":"2023-07-18T03:39:57.348720+0000","last_clean":"2023-07-18T03:39:57.348720+0000","last_became_active":"2023-07-18T03:39:57.337330+0000","last_became_peered":"2023-07-18T03:39:57.337330+0000","last_unstale":"2023-07-18T03:39:57.348720+0000","last_undegraded":"2023-07-18T03:39:57.348720+0000","last_fullsized":"2023-07-18T03:39:57.348720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:22:11.047392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354699+0000","last_change":"2023-07-18T03:39:57.345108+0000","last_active":"2023-07-18T03:39:57.354699+0000","last_peered":"2023-07-18T03:39:57.354699+0000","last_clean":"2023-07-18T03:39:57.354699+0000","last_became_active":"2023-07-18T03:39:57.344825+0000","last_became_peered":"2023-07-18T03:39:57.344825+0000","last_unstale":"2023-07-18T03:39:57.354699+0000","last_undegraded":"2023-07-18T03:39:57.354699+0000","last_fullsized":"2023-07-18T03:39:57.354699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:07:59.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.365129+0000","last_change":"2023-07-18T03:39:57.361051+0000","last_active":"2023-07-18T03:39:57.365129+0000","last_peered":"2023-07-18T03:39:57.365129+0000","last_clean":"2023-07-18T03:39:57.365129+0000","last_became_active":"2023-07-18T03:39:57.360746+0000","last_became_peered":"2023-07-18T03:39:57.360746+0000","last_unstale":"2023-07-18T03:39:57.365129+0000","last_undegraded":"2023-07-18T03:39:57.365129+0000","last_fullsized":"2023-07-18T03:39:57.365129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:11:30.468969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359427+0000","last_change":"2023-07-18T03:39:57.351301+0000","last_active":"2023-07-18T03:39:57.359427+0000","last_peered":"2023-07-18T03:39:57.359427+0000","last_clean":"2023-07-18T03:39:57.359427+0000","last_became_active":"2023-07-18T03:39:57.351033+0000","last_became_peered":"2023-07-18T03:39:57.351033+0000","last_unstale":"2023-07-18T03:39:57.359427+0000","last_undegraded":"2023-07-18T03:39:57.359427+0000","last_fullsized":"2023-07-18T03:39:57.359427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:50:08.481909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.365031+0000","last_change":"2023-07-18T03:39:57.360471+0000","last_active":"2023-07-18T03:39:57.365031+0000","last_peered":"2023-07-18T03:39:57.365031+0000","last_clean":"2023-07-18T03:39:57.365031+0000","last_became_active":"2023-07-18T03:39:57.360202+0000","last_became_peered":"2023-07-18T03:39:57.360202+0000","last_unstale":"2023-07-18T03:39:57.365031+0000","last_undegraded":"2023-07-18T03:39:57.365031+0000","last_fullsized":"2023-07-18T03:39:57.365031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:32.300586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.331681+0000","last_change":"2023-07-18T03:39:56.331681+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.331681+0000","last_undegraded":"2023-07-18T03:39:56.331681+0000","last_fullsized":"2023-07-18T03:39:56.331681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.329011+0000","last_change":"2023-07-18T03:39:56.329011+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.329011+0000","last_undegraded":"2023-07-18T03:39:56.329011+0000","last_fullsized":"2023-07-18T03:39:56.329011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354754+0000","last_change":"2023-07-18T03:39:57.347725+0000","last_active":"2023-07-18T03:39:57.354754+0000","last_peered":"2023-07-18T03:39:57.354754+0000","last_clean":"2023-07-18T03:39:57.354754+0000","last_became_active":"2023-07-18T03:39:57.346990+0000","last_became_peered":"2023-07-18T03:39:57.346990+0000","last_unstale":"2023-07-18T03:39:57.354754+0000","last_undegraded":"2023-07-18T03:39:57.354754+0000","last_fullsized":"2023-07-18T03:39:57.354754+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:33:51.412210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.361028+0000","last_change":"2023-07-18T03:39:57.355821+0000","last_active":"2023-07-18T03:39:57.361028+0000","last_peered":"2023-07-18T03:39:57.361028+0000","last_clean":"2023-07-18T03:39:57.361028+0000","last_became_active":"2023-07-18T03:39:57.355505+0000","last_became_peered":"2023-07-18T03:39:57.355505+0000","last_unstale":"2023-07-18T03:39:57.361028+0000","last_undegraded":"2023-07-18T03:39:57.361028+0000","last_fullsized":"2023-07-18T03:39:57.361028+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:11:11.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.358089+0000","last_change":"2023-07-18T03:39:57.347320+0000","last_active":"2023-07-18T03:39:57.358089+0000","last_peered":"2023-07-18T03:39:57.358089+0000","last_clean":"2023-07-18T03:39:57.358089+0000","last_became_active":"2023-07-18T03:39:57.347051+0000","last_became_peered":"2023-07-18T03:39:57.347051+0000","last_unstale":"2023-07-18T03:39:57.358089+0000","last_undegraded":"2023-07-18T03:39:57.358089+0000","last_fullsized":"2023-07-18T03:39:57.358089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:45:31.179450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328947+0000","last_change":"2023-07-18T03:39:56.328947+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328947+0000","last_undegraded":"2023-07-18T03:39:56.328947+0000","last_fullsized":"2023-07-18T03:39:56.328947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.362856+0000","last_change":"2023-07-18T03:39:57.349732+0000","last_active":"2023-07-18T03:39:57.362856+0000","last_peered":"2023-07-18T03:39:57.362856+0000","last_clean":"2023-07-18T03:39:57.362856+0000","last_became_active":"2023-07-18T03:39:57.349421+0000","last_became_peered":"2023-07-18T03:39:57.349421+0000","last_unstale":"2023-07-18T03:39:57.362856+0000","last_undegraded":"2023-07-18T03:39:57.362856+0000","last_fullsized":"2023-07-18T03:39:57.362856+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:03:41.731722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.337384+0000","last_change":"2023-07-18T03:39:56.337384+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.337384+0000","last_undegraded":"2023-07-18T03:39:56.337384+0000","last_fullsized":"2023-07-18T03:39:56.337384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.362756+0000","last_change":"2023-07-18T03:39:57.354325+0000","last_active":"2023-07-18T03:39:57.362756+0000","last_peered":"2023-07-18T03:39:57.362756+0000","last_clean":"2023-07-18T03:39:57.362756+0000","last_became_active":"2023-07-18T03:39:57.353910+0000","last_became_peered":"2023-07-18T03:39:57.353910+0000","last_unstale":"2023-07-18T03:39:57.362756+0000","last_undegraded":"2023-07-18T03:39:57.362756+0000","last_fullsized":"2023-07-18T03:39:57.362756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:18:46.390308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.364995+0000","last_change":"2023-07-18T03:39:57.362161+0000","last_active":"2023-07-18T03:39:57.364995+0000","last_peered":"2023-07-18T03:39:57.364995+0000","last_clean":"2023-07-18T03:39:57.364995+0000","last_became_active":"2023-07-18T03:39:57.361882+0000","last_became_peered":"2023-07-18T03:39:57.361882+0000","last_unstale":"2023-07-18T03:39:57.364995+0000","last_undegraded":"2023-07-18T03:39:57.364995+0000","last_fullsized":"2023-07-18T03:39:57.364995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:39:10.257984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354777+0000","last_change":"2023-07-18T03:39:57.347923+0000","last_active":"2023-07-18T03:39:57.354777+0000","last_peered":"2023-07-18T03:39:57.354777+0000","last_clean":"2023-07-18T03:39:57.354777+0000","last_became_active":"2023-07-18T03:39:57.347419+0000","last_became_peered":"2023-07-18T03:39:57.347419+0000","last_unstale":"2023-07-18T03:39:57.354777+0000","last_undegraded":"2023-07-18T03:39:57.354777+0000","last_fullsized":"2023-07-18T03:39:57.354777+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:54:59.548413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328673+0000","last_change":"2023-07-18T03:39:56.328673+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328673+0000","last_undegraded":"2023-07-18T03:39:56.328673+0000","last_fullsized":"2023-07-18T03:39:56.328673+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.348817+0000","last_change":"2023-07-18T03:39:57.338235+0000","last_active":"2023-07-18T03:39:57.348817+0000","last_peered":"2023-07-18T03:39:57.348817+0000","last_clean":"2023-07-18T03:39:57.348817+0000","last_became_active":"2023-07-18T03:39:57.337940+0000","last_became_peered":"2023-07-18T03:39:57.337940+0000","last_unstale":"2023-07-18T03:39:57.348817+0000","last_undegraded":"2023-07-18T03:39:57.348817+0000","last_fullsized":"2023-07-18T03:39:57.348817+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:42:25.160054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.365698+0000","last_change":"2023-07-18T03:39:56.337657+0000","last_active":"2023-07-18T03:39:57.365698+0000","last_peered":"2023-07-18T03:39:57.365698+0000","last_clean":"2023-07-18T03:39:57.365698+0000","last_became_active":"2023-07-18T03:39:56.337379+0000","last_became_peered":"2023-07-18T03:39:56.337379+0000","last_unstale":"2023-07-18T03:39:57.365698+0000","last_undegraded":"2023-07-18T03:39:57.365698+0000","last_fullsized":"2023-07-18T03:39:57.365698+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:30.478922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.330756+0000","last_change":"2023-07-18T03:39:56.330756+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.330756+0000","last_undegraded":"2023-07-18T03:39:56.330756+0000","last_fullsized":"2023-07-18T03:39:56.330756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355305+0000","last_change":"2023-07-18T03:39:56.345284+0000","last_active":"2023-07-18T03:39:57.355305+0000","last_peered":"2023-07-18T03:39:57.355305+0000","last_clean":"2023-07-18T03:39:57.355305+0000","last_became_active":"2023-07-18T03:39:56.344955+0000","last_became_peered":"2023-07-18T03:39:56.344955+0000","last_unstale":"2023-07-18T03:39:57.355305+0000","last_undegraded":"2023-07-18T03:39:57.355305+0000","last_fullsized":"2023-07-18T03:39:57.355305+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:50:11.765103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328243+0000","last_change":"2023-07-18T03:39:56.328243+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328243+0000","last_undegraded":"2023-07-18T03:39:56.328243+0000","last_fullsized":"2023-07-18T03:39:56.328243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.345414+0000","last_change":"2023-07-18T03:39:56.335839+0000","last_active":"2023-07-18T03:39:56.345414+0000","last_peered":"2023-07-18T03:39:56.345414+0000","last_clean":"2023-07-18T03:39:56.345414+0000","last_became_active":"2023-07-18T03:39:56.335506+0000","last_became_peered":"2023-07-18T03:39:56.335506+0000","last_unstale":"2023-07-18T03:39:56.345414+0000","last_undegraded":"2023-07-18T03:39:56.345414+0000","last_fullsized":"2023-07-18T03:39:56.345414+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:55:30.319672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.338278+0000","last_change":"2023-07-18T03:39:56.338278+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.338278+0000","last_undegraded":"2023-07-18T03:39:56.338278+0000","last_fullsized":"2023-07-18T03:39:56.338278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.355959+0000","last_change":"2023-07-18T03:39:56.353462+0000","last_active":"2023-07-18T03:39:56.355959+0000","last_peered":"2023-07-18T03:39:56.355959+0000","last_clean":"2023-07-18T03:39:56.355959+0000","last_became_active":"2023-07-18T03:39:56.353165+0000","last_became_peered":"2023-07-18T03:39:56.353165+0000","last_unstale":"2023-07-18T03:39:56.355959+0000","last_undegraded":"2023-07-18T03:39:56.355959+0000","last_fullsized":"2023-07-18T03:39:56.355959+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:24.138081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.350712+0000","last_change":"2023-07-18T03:39:57.345776+0000","last_active":"2023-07-18T03:39:57.350712+0000","last_peered":"2023-07-18T03:39:57.350712+0000","last_clean":"2023-07-18T03:39:57.350712+0000","last_became_active":"2023-07-18T03:39:57.345450+0000","last_became_peered":"2023-07-18T03:39:57.345450+0000","last_unstale":"2023-07-18T03:39:57.350712+0000","last_undegraded":"2023-07-18T03:39:57.350712+0000","last_fullsized":"2023-07-18T03:39:57.350712+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:39:58.863405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.348456+0000","last_change":"2023-07-18T03:39:56.343054+0000","last_active":"2023-07-18T03:39:56.348456+0000","last_peered":"2023-07-18T03:39:56.348456+0000","last_clean":"2023-07-18T03:39:56.348456+0000","last_became_active":"2023-07-18T03:39:56.342752+0000","last_became_peered":"2023-07-18T03:39:56.342752+0000","last_unstale":"2023-07-18T03:39:56.348456+0000","last_undegraded":"2023-07-18T03:39:56.348456+0000","last_fullsized":"2023-07-18T03:39:56.348456+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:42:03.205955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.332552+0000","last_change":"2023-07-18T03:39:56.332552+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.332552+0000","last_undegraded":"2023-07-18T03:39:56.332552+0000","last_fullsized":"2023-07-18T03:39:56.332552+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.356389+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:57.356389+0000","last_peered":"2023-07-18T03:39:57.356389+0000","last_clean":"2023-07-18T03:39:57.356389+0000","last_became_active":"2023-07-18T03:39:56.345390+0000","last_became_peered":"2023-07-18T03:39:56.345390+0000","last_unstale":"2023-07-18T03:39:57.356389+0000","last_undegraded":"2023-07-18T03:39:57.356389+0000","last_fullsized":"2023-07-18T03:39:57.356389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:57:57.797956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328578+0000","last_change":"2023-07-18T03:39:56.328578+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328578+0000","last_undegraded":"2023-07-18T03:39:56.328578+0000","last_fullsized":"2023-07-18T03:39:56.328578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.347760+0000","last_change":"2023-07-18T03:39:56.346375+0000","last_active":"2023-07-18T03:39:56.347760+0000","last_peered":"2023-07-18T03:39:56.347760+0000","last_clean":"2023-07-18T03:39:56.347760+0000","last_became_active":"2023-07-18T03:39:56.346098+0000","last_became_peered":"2023-07-18T03:39:56.346098+0000","last_unstale":"2023-07-18T03:39:56.347760+0000","last_undegraded":"2023-07-18T03:39:56.347760+0000","last_fullsized":"2023-07-18T03:39:56.347760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:50:22.604564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.339195+0000","last_change":"2023-07-18T03:39:56.339195+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.339195+0000","last_undegraded":"2023-07-18T03:39:56.339195+0000","last_fullsized":"2023-07-18T03:39:56.339195+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.341115+0000","last_change":"2023-07-18T03:39:56.337739+0000","last_active":"2023-07-18T03:39:56.341115+0000","last_peered":"2023-07-18T03:39:56.341115+0000","last_clean":"2023-07-18T03:39:56.341115+0000","last_became_active":"2023-07-18T03:39:56.337495+0000","last_became_peered":"2023-07-18T03:39:56.337495+0000","last_unstale":"2023-07-18T03:39:56.341115+0000","last_undegraded":"2023-07-18T03:39:56.341115+0000","last_fullsized":"2023-07-18T03:39:56.341115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:01.077387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.360048+0000","last_change":"2023-07-18T03:39:57.356339+0000","last_active":"2023-07-18T03:39:57.360048+0000","last_peered":"2023-07-18T03:39:57.360048+0000","last_clean":"2023-07-18T03:39:57.360048+0000","last_became_active":"2023-07-18T03:39:57.356008+0000","last_became_peered":"2023-07-18T03:39:57.356008+0000","last_unstale":"2023-07-18T03:39:57.360048+0000","last_undegraded":"2023-07-18T03:39:57.360048+0000","last_fullsized":"2023-07-18T03:39:57.360048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:20:26.761581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354507+0000","last_change":"2023-07-18T03:39:56.352591+0000","last_active":"2023-07-18T03:39:56.354507+0000","last_peered":"2023-07-18T03:39:56.354507+0000","last_clean":"2023-07-18T03:39:56.354507+0000","last_became_active":"2023-07-18T03:39:56.352257+0000","last_became_peered":"2023-07-18T03:39:56.352257+0000","last_unstale":"2023-07-18T03:39:56.354507+0000","last_undegraded":"2023-07-18T03:39:56.354507+0000","last_fullsized":"2023-07-18T03:39:56.354507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:40:18.228711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.357301+0000","last_change":"2023-07-18T03:39:57.354324+0000","last_active":"2023-07-18T03:39:57.357301+0000","last_peered":"2023-07-18T03:39:57.357301+0000","last_clean":"2023-07-18T03:39:57.357301+0000","last_became_active":"2023-07-18T03:39:57.354028+0000","last_became_peered":"2023-07-18T03:39:57.354028+0000","last_unstale":"2023-07-18T03:39:57.357301+0000","last_undegraded":"2023-07-18T03:39:57.357301+0000","last_fullsized":"2023-07-18T03:39:57.357301+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:48:08.031962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354264+0000","last_change":"2023-07-18T03:39:56.353133+0000","last_active":"2023-07-18T03:39:56.354264+0000","last_peered":"2023-07-18T03:39:56.354264+0000","last_clean":"2023-07-18T03:39:56.354264+0000","last_became_active":"2023-07-18T03:39:56.352847+0000","last_became_peered":"2023-07-18T03:39:56.352847+0000","last_unstale":"2023-07-18T03:39:56.354264+0000","last_undegraded":"2023-07-18T03:39:56.354264+0000","last_fullsized":"2023-07-18T03:39:56.354264+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:21:01.203032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354872+0000","last_change":"2023-07-18T03:39:57.350961+0000","last_active":"2023-07-18T03:39:57.354872+0000","last_peered":"2023-07-18T03:39:57.354872+0000","last_clean":"2023-07-18T03:39:57.354872+0000","last_became_active":"2023-07-18T03:39:57.350643+0000","last_became_peered":"2023-07-18T03:39:57.350643+0000","last_unstale":"2023-07-18T03:39:57.354872+0000","last_undegraded":"2023-07-18T03:39:57.354872+0000","last_fullsized":"2023-07-18T03:39:57.354872+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:10:28.224008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.349402+0000","last_change":"2023-07-18T03:39:56.348133+0000","last_active":"2023-07-18T03:39:56.349402+0000","last_peered":"2023-07-18T03:39:56.349402+0000","last_clean":"2023-07-18T03:39:56.349402+0000","last_became_active":"2023-07-18T03:39:56.347930+0000","last_became_peered":"2023-07-18T03:39:56.347930+0000","last_unstale":"2023-07-18T03:39:56.349402+0000","last_undegraded":"2023-07-18T03:39:56.349402+0000","last_fullsized":"2023-07-18T03:39:56.349402+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:25:49.425965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.327915+0000","last_change":"2023-07-18T03:39:56.327915+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.327915+0000","last_undegraded":"2023-07-18T03:39:56.327915+0000","last_fullsized":"2023-07-18T03:39:56.327915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354562+0000","last_change":"2023-07-18T03:39:56.352639+0000","last_active":"2023-07-18T03:39:56.354562+0000","last_peered":"2023-07-18T03:39:56.354562+0000","last_clean":"2023-07-18T03:39:56.354562+0000","last_became_active":"2023-07-18T03:39:56.352134+0000","last_became_peered":"2023-07-18T03:39:56.352134+0000","last_unstale":"2023-07-18T03:39:56.354562+0000","last_undegraded":"2023-07-18T03:39:56.354562+0000","last_fullsized":"2023-07-18T03:39:56.354562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:26:35.806559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.352144+0000","last_change":"2023-07-18T03:39:57.349224+0000","last_active":"2023-07-18T03:39:57.352144+0000","last_peered":"2023-07-18T03:39:57.352144+0000","last_clean":"2023-07-18T03:39:57.352144+0000","last_became_active":"2023-07-18T03:39:57.348944+0000","last_became_peered":"2023-07-18T03:39:57.348944+0000","last_unstale":"2023-07-18T03:39:57.352144+0000","last_undegraded":"2023-07-18T03:39:57.352144+0000","last_fullsized":"2023-07-18T03:39:57.352144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:55:01.968006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354442+0000","last_change":"2023-07-18T03:39:56.352532+0000","last_active":"2023-07-18T03:39:56.354442+0000","last_peered":"2023-07-18T03:39:56.354442+0000","last_clean":"2023-07-18T03:39:56.354442+0000","last_became_active":"2023-07-18T03:39:56.352201+0000","last_became_peered":"2023-07-18T03:39:56.352201+0000","last_unstale":"2023-07-18T03:39:56.354442+0000","last_undegraded":"2023-07-18T03:39:56.354442+0000","last_fullsized":"2023-07-18T03:39:56.354442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:57:40.583880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.337060+0000","last_change":"2023-07-18T03:39:56.337060+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.337060+0000","last_undegraded":"2023-07-18T03:39:56.337060+0000","last_fullsized":"2023-07-18T03:39:56.337060+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.347811+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:56.347811+0000","last_peered":"2023-07-18T03:39:56.347811+0000","last_clean":"2023-07-18T03:39:56.347811+0000","last_became_active":"2023-07-18T03:39:56.345735+0000","last_became_peered":"2023-07-18T03:39:56.345735+0000","last_unstale":"2023-07-18T03:39:56.347811+0000","last_undegraded":"2023-07-18T03:39:56.347811+0000","last_fullsized":"2023-07-18T03:39:56.347811+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:49:19.376544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.354809+0000","last_change":"2023-07-18T03:39:57.347143+0000","last_active":"2023-07-18T03:39:57.354809+0000","last_peered":"2023-07-18T03:39:57.354809+0000","last_clean":"2023-07-18T03:39:57.354809+0000","last_became_active":"2023-07-18T03:39:57.346833+0000","last_became_peered":"2023-07-18T03:39:57.346833+0000","last_unstale":"2023-07-18T03:39:57.354809+0000","last_undegraded":"2023-07-18T03:39:57.354809+0000","last_fullsized":"2023-07-18T03:39:57.354809+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:26:07.789214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.339763+0000","last_change":"2023-07-18T03:39:56.339148+0000","last_active":"2023-07-18T03:39:57.339763+0000","last_peered":"2023-07-18T03:39:57.339763+0000","last_clean":"2023-07-18T03:39:57.339763+0000","last_became_active":"2023-07-18T03:39:56.338862+0000","last_became_peered":"2023-07-18T03:39:56.338862+0000","last_unstale":"2023-07-18T03:39:57.339763+0000","last_undegraded":"2023-07-18T03:39:57.339763+0000","last_fullsized":"2023-07-18T03:39:57.339763+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:14:22.006899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.340136+0000","last_change":"2023-07-18T03:39:56.340136+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.340136+0000","last_undegraded":"2023-07-18T03:39:56.340136+0000","last_fullsized":"2023-07-18T03:39:56.340136+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.364107+0000","last_change":"2023-07-18T03:39:56.345149+0000","last_active":"2023-07-18T03:39:57.364107+0000","last_peered":"2023-07-18T03:39:57.364107+0000","last_clean":"2023-07-18T03:39:57.364107+0000","last_became_active":"2023-07-18T03:39:56.344848+0000","last_became_peered":"2023-07-18T03:39:56.344848+0000","last_unstale":"2023-07-18T03:39:57.364107+0000","last_undegraded":"2023-07-18T03:39:57.364107+0000","last_fullsized":"2023-07-18T03:39:57.364107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:51:09.697807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335929+0000","last_change":"2023-07-18T03:39:56.335929+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335929+0000","last_undegraded":"2023-07-18T03:39:56.335929+0000","last_fullsized":"2023-07-18T03:39:56.335929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.349330+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:56.349330+0000","last_peered":"2023-07-18T03:39:56.349330+0000","last_clean":"2023-07-18T03:39:56.349330+0000","last_became_active":"2023-07-18T03:39:56.345797+0000","last_became_peered":"2023-07-18T03:39:56.345797+0000","last_unstale":"2023-07-18T03:39:56.349330+0000","last_undegraded":"2023-07-18T03:39:56.349330+0000","last_fullsized":"2023-07-18T03:39:56.349330+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:12.877614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.339214+0000","last_change":"2023-07-18T03:39:56.339214+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.339214+0000","last_undegraded":"2023-07-18T03:39:56.339214+0000","last_fullsized":"2023-07-18T03:39:56.339214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.350939+0000","last_change":"2023-07-18T03:39:56.349196+0000","last_active":"2023-07-18T03:39:56.350939+0000","last_peered":"2023-07-18T03:39:56.350939+0000","last_clean":"2023-07-18T03:39:56.350939+0000","last_became_active":"2023-07-18T03:39:56.347176+0000","last_became_peered":"2023-07-18T03:39:56.347176+0000","last_unstale":"2023-07-18T03:39:56.350939+0000","last_undegraded":"2023-07-18T03:39:56.350939+0000","last_fullsized":"2023-07-18T03:39:56.350939+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:09.700401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.327716+0000","last_change":"2023-07-18T03:39:56.327716+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.327716+0000","last_undegraded":"2023-07-18T03:39:56.327716+0000","last_fullsized":"2023-07-18T03:39:56.327716+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359507+0000","last_change":"2023-07-18T03:39:56.344722+0000","last_active":"2023-07-18T03:39:57.359507+0000","last_peered":"2023-07-18T03:39:57.359507+0000","last_clean":"2023-07-18T03:39:57.359507+0000","last_became_active":"2023-07-18T03:39:56.344439+0000","last_became_peered":"2023-07-18T03:39:56.344439+0000","last_unstale":"2023-07-18T03:39:57.359507+0000","last_undegraded":"2023-07-18T03:39:57.359507+0000","last_fullsized":"2023-07-18T03:39:57.359507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:47:42.656638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.366921+0000","last_change":"2023-07-18T03:39:57.363607+0000","last_active":"2023-07-18T03:39:57.366921+0000","last_peered":"2023-07-18T03:39:57.366921+0000","last_clean":"2023-07-18T03:39:57.366921+0000","last_became_active":"2023-07-18T03:39:57.363312+0000","last_became_peered":"2023-07-18T03:39:57.363312+0000","last_unstale":"2023-07-18T03:39:57.366921+0000","last_undegraded":"2023-07-18T03:39:57.366921+0000","last_fullsized":"2023-07-18T03:39:57.366921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:56:30.355299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.344690+0000","last_change":"2023-07-18T03:39:56.336359+0000","last_active":"2023-07-18T03:39:56.344690+0000","last_peered":"2023-07-18T03:39:56.344690+0000","last_clean":"2023-07-18T03:39:56.344690+0000","last_became_active":"2023-07-18T03:39:56.336094+0000","last_became_peered":"2023-07-18T03:39:56.336094+0000","last_unstale":"2023-07-18T03:39:56.344690+0000","last_undegraded":"2023-07-18T03:39:56.344690+0000","last_fullsized":"2023-07-18T03:39:56.344690+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:30.478922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355019+0000","last_change":"2023-07-18T03:39:57.348592+0000","last_active":"2023-07-18T03:39:57.355019+0000","last_peered":"2023-07-18T03:39:57.355019+0000","last_clean":"2023-07-18T03:39:57.355019+0000","last_became_active":"2023-07-18T03:39:57.348257+0000","last_became_peered":"2023-07-18T03:39:57.348257+0000","last_unstale":"2023-07-18T03:39:57.355019+0000","last_undegraded":"2023-07-18T03:39:57.355019+0000","last_fullsized":"2023-07-18T03:39:57.355019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:36:37.474323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.359688+0000","last_change":"2023-07-18T03:39:56.337863+0000","last_active":"2023-07-18T03:39:57.359688+0000","last_peered":"2023-07-18T03:39:57.359688+0000","last_clean":"2023-07-18T03:39:57.359688+0000","last_became_active":"2023-07-18T03:39:56.337561+0000","last_became_peered":"2023-07-18T03:39:56.337561+0000","last_unstale":"2023-07-18T03:39:57.359688+0000","last_undegraded":"2023-07-18T03:39:57.359688+0000","last_fullsized":"2023-07-18T03:39:57.359688+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:10.926722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.353775+0000","last_change":"2023-07-18T03:39:57.351197+0000","last_active":"2023-07-18T03:39:57.353775+0000","last_peered":"2023-07-18T03:39:57.353775+0000","last_clean":"2023-07-18T03:39:57.353775+0000","last_became_active":"2023-07-18T03:39:57.350814+0000","last_became_peered":"2023-07-18T03:39:57.350814+0000","last_unstale":"2023-07-18T03:39:57.353775+0000","last_undegraded":"2023-07-18T03:39:57.353775+0000","last_fullsized":"2023-07-18T03:39:57.353775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:06:35.294279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.339081+0000","last_change":"2023-07-18T03:39:56.336889+0000","last_active":"2023-07-18T03:39:57.339081+0000","last_peered":"2023-07-18T03:39:57.339081+0000","last_clean":"2023-07-18T03:39:57.339081+0000","last_became_active":"2023-07-18T03:39:56.336599+0000","last_became_peered":"2023-07-18T03:39:56.336599+0000","last_unstale":"2023-07-18T03:39:57.339081+0000","last_undegraded":"2023-07-18T03:39:57.339081+0000","last_fullsized":"2023-07-18T03:39:57.339081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:10.544538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.333821+0000","last_change":"2023-07-18T03:39:56.333821+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.333821+0000","last_undegraded":"2023-07-18T03:39:56.333821+0000","last_fullsized":"2023-07-18T03:39:56.333821+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.331084+0000","last_change":"2023-07-18T03:39:56.350386+0000","last_active":"2023-07-18T03:39:57.331084+0000","last_peered":"2023-07-18T03:39:57.331084+0000","last_clean":"2023-07-18T03:39:57.331084+0000","last_became_active":"2023-07-18T03:39:56.350046+0000","last_became_peered":"2023-07-18T03:39:56.350046+0000","last_unstale":"2023-07-18T03:39:57.331084+0000","last_undegraded":"2023-07-18T03:39:57.331084+0000","last_fullsized":"2023-07-18T03:39:57.331084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:57:37.239962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.339618+0000","last_change":"2023-07-18T03:39:56.339618+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.339618+0000","last_undegraded":"2023-07-18T03:39:56.339618+0000","last_fullsized":"2023-07-18T03:39:56.339618+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.338697+0000","last_change":"2023-07-18T03:39:56.338127+0000","last_active":"2023-07-18T03:39:57.338697+0000","last_peered":"2023-07-18T03:39:57.338697+0000","last_clean":"2023-07-18T03:39:57.338697+0000","last_became_active":"2023-07-18T03:39:56.337747+0000","last_became_peered":"2023-07-18T03:39:56.337747+0000","last_unstale":"2023-07-18T03:39:57.338697+0000","last_undegraded":"2023-07-18T03:39:57.338697+0000","last_fullsized":"2023-07-18T03:39:57.338697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:08:17.902329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.328126+0000","last_change":"2023-07-18T03:39:56.328126+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.328126+0000","last_undegraded":"2023-07-18T03:39:56.328126+0000","last_fullsized":"2023-07-18T03:39:56.328126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.355935+0000","last_change":"2023-07-18T03:39:56.353087+0000","last_active":"2023-07-18T03:39:56.355935+0000","last_peered":"2023-07-18T03:39:56.355935+0000","last_clean":"2023-07-18T03:39:56.355935+0000","last_became_active":"2023-07-18T03:39:56.352738+0000","last_became_peered":"2023-07-18T03:39:56.352738+0000","last_unstale":"2023-07-18T03:39:56.355935+0000","last_undegraded":"2023-07-18T03:39:56.355935+0000","last_fullsized":"2023-07-18T03:39:56.355935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:38:20.581738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.329523+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:39:56.329523+0000","last_peered":"2023-07-18T03:39:56.329523+0000","last_clean":"2023-07-18T03:39:56.329523+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:39:56.329523+0000","last_undegraded":"2023-07-18T03:39:56.329523+0000","last_fullsized":"2023-07-18T03:39:56.329523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:58:20.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355192+0000","last_change":"2023-07-18T03:39:57.348865+0000","last_active":"2023-07-18T03:39:57.355192+0000","last_peered":"2023-07-18T03:39:57.355192+0000","last_clean":"2023-07-18T03:39:57.355192+0000","last_became_active":"2023-07-18T03:39:57.348246+0000","last_became_peered":"2023-07-18T03:39:57.348246+0000","last_unstale":"2023-07-18T03:39:57.355192+0000","last_undegraded":"2023-07-18T03:39:57.355192+0000","last_fullsized":"2023-07-18T03:39:57.355192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:47:29.972115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.354659+0000","last_change":"2023-07-18T03:39:56.352671+0000","last_active":"2023-07-18T03:39:56.354659+0000","last_peered":"2023-07-18T03:39:56.354659+0000","last_clean":"2023-07-18T03:39:56.354659+0000","last_became_active":"2023-07-18T03:39:56.351865+0000","last_became_peered":"2023-07-18T03:39:56.351865+0000","last_unstale":"2023-07-18T03:39:56.354659+0000","last_undegraded":"2023-07-18T03:39:56.354659+0000","last_fullsized":"2023-07-18T03:39:56.354659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:01:05.415848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.342292+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:39:56.342292+0000","last_peered":"2023-07-18T03:39:56.342292+0000","last_clean":"2023-07-18T03:39:56.342292+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:39:56.342292+0000","last_undegraded":"2023-07-18T03:39:56.342292+0000","last_fullsized":"2023-07-18T03:39:56.342292+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:40:43.825533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.329925+0000","last_change":"2023-07-18T03:39:56.329925+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.329925+0000","last_undegraded":"2023-07-18T03:39:56.329925+0000","last_fullsized":"2023-07-18T03:39:56.329925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.349349+0000","last_change":"2023-07-18T03:39:56.347782+0000","last_active":"2023-07-18T03:39:56.349349+0000","last_peered":"2023-07-18T03:39:56.349349+0000","last_clean":"2023-07-18T03:39:56.349349+0000","last_became_active":"2023-07-18T03:39:56.347519+0000","last_became_peered":"2023-07-18T03:39:56.347519+0000","last_unstale":"2023-07-18T03:39:56.349349+0000","last_undegraded":"2023-07-18T03:39:56.349349+0000","last_fullsized":"2023-07-18T03:39:56.349349+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:25.713685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.332914+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:39:56.332914+0000","last_peered":"2023-07-18T03:39:56.332914+0000","last_clean":"2023-07-18T03:39:56.332914+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:39:56.332914+0000","last_undegraded":"2023-07-18T03:39:56.332914+0000","last_fullsized":"2023-07-18T03:39:56.332914+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:40:36.216166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.330424+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:39:57.330424+0000","last_peered":"2023-07-18T03:39:57.330424+0000","last_clean":"2023-07-18T03:39:57.330424+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:39:57.330424+0000","last_undegraded":"2023-07-18T03:39:57.330424+0000","last_fullsized":"2023-07-18T03:39:57.330424+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:22:09.535618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.355161+0000","last_change":"2023-07-18T03:39:57.350408+0000","last_active":"2023-07-18T03:39:57.355161+0000","last_peered":"2023-07-18T03:39:57.355161+0000","last_clean":"2023-07-18T03:39:57.355161+0000","last_became_active":"2023-07-18T03:39:57.350103+0000","last_became_peered":"2023-07-18T03:39:57.350103+0000","last_unstale":"2023-07-18T03:39:57.355161+0000","last_undegraded":"2023-07-18T03:39:57.355161+0000","last_fullsized":"2023-07-18T03:39:57.355161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:47:42.227277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.338099+0000","last_change":"2023-07-18T03:39:56.346081+0000","last_active":"2023-07-18T03:39:57.338099+0000","last_peered":"2023-07-18T03:39:57.338099+0000","last_clean":"2023-07-18T03:39:57.338099+0000","last_became_active":"2023-07-18T03:39:56.345762+0000","last_became_peered":"2023-07-18T03:39:56.345762+0000","last_unstale":"2023-07-18T03:39:57.338099+0000","last_undegraded":"2023-07-18T03:39:57.338099+0000","last_fullsized":"2023-07-18T03:39:57.338099+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:15:29.121017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":45,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.329508+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:39:57.329508+0000","last_peered":"2023-07-18T03:39:57.329508+0000","last_clean":"2023-07-18T03:39:57.329508+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:39:57.329508+0000","last_undegraded":"2023-07-18T03:39:57.329508+0000","last_fullsized":"2023-07-18T03:39:57.329508+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:13:09.287323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.336448+0000","last_change":"2023-07-18T03:39:56.336448+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.336448+0000","last_undegraded":"2023-07-18T03:39:56.336448+0000","last_fullsized":"2023-07-18T03:39:56.336448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.330231+0000","last_change":"2023-07-18T03:39:56.348881+0000","last_active":"2023-07-18T03:39:57.330231+0000","last_peered":"2023-07-18T03:39:57.330231+0000","last_clean":"2023-07-18T03:39:57.330231+0000","last_became_active":"2023-07-18T03:39:56.348622+0000","last_became_peered":"2023-07-18T03:39:56.348622+0000","last_unstale":"2023-07-18T03:39:57.330231+0000","last_undegraded":"2023-07-18T03:39:57.330231+0000","last_fullsized":"2023-07-18T03:39:57.330231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:53:21.359259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":45,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.330272+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:39:57.330272+0000","last_peered":"2023-07-18T03:39:57.330272+0000","last_clean":"2023-07-18T03:39:57.330272+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:39:57.330272+0000","last_undegraded":"2023-07-18T03:39:57.330272+0000","last_fullsized":"2023-07-18T03:39:57.330272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:39:12.642880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.335532+0000","last_change":"2023-07-18T03:39:56.335532+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.335532+0000","last_undegraded":"2023-07-18T03:39:56.335532+0000","last_fullsized":"2023-07-18T03:39:56.335532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.336107+0000","last_change":"2023-07-18T03:39:56.337331+0000","last_active":"2023-07-18T03:39:57.336107+0000","last_peered":"2023-07-18T03:39:57.336107+0000","last_clean":"2023-07-18T03:39:57.336107+0000","last_became_active":"2023-07-18T03:39:56.337021+0000","last_became_peered":"2023-07-18T03:39:56.337021+0000","last_unstale":"2023-07-18T03:39:57.336107+0000","last_undegraded":"2023-07-18T03:39:57.336107+0000","last_fullsized":"2023-07-18T03:39:57.336107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:22:50.209633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-18T03:39:56.340413+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:39:56.340413+0000","last_peered":"2023-07-18T03:39:56.340413+0000","last_clean":"2023-07-18T03:39:56.340413+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:39:56.340413+0000","last_undegraded":"2023-07-18T03:39:56.340413+0000","last_fullsized":"2023-07-18T03:39:56.340413+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:45:16.320741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-07-18T03:39:56.329526+0000","last_change":"2023-07-18T03:39:56.329526+0000","last_active":"2023-07-18T03:39:56.318026+0000","last_peered":"2023-07-18T03:39:56.318026+0000","last_clean":"2023-07-18T03:39:56.318026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-18T03:39:56.329526+0000","last_undegraded":"2023-07-18T03:39:56.329526+0000","last_fullsized":"2023-07-18T03:39:56.329526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.340100+0000","last_change":"2023-07-18T03:39:56.341357+0000","last_active":"2023-07-18T03:39:57.340100+0000","last_peered":"2023-07-18T03:39:57.340100+0000","last_clean":"2023-07-18T03:39:57.340100+0000","last_became_active":"2023-07-18T03:39:56.341092+0000","last_became_peered":"2023-07-18T03:39:56.341092+0000","last_unstale":"2023-07-18T03:39:57.340100+0000","last_undegraded":"2023-07-18T03:39:57.340100+0000","last_fullsized":"2023-07-18T03:39:57.340100+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:48:52.596611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.339973+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:39:57.339973+0000","last_peered":"2023-07-18T03:39:57.339973+0000","last_clean":"2023-07-18T03:39:57.339973+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:39:57.339973+0000","last_undegraded":"2023-07-18T03:39:57.339973+0000","last_fullsized":"2023-07-18T03:39:57.339973+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:48:55.614775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.363216+0000","last_change":"2023-07-18T03:39:57.360948+0000","last_active":"2023-07-18T03:39:57.363216+0000","last_peered":"2023-07-18T03:39:57.363216+0000","last_clean":"2023-07-18T03:39:57.363216+0000","last_became_active":"2023-07-18T03:39:57.360601+0000","last_became_peered":"2023-07-18T03:39:57.360601+0000","last_unstale":"2023-07-18T03:39:57.363216+0000","last_undegraded":"2023-07-18T03:39:57.363216+0000","last_fullsized":"2023-07-18T03:39:57.363216+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:49:44.089477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.336599+0000","last_change":"2023-07-18T03:39:56.342369+0000","last_active":"2023-07-18T03:39:57.336599+0000","last_peered":"2023-07-18T03:39:57.336599+0000","last_clean":"2023-07-18T03:39:57.336599+0000","last_became_active":"2023-07-18T03:39:56.342108+0000","last_became_peered":"2023-07-18T03:39:56.342108+0000","last_unstale":"2023-07-18T03:39:57.336599+0000","last_undegraded":"2023-07-18T03:39:57.336599+0000","last_fullsized":"2023-07-18T03:39:57.336599+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:44.715606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-18T03:39:57.340446+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:39:57.340446+0000","last_peered":"2023-07-18T03:39:57.340446+0000","last_clean":"2023-07-18T03:39:57.340446+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:39:57.340446+0000","last_undegraded":"2023-07-18T03:39:57.340446+0000","last_fullsized":"2023-07-18T03:39:57.340446+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:54:03.329884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":652,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":999424,"data_stored":752338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":1.7729999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":2.2130000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":3.0030000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.2290000000000001,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":2.2570000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.0249999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":4.0010000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":1.2889999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":2.2989999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":3.2029999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":3.1219999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":1.0609999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":393216,"data_stored":160918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.626}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.72899999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.69299999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67800000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.82499999999999996}]}]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":376832,"data_stored":153208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52600000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.8290000000000002,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.747},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.80400000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":970752,"data_stored":743576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32200000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.495}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79000000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.73899999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":376832,"data_stored":153208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.82399999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.78500000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64700000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.75800000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.68899999999999995}]}]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":376832,"data_stored":153208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":4.9260000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":2.1699999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":8.3780000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":5.5060000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":4.8769999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":2.1230000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":2.2370000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":1.6599999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":4.9690000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":3.6829999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":5.3140000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":3.7389999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.71799999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":393216,"data_stored":160370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":8.3040000000000003,"5min":8.3040000000000003,"15min":8.3040000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.502}]},{"osd":3,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.0229999999999997,"5min":8.0229999999999997,"15min":8.0229999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":8.359,"5min":8.359,"15min":8.359},"last":0.48099999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.89900000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.78300000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.88200000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.97399999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":393216,"data_stored":160406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39100000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.438}]},{"osd":3,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.78100000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.79600000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.60499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:40:06.825 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:40:07.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.046+0000 7f4218bff700 1 Processor -- start 2023-07-18T03:40:07.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.046+0000 7f4218bff700 1 -- start start 2023-07-18T03:40:07.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.047+0000 7f4218bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f421415c950 0x7f42140b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f4218bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f42140d1220 0x7f42140afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f4218bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42140d2ce0 con 0x7f42140b1e00 2023-07-18T03:40:07.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f4218bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42140b3350 con 0x7f42140d1220 2023-07-18T03:40:07.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f4218bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42140b3630 con 0x7f421415c950 2023-07-18T03:40:07.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f4212d9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f42140d1220 0x7f42140afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f420bfff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f42140b1e00 0x7f42140d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56866/0 (socket says 172.21.15.49:56866) 2023-07-18T03:40:07.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f420bfff700 1 -- 172.21.15.49:0/545983980 learned_addr learned my addr 172.21.15.49:0/545983980 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:07.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f421259c700 1 --2- 172.21.15.49:0/545983980 >> v2:172.21.15.107:3300/0 conn(0x7f421415c950 0x7f42140b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f4212d9d700 1 -- 172.21.15.49:0/545983980 >> v2:172.21.15.107:3300/0 conn(0x7f421415c950 msgr2=0x7f42140b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:07.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f4212d9d700 1 --2- 172.21.15.49:0/545983980 >> v2:172.21.15.107:3300/0 conn(0x7f421415c950 0x7f42140b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f4212d9d700 1 -- 172.21.15.49:0/545983980 >> v1:172.21.15.49:6789/0 conn(0x7f42140b1e00 legacy=0x7f42140d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f4212d9d700 1 -- 172.21.15.49:0/545983980 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42140b3910 con 0x7f42140d1220 2023-07-18T03:40:07.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.048+0000 7f4211d9b700 1 -- 172.21.15.49:0/545983980 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1873023842 0 0) 0x7f42140d2ce0 con 0x7f42140b1e00 2023-07-18T03:40:07.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.049+0000 7f4212d9d700 1 --2- 172.21.15.49:0/545983980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42140d1220 0x7f42140afac0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f421408d170 tx=0x7f420c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2969b42a8fc08190 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:07.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.049+0000 7f4211d9b700 1 -- 172.21.15.49:0/545983980 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c00e070 con 0x7f42140d1220 2023-07-18T03:40:07.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.049+0000 7f4211d9b700 1 -- 172.21.15.49:0/545983980 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c00bbd0 con 0x7f42140d1220 2023-07-18T03:40:07.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.049+0000 7f4211d9b700 1 -- 172.21.15.49:0/545983980 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c012430 con 0x7f42140d1220 2023-07-18T03:40:07.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.049+0000 7f4218bff700 1 -- 172.21.15.49:0/545983980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42140d1220 msgr2=0x7f42140afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.049+0000 7f4218bff700 1 --2- 172.21.15.49:0/545983980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42140d1220 0x7f42140afac0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f421408d170 tx=0x7f420c00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:07.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.050+0000 7f4218bff700 1 -- 172.21.15.49:0/545983980 shutdown_connections 2023-07-18T03:40:07.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.050+0000 7f4218bff700 1 --2- 172.21.15.49:0/545983980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42140d1220 0x7f42140afac0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.050+0000 7f4218bff700 1 --2- 172.21.15.49:0/545983980 >> v2:172.21.15.107:3300/0 conn(0x7f421415c950 0x7f42140b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.050+0000 7f4218bff700 1 -- 172.21.15.49:0/545983980 >> 172.21.15.49:0/545983980 conn(0x7f421409c1d0 msgr2=0x7f42140d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:07.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.050+0000 7f4218bff700 1 -- 172.21.15.49:0/545983980 shutdown_connections 2023-07-18T03:40:07.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.050+0000 7f4218bff700 1 -- 172.21.15.49:0/545983980 wait complete. 2023-07-18T03:40:07.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.052+0000 7f4218bff700 1 Processor -- start 2023-07-18T03:40:07.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.052+0000 7f4218bff700 1 -- start start 2023-07-18T03:40:07.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f4218bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42140b1e00 0x7f42140b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f4218bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42140d1220 0x7f42140cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f4218bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f421415c950 0x7f42140b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f4218bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42140ca010 con 0x7f42140b1e00 2023-07-18T03:40:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f4218bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42140ca5d0 con 0x7f42140d1220 2023-07-18T03:40:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f4218bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42141518a0 con 0x7f421415c950 2023-07-18T03:40:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f421259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42140b1e00 0x7f42140b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f421259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42140b1e00 0x7f42140b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38512/0 (socket says 172.21.15.49:38512) 2023-07-18T03:40:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f421259c700 1 -- 172.21.15.49:0/2656780286 learned_addr learned my addr 172.21.15.49:0/2656780286 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f420bfff700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42140d1220 0x7f42140cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f4212d9d700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f421415c950 0x7f42140b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f421259c700 1 -- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f421415c950 msgr2=0x7f42140b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f421259c700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f421415c950 0x7f42140b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f421259c700 1 -- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42140d1220 msgr2=0x7f42140cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f421259c700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42140d1220 0x7f42140cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f421259c700 1 -- 172.21.15.49:0/2656780286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c009a70 con 0x7f42140b1e00 2023-07-18T03:40:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.054+0000 7f4212d9d700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f421415c950 0x7f42140b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:07.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.055+0000 7f421259c700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42140b1e00 0x7f42140b8810 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f420400d800 tx=0x7f420400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:07.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.055+0000 7f420affd700 1 -- 172.21.15.49:0/2656780286 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42040125f0 con 0x7f42140b1e00 2023-07-18T03:40:07.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.055+0000 7f420affd700 1 -- 172.21.15.49:0/2656780286 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4204012750 con 0x7f42140b1e00 2023-07-18T03:40:07.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.055+0000 7f4218bff700 1 -- 172.21.15.49:0/2656780286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42140cf930 con 0x7f42140b1e00 2023-07-18T03:40:07.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.055+0000 7f420affd700 1 -- 172.21.15.49:0/2656780286 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204012b00 con 0x7f42140b1e00 2023-07-18T03:40:07.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.055+0000 7f4218bff700 1 -- 172.21.15.49:0/2656780286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42140ca260 con 0x7f42140b1e00 2023-07-18T03:40:07.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.056+0000 7f4218bff700 1 -- 172.21.15.49:0/2656780286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42140ca8b0 con 0x7f42140b1e00 2023-07-18T03:40:07.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.057+0000 7f420affd700 1 -- 172.21.15.49:0/2656780286 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4204012c60 con 0x7f42140b1e00 2023-07-18T03:40:07.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.058+0000 7f420affd700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f41f406ace0 0x7f41f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.058+0000 7f420affd700 1 -- 172.21.15.49:0/2656780286 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4204089370 con 0x7f42140b1e00 2023-07-18T03:40:07.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.059+0000 7f420bfff700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f41f406ace0 0x7f41f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.059+0000 7f420bfff700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f41f406ace0 0x7f41f406d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f42141e7230 tx=0x7f41fc007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:07.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.063+0000 7f420affd700 1 -- 172.21.15.49:0/2656780286 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4204054410 con 0x7f42140b1e00 2023-07-18T03:40:07.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.266+0000 7f4218bff700 1 -- 172.21.15.49:0/2656780286 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42140b3ed0 con 0x7f41f406ace0 2023-07-18T03:40:07.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.276+0000 7f420affd700 1 -- 172.21.15.49:0/2656780286 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311704 (secure 0 0 0) 0x7f41fc001370 con 0x7f41f406ace0 2023-07-18T03:40:07.274 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:07.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.280+0000 7f4218bff700 1 -- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f41f406ace0 msgr2=0x7f41f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.280+0000 7f4218bff700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f41f406ace0 0x7f41f406d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f42141e7230 tx=0x7f41fc007de0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.280+0000 7f4218bff700 1 -- 172.21.15.49:0/2656780286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42140b1e00 msgr2=0x7f42140b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.280+0000 7f4218bff700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42140b1e00 0x7f42140b8810 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f420400d800 tx=0x7f420400db10 comp rx=0 tx=0).stop 2023-07-18T03:40:07.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.280+0000 7f4218bff700 1 -- 172.21.15.49:0/2656780286 shutdown_connections 2023-07-18T03:40:07.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.280+0000 7f4218bff700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f421415c950 0x7f42140b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.280+0000 7f4218bff700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f41f406ace0 0x7f41f406d190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.280+0000 7f4218bff700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42140b1e00 0x7f42140b8810 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.280+0000 7f4218bff700 1 --2- 172.21.15.49:0/2656780286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42140d1220 0x7f42140cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.280+0000 7f4218bff700 1 -- 172.21.15.49:0/2656780286 >> 172.21.15.49:0/2656780286 conn(0x7f421409c1d0 msgr2=0x7f42140afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:07.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.280+0000 7f4218bff700 1 -- 172.21.15.49:0/2656780286 shutdown_connections 2023-07-18T03:40:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.280+0000 7f4218bff700 1 -- 172.21.15.49:0/2656780286 wait complete. 2023-07-18T03:40:07.278 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:40:07.293 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-07-18T03:40:06.118502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236360,"kb_used_data":4360,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738360,"statfs":{"total":773094113280,"available":772852080640,"internally_reserved":0,"allocated":4464640,"data_stored":2521372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.792131"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.437276+0000","last_change":"2023-07-18T03:39:57.362248+0000","last_active":"2023-07-18T03:39:58.437276+0000","last_peered":"2023-07-18T03:39:58.437276+0000","last_clean":"2023-07-18T03:39:58.437276+0000","last_became_active":"2023-07-18T03:39:57.361959+0000","last_became_peered":"2023-07-18T03:39:57.361959+0000","last_unstale":"2023-07-18T03:39:58.437276+0000","last_undegraded":"2023-07-18T03:39:58.437276+0000","last_fullsized":"2023-07-18T03:39:58.437276+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:27:17.588955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.437038+0000","last_change":"2023-07-18T03:39:57.360885+0000","last_active":"2023-07-18T03:39:58.437038+0000","last_peered":"2023-07-18T03:39:58.437038+0000","last_clean":"2023-07-18T03:39:58.437038+0000","last_became_active":"2023-07-18T03:39:57.360622+0000","last_became_peered":"2023-07-18T03:39:57.360622+0000","last_unstale":"2023-07-18T03:39:58.437038+0000","last_undegraded":"2023-07-18T03:39:58.437038+0000","last_fullsized":"2023-07-18T03:39:58.437038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:38:21.588874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436509+0000","last_change":"2023-07-18T03:39:57.349470+0000","last_active":"2023-07-18T03:39:58.436509+0000","last_peered":"2023-07-18T03:39:58.436509+0000","last_clean":"2023-07-18T03:39:58.436509+0000","last_became_active":"2023-07-18T03:39:57.349188+0000","last_became_peered":"2023-07-18T03:39:57.349188+0000","last_unstale":"2023-07-18T03:39:58.436509+0000","last_undegraded":"2023-07-18T03:39:58.436509+0000","last_fullsized":"2023-07-18T03:39:58.436509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:14:36.970643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436489+0000","last_change":"2023-07-18T03:39:57.356421+0000","last_active":"2023-07-18T03:39:58.436489+0000","last_peered":"2023-07-18T03:39:58.436489+0000","last_clean":"2023-07-18T03:39:58.436489+0000","last_became_active":"2023-07-18T03:39:57.356128+0000","last_became_peered":"2023-07-18T03:39:57.356128+0000","last_unstale":"2023-07-18T03:39:58.436489+0000","last_undegraded":"2023-07-18T03:39:58.436489+0000","last_fullsized":"2023-07-18T03:39:58.436489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:20:01.043105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436026+0000","last_change":"2023-07-18T03:39:57.356364+0000","last_active":"2023-07-18T03:39:58.436026+0000","last_peered":"2023-07-18T03:39:58.436026+0000","last_clean":"2023-07-18T03:39:58.436026+0000","last_became_active":"2023-07-18T03:39:57.356079+0000","last_became_peered":"2023-07-18T03:39:57.356079+0000","last_unstale":"2023-07-18T03:39:58.436026+0000","last_undegraded":"2023-07-18T03:39:58.436026+0000","last_fullsized":"2023-07-18T03:39:58.436026+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:03:08.812742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.350786+0000","last_change":"2023-07-18T03:39:57.364482+0000","last_active":"2023-07-18T03:39:58.350786+0000","last_peered":"2023-07-18T03:39:58.350786+0000","last_clean":"2023-07-18T03:39:58.350786+0000","last_became_active":"2023-07-18T03:39:57.364161+0000","last_became_peered":"2023-07-18T03:39:57.364161+0000","last_unstale":"2023-07-18T03:39:58.350786+0000","last_undegraded":"2023-07-18T03:39:58.350786+0000","last_fullsized":"2023-07-18T03:39:58.350786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:18:34.135145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435841+0000","last_change":"2023-07-18T03:39:57.361910+0000","last_active":"2023-07-18T03:39:58.435841+0000","last_peered":"2023-07-18T03:39:58.435841+0000","last_clean":"2023-07-18T03:39:58.435841+0000","last_became_active":"2023-07-18T03:39:57.361546+0000","last_became_peered":"2023-07-18T03:39:57.361546+0000","last_unstale":"2023-07-18T03:39:58.435841+0000","last_undegraded":"2023-07-18T03:39:58.435841+0000","last_fullsized":"2023-07-18T03:39:58.435841+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:14:37.284247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435986+0000","last_change":"2023-07-18T03:39:57.360261+0000","last_active":"2023-07-18T03:39:58.435986+0000","last_peered":"2023-07-18T03:39:58.435986+0000","last_clean":"2023-07-18T03:39:58.435986+0000","last_became_active":"2023-07-18T03:39:57.359856+0000","last_became_peered":"2023-07-18T03:39:57.359856+0000","last_unstale":"2023-07-18T03:39:58.435986+0000","last_undegraded":"2023-07-18T03:39:58.435986+0000","last_fullsized":"2023-07-18T03:39:58.435986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:47:29.972115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436689+0000","last_change":"2023-07-18T03:39:57.352415+0000","last_active":"2023-07-18T03:39:58.436689+0000","last_peered":"2023-07-18T03:39:58.436689+0000","last_clean":"2023-07-18T03:39:58.436689+0000","last_became_active":"2023-07-18T03:39:57.352137+0000","last_became_peered":"2023-07-18T03:39:57.352137+0000","last_unstale":"2023-07-18T03:39:58.436689+0000","last_undegraded":"2023-07-18T03:39:58.436689+0000","last_fullsized":"2023-07-18T03:39:58.436689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:28:04.418088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436092+0000","last_change":"2023-07-18T03:39:57.339483+0000","last_active":"2023-07-18T03:39:58.436092+0000","last_peered":"2023-07-18T03:39:58.436092+0000","last_clean":"2023-07-18T03:39:58.436092+0000","last_became_active":"2023-07-18T03:39:57.339191+0000","last_became_peered":"2023-07-18T03:39:57.339191+0000","last_unstale":"2023-07-18T03:39:58.436092+0000","last_undegraded":"2023-07-18T03:39:58.436092+0000","last_fullsized":"2023-07-18T03:39:58.436092+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:36:37.474323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436267+0000","last_change":"2023-07-18T03:39:57.358531+0000","last_active":"2023-07-18T03:39:58.436267+0000","last_peered":"2023-07-18T03:39:58.436267+0000","last_clean":"2023-07-18T03:39:58.436267+0000","last_became_active":"2023-07-18T03:39:57.357967+0000","last_became_peered":"2023-07-18T03:39:57.357967+0000","last_unstale":"2023-07-18T03:39:58.436267+0000","last_undegraded":"2023-07-18T03:39:58.436267+0000","last_fullsized":"2023-07-18T03:39:58.436267+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:40:40.639805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436317+0000","last_change":"2023-07-18T03:39:57.345772+0000","last_active":"2023-07-18T03:39:58.436317+0000","last_peered":"2023-07-18T03:39:58.436317+0000","last_clean":"2023-07-18T03:39:58.436317+0000","last_became_active":"2023-07-18T03:39:57.345465+0000","last_became_peered":"2023-07-18T03:39:57.345465+0000","last_unstale":"2023-07-18T03:39:58.436317+0000","last_undegraded":"2023-07-18T03:39:58.436317+0000","last_fullsized":"2023-07-18T03:39:58.436317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:27:14.329623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435736+0000","last_change":"2023-07-18T03:39:57.345468+0000","last_active":"2023-07-18T03:39:58.435736+0000","last_peered":"2023-07-18T03:39:58.435736+0000","last_clean":"2023-07-18T03:39:58.435736+0000","last_became_active":"2023-07-18T03:39:57.344801+0000","last_became_peered":"2023-07-18T03:39:57.344801+0000","last_unstale":"2023-07-18T03:39:58.435736+0000","last_undegraded":"2023-07-18T03:39:58.435736+0000","last_fullsized":"2023-07-18T03:39:58.435736+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:44:19.717523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353754+0000","last_change":"2023-07-18T03:39:57.359845+0000","last_active":"2023-07-18T03:39:58.353754+0000","last_peered":"2023-07-18T03:39:58.353754+0000","last_clean":"2023-07-18T03:39:58.353754+0000","last_became_active":"2023-07-18T03:39:57.359520+0000","last_became_peered":"2023-07-18T03:39:57.359520+0000","last_unstale":"2023-07-18T03:39:58.353754+0000","last_undegraded":"2023-07-18T03:39:58.353754+0000","last_fullsized":"2023-07-18T03:39:58.353754+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:56:20.592306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435329+0000","last_change":"2023-07-18T03:39:57.347532+0000","last_active":"2023-07-18T03:39:58.435329+0000","last_peered":"2023-07-18T03:39:58.435329+0000","last_clean":"2023-07-18T03:39:58.435329+0000","last_became_active":"2023-07-18T03:39:57.347037+0000","last_became_peered":"2023-07-18T03:39:57.347037+0000","last_unstale":"2023-07-18T03:39:58.435329+0000","last_undegraded":"2023-07-18T03:39:58.435329+0000","last_fullsized":"2023-07-18T03:39:58.435329+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:42:11.822457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.352867+0000","last_change":"2023-07-18T03:39:57.337636+0000","last_active":"2023-07-18T03:39:58.352867+0000","last_peered":"2023-07-18T03:39:58.352867+0000","last_clean":"2023-07-18T03:39:58.352867+0000","last_became_active":"2023-07-18T03:39:57.337330+0000","last_became_peered":"2023-07-18T03:39:57.337330+0000","last_unstale":"2023-07-18T03:39:58.352867+0000","last_undegraded":"2023-07-18T03:39:58.352867+0000","last_fullsized":"2023-07-18T03:39:58.352867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:25.145218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.354184+0000","last_change":"2023-07-18T03:39:57.345108+0000","last_active":"2023-07-18T03:39:58.354184+0000","last_peered":"2023-07-18T03:39:58.354184+0000","last_clean":"2023-07-18T03:39:58.354184+0000","last_became_active":"2023-07-18T03:39:57.344825+0000","last_became_peered":"2023-07-18T03:39:57.344825+0000","last_unstale":"2023-07-18T03:39:58.354184+0000","last_undegraded":"2023-07-18T03:39:58.354184+0000","last_fullsized":"2023-07-18T03:39:58.354184+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:07:23.780289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436398+0000","last_change":"2023-07-18T03:39:57.361051+0000","last_active":"2023-07-18T03:39:58.436398+0000","last_peered":"2023-07-18T03:39:58.436398+0000","last_clean":"2023-07-18T03:39:58.436398+0000","last_became_active":"2023-07-18T03:39:57.360746+0000","last_became_peered":"2023-07-18T03:39:57.360746+0000","last_unstale":"2023-07-18T03:39:58.436398+0000","last_undegraded":"2023-07-18T03:39:58.436398+0000","last_fullsized":"2023-07-18T03:39:58.436398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:08:55.975598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436411+0000","last_change":"2023-07-18T03:39:57.351301+0000","last_active":"2023-07-18T03:39:58.436411+0000","last_peered":"2023-07-18T03:39:58.436411+0000","last_clean":"2023-07-18T03:39:58.436411+0000","last_became_active":"2023-07-18T03:39:57.351033+0000","last_became_peered":"2023-07-18T03:39:57.351033+0000","last_unstale":"2023-07-18T03:39:58.436411+0000","last_undegraded":"2023-07-18T03:39:58.436411+0000","last_fullsized":"2023-07-18T03:39:58.436411+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:23:30.249906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436251+0000","last_change":"2023-07-18T03:39:57.360471+0000","last_active":"2023-07-18T03:39:58.436251+0000","last_peered":"2023-07-18T03:39:58.436251+0000","last_clean":"2023-07-18T03:39:58.436251+0000","last_became_active":"2023-07-18T03:39:57.360202+0000","last_became_peered":"2023-07-18T03:39:57.360202+0000","last_unstale":"2023-07-18T03:39:58.436251+0000","last_undegraded":"2023-07-18T03:39:58.436251+0000","last_fullsized":"2023-07-18T03:39:58.436251+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:26.720821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435949+0000","last_change":"2023-07-18T03:39:57.345680+0000","last_active":"2023-07-18T03:39:58.435949+0000","last_peered":"2023-07-18T03:39:58.435949+0000","last_clean":"2023-07-18T03:39:58.435949+0000","last_became_active":"2023-07-18T03:39:57.345166+0000","last_became_peered":"2023-07-18T03:39:57.345166+0000","last_unstale":"2023-07-18T03:39:58.435949+0000","last_undegraded":"2023-07-18T03:39:58.435949+0000","last_fullsized":"2023-07-18T03:39:58.435949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:54:59.548413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353384+0000","last_change":"2023-07-18T03:39:57.364189+0000","last_active":"2023-07-18T03:39:58.353384+0000","last_peered":"2023-07-18T03:39:58.353384+0000","last_clean":"2023-07-18T03:39:58.353384+0000","last_became_active":"2023-07-18T03:39:57.363818+0000","last_became_peered":"2023-07-18T03:39:57.363818+0000","last_unstale":"2023-07-18T03:39:58.353384+0000","last_undegraded":"2023-07-18T03:39:58.353384+0000","last_fullsized":"2023-07-18T03:39:58.353384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:28:17.397234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353690+0000","last_change":"2023-07-18T03:39:57.347725+0000","last_active":"2023-07-18T03:39:58.353690+0000","last_peered":"2023-07-18T03:39:58.353690+0000","last_clean":"2023-07-18T03:39:58.353690+0000","last_became_active":"2023-07-18T03:39:57.346990+0000","last_became_peered":"2023-07-18T03:39:57.346990+0000","last_unstale":"2023-07-18T03:39:58.353690+0000","last_undegraded":"2023-07-18T03:39:58.353690+0000","last_fullsized":"2023-07-18T03:39:58.353690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:52:20.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435527+0000","last_change":"2023-07-18T03:39:57.355821+0000","last_active":"2023-07-18T03:39:58.435527+0000","last_peered":"2023-07-18T03:39:58.435527+0000","last_clean":"2023-07-18T03:39:58.435527+0000","last_became_active":"2023-07-18T03:39:57.355505+0000","last_became_peered":"2023-07-18T03:39:57.355505+0000","last_unstale":"2023-07-18T03:39:58.435527+0000","last_undegraded":"2023-07-18T03:39:58.435527+0000","last_fullsized":"2023-07-18T03:39:58.435527+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:26:19.513754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436366+0000","last_change":"2023-07-18T03:39:57.347320+0000","last_active":"2023-07-18T03:39:58.436366+0000","last_peered":"2023-07-18T03:39:58.436366+0000","last_clean":"2023-07-18T03:39:58.436366+0000","last_became_active":"2023-07-18T03:39:57.347051+0000","last_became_peered":"2023-07-18T03:39:57.347051+0000","last_unstale":"2023-07-18T03:39:58.436366+0000","last_undegraded":"2023-07-18T03:39:58.436366+0000","last_fullsized":"2023-07-18T03:39:58.436366+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:01:56.508101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.351103+0000","last_change":"2023-07-18T03:39:57.343291+0000","last_active":"2023-07-18T03:39:58.351103+0000","last_peered":"2023-07-18T03:39:58.351103+0000","last_clean":"2023-07-18T03:39:58.351103+0000","last_became_active":"2023-07-18T03:39:57.342892+0000","last_became_peered":"2023-07-18T03:39:57.342892+0000","last_unstale":"2023-07-18T03:39:58.351103+0000","last_undegraded":"2023-07-18T03:39:58.351103+0000","last_fullsized":"2023-07-18T03:39:58.351103+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:31:21.950146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436262+0000","last_change":"2023-07-18T03:39:57.349732+0000","last_active":"2023-07-18T03:39:58.436262+0000","last_peered":"2023-07-18T03:39:58.436262+0000","last_clean":"2023-07-18T03:39:58.436262+0000","last_became_active":"2023-07-18T03:39:57.349421+0000","last_became_peered":"2023-07-18T03:39:57.349421+0000","last_unstale":"2023-07-18T03:39:58.436262+0000","last_undegraded":"2023-07-18T03:39:58.436262+0000","last_fullsized":"2023-07-18T03:39:58.436262+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:46:44.317665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435786+0000","last_change":"2023-07-18T03:39:57.367474+0000","last_active":"2023-07-18T03:39:58.435786+0000","last_peered":"2023-07-18T03:39:58.435786+0000","last_clean":"2023-07-18T03:39:58.435786+0000","last_became_active":"2023-07-18T03:39:57.367115+0000","last_became_peered":"2023-07-18T03:39:57.367115+0000","last_unstale":"2023-07-18T03:39:58.435786+0000","last_undegraded":"2023-07-18T03:39:58.435786+0000","last_fullsized":"2023-07-18T03:39:58.435786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:36:40.644576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435446+0000","last_change":"2023-07-18T03:39:57.354325+0000","last_active":"2023-07-18T03:39:58.435446+0000","last_peered":"2023-07-18T03:39:58.435446+0000","last_clean":"2023-07-18T03:39:58.435446+0000","last_became_active":"2023-07-18T03:39:57.353910+0000","last_became_peered":"2023-07-18T03:39:57.353910+0000","last_unstale":"2023-07-18T03:39:58.435446+0000","last_undegraded":"2023-07-18T03:39:58.435446+0000","last_fullsized":"2023-07-18T03:39:58.435446+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:42:04.213091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435471+0000","last_change":"2023-07-18T03:39:57.362161+0000","last_active":"2023-07-18T03:39:58.435471+0000","last_peered":"2023-07-18T03:39:58.435471+0000","last_clean":"2023-07-18T03:39:58.435471+0000","last_became_active":"2023-07-18T03:39:57.361882+0000","last_became_peered":"2023-07-18T03:39:57.361882+0000","last_unstale":"2023-07-18T03:39:58.435471+0000","last_undegraded":"2023-07-18T03:39:58.435471+0000","last_fullsized":"2023-07-18T03:39:58.435471+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:25:50.433101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353542+0000","last_change":"2023-07-18T03:39:57.347923+0000","last_active":"2023-07-18T03:39:58.353542+0000","last_peered":"2023-07-18T03:39:58.353542+0000","last_clean":"2023-07-18T03:39:58.353542+0000","last_became_active":"2023-07-18T03:39:57.347419+0000","last_became_peered":"2023-07-18T03:39:57.347419+0000","last_unstale":"2023-07-18T03:39:58.353542+0000","last_undegraded":"2023-07-18T03:39:58.353542+0000","last_fullsized":"2023-07-18T03:39:58.353542+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:31:40.988452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.350611+0000","last_change":"2023-07-18T03:39:57.352927+0000","last_active":"2023-07-18T03:39:58.350611+0000","last_peered":"2023-07-18T03:39:58.350611+0000","last_clean":"2023-07-18T03:39:58.350611+0000","last_became_active":"2023-07-18T03:39:57.352561+0000","last_became_peered":"2023-07-18T03:39:57.352561+0000","last_unstale":"2023-07-18T03:39:58.350611+0000","last_undegraded":"2023-07-18T03:39:58.350611+0000","last_fullsized":"2023-07-18T03:39:58.350611+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:43.945537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.352784+0000","last_change":"2023-07-18T03:39:57.338235+0000","last_active":"2023-07-18T03:39:58.352784+0000","last_peered":"2023-07-18T03:39:58.352784+0000","last_clean":"2023-07-18T03:39:58.352784+0000","last_became_active":"2023-07-18T03:39:57.337940+0000","last_became_peered":"2023-07-18T03:39:57.337940+0000","last_unstale":"2023-07-18T03:39:58.352784+0000","last_undegraded":"2023-07-18T03:39:58.352784+0000","last_fullsized":"2023-07-18T03:39:58.352784+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:38:21.588874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436454+0000","last_change":"2023-07-18T03:39:56.337657+0000","last_active":"2023-07-18T03:39:58.436454+0000","last_peered":"2023-07-18T03:39:58.436454+0000","last_clean":"2023-07-18T03:39:58.436454+0000","last_became_active":"2023-07-18T03:39:56.337379+0000","last_became_peered":"2023-07-18T03:39:56.337379+0000","last_unstale":"2023-07-18T03:39:58.436454+0000","last_undegraded":"2023-07-18T03:39:58.436454+0000","last_fullsized":"2023-07-18T03:39:58.436454+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:42:54.567568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.351154+0000","last_change":"2023-07-18T03:39:57.343868+0000","last_active":"2023-07-18T03:39:58.351154+0000","last_peered":"2023-07-18T03:39:58.351154+0000","last_clean":"2023-07-18T03:39:58.351154+0000","last_became_active":"2023-07-18T03:39:57.343588+0000","last_became_peered":"2023-07-18T03:39:57.343588+0000","last_unstale":"2023-07-18T03:39:58.351154+0000","last_undegraded":"2023-07-18T03:39:58.351154+0000","last_fullsized":"2023-07-18T03:39:58.351154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:01:06.422984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436418+0000","last_change":"2023-07-18T03:39:56.345284+0000","last_active":"2023-07-18T03:39:58.436418+0000","last_peered":"2023-07-18T03:39:58.436418+0000","last_clean":"2023-07-18T03:39:58.436418+0000","last_became_active":"2023-07-18T03:39:56.344955+0000","last_became_peered":"2023-07-18T03:39:56.344955+0000","last_unstale":"2023-07-18T03:39:58.436418+0000","last_undegraded":"2023-07-18T03:39:58.436418+0000","last_fullsized":"2023-07-18T03:39:58.436418+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:55:26.550751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.352866+0000","last_change":"2023-07-18T03:39:57.360491+0000","last_active":"2023-07-18T03:39:58.352866+0000","last_peered":"2023-07-18T03:39:58.352866+0000","last_clean":"2023-07-18T03:39:58.352866+0000","last_became_active":"2023-07-18T03:39:57.359864+0000","last_became_peered":"2023-07-18T03:39:57.359864+0000","last_unstale":"2023-07-18T03:39:58.352866+0000","last_undegraded":"2023-07-18T03:39:58.352866+0000","last_fullsized":"2023-07-18T03:39:58.352866+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:26:10.589134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.351240+0000","last_change":"2023-07-18T03:39:56.335839+0000","last_active":"2023-07-18T03:39:58.351240+0000","last_peered":"2023-07-18T03:39:58.351240+0000","last_clean":"2023-07-18T03:39:58.351240+0000","last_became_active":"2023-07-18T03:39:56.335506+0000","last_became_peered":"2023-07-18T03:39:56.335506+0000","last_unstale":"2023-07-18T03:39:58.351240+0000","last_undegraded":"2023-07-18T03:39:58.351240+0000","last_fullsized":"2023-07-18T03:39:58.351240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:26:35.806559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436895+0000","last_change":"2023-07-18T03:39:57.368431+0000","last_active":"2023-07-18T03:39:58.436895+0000","last_peered":"2023-07-18T03:39:58.436895+0000","last_clean":"2023-07-18T03:39:58.436895+0000","last_became_active":"2023-07-18T03:39:57.367916+0000","last_became_peered":"2023-07-18T03:39:57.367916+0000","last_unstale":"2023-07-18T03:39:58.436895+0000","last_undegraded":"2023-07-18T03:39:58.436895+0000","last_fullsized":"2023-07-18T03:39:58.436895+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:26:07.789214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353361+0000","last_change":"2023-07-18T03:39:56.353462+0000","last_active":"2023-07-18T03:39:58.353361+0000","last_peered":"2023-07-18T03:39:58.353361+0000","last_clean":"2023-07-18T03:39:58.353361+0000","last_became_active":"2023-07-18T03:39:56.353165+0000","last_became_peered":"2023-07-18T03:39:56.353165+0000","last_unstale":"2023-07-18T03:39:58.353361+0000","last_undegraded":"2023-07-18T03:39:58.353361+0000","last_fullsized":"2023-07-18T03:39:58.353361+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:33:52.202606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435409+0000","last_change":"2023-07-18T03:39:57.345776+0000","last_active":"2023-07-18T03:39:58.435409+0000","last_peered":"2023-07-18T03:39:58.435409+0000","last_clean":"2023-07-18T03:39:58.435409+0000","last_became_active":"2023-07-18T03:39:57.345450+0000","last_became_peered":"2023-07-18T03:39:57.345450+0000","last_unstale":"2023-07-18T03:39:58.435409+0000","last_undegraded":"2023-07-18T03:39:58.435409+0000","last_fullsized":"2023-07-18T03:39:58.435409+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:57:14.694115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435787+0000","last_change":"2023-07-18T03:39:56.343054+0000","last_active":"2023-07-18T03:39:58.435787+0000","last_peered":"2023-07-18T03:39:58.435787+0000","last_clean":"2023-07-18T03:39:58.435787+0000","last_became_active":"2023-07-18T03:39:56.342752+0000","last_became_peered":"2023-07-18T03:39:56.342752+0000","last_unstale":"2023-07-18T03:39:58.435787+0000","last_undegraded":"2023-07-18T03:39:58.435787+0000","last_fullsized":"2023-07-18T03:39:58.435787+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:10.541637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435498+0000","last_change":"2023-07-18T03:39:57.347359+0000","last_active":"2023-07-18T03:39:58.435498+0000","last_peered":"2023-07-18T03:39:58.435498+0000","last_clean":"2023-07-18T03:39:58.435498+0000","last_became_active":"2023-07-18T03:39:57.346683+0000","last_became_peered":"2023-07-18T03:39:57.346683+0000","last_unstale":"2023-07-18T03:39:58.435498+0000","last_undegraded":"2023-07-18T03:39:58.435498+0000","last_fullsized":"2023-07-18T03:39:58.435498+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:50:46.529521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.354255+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:58.354255+0000","last_peered":"2023-07-18T03:39:58.354255+0000","last_clean":"2023-07-18T03:39:58.354255+0000","last_became_active":"2023-07-18T03:39:56.345390+0000","last_became_peered":"2023-07-18T03:39:56.345390+0000","last_unstale":"2023-07-18T03:39:58.354255+0000","last_undegraded":"2023-07-18T03:39:58.354255+0000","last_fullsized":"2023-07-18T03:39:58.354255+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:00:50.710213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353525+0000","last_change":"2023-07-18T03:39:57.354996+0000","last_active":"2023-07-18T03:39:58.353525+0000","last_peered":"2023-07-18T03:39:58.353525+0000","last_clean":"2023-07-18T03:39:58.353525+0000","last_became_active":"2023-07-18T03:39:57.354719+0000","last_became_peered":"2023-07-18T03:39:57.354719+0000","last_unstale":"2023-07-18T03:39:58.353525+0000","last_undegraded":"2023-07-18T03:39:58.353525+0000","last_fullsized":"2023-07-18T03:39:58.353525+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:03:18.809175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.350703+0000","last_change":"2023-07-18T03:39:56.346375+0000","last_active":"2023-07-18T03:39:58.350703+0000","last_peered":"2023-07-18T03:39:58.350703+0000","last_clean":"2023-07-18T03:39:58.350703+0000","last_became_active":"2023-07-18T03:39:56.346098+0000","last_became_peered":"2023-07-18T03:39:56.346098+0000","last_unstale":"2023-07-18T03:39:58.350703+0000","last_undegraded":"2023-07-18T03:39:58.350703+0000","last_fullsized":"2023-07-18T03:39:58.350703+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:54:11.930622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436972+0000","last_change":"2023-07-18T03:39:57.362283+0000","last_active":"2023-07-18T03:39:58.436972+0000","last_peered":"2023-07-18T03:39:58.436972+0000","last_clean":"2023-07-18T03:39:58.436972+0000","last_became_active":"2023-07-18T03:39:57.361509+0000","last_became_peered":"2023-07-18T03:39:57.361509+0000","last_unstale":"2023-07-18T03:39:58.436972+0000","last_undegraded":"2023-07-18T03:39:58.436972+0000","last_fullsized":"2023-07-18T03:39:58.436972+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:38:00.602599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.352535+0000","last_change":"2023-07-18T03:39:56.337739+0000","last_active":"2023-07-18T03:39:58.352535+0000","last_peered":"2023-07-18T03:39:58.352535+0000","last_clean":"2023-07-18T03:39:58.352535+0000","last_became_active":"2023-07-18T03:39:56.337495+0000","last_became_peered":"2023-07-18T03:39:56.337495+0000","last_unstale":"2023-07-18T03:39:58.352535+0000","last_undegraded":"2023-07-18T03:39:58.352535+0000","last_fullsized":"2023-07-18T03:39:58.352535+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:26:18.548517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436623+0000","last_change":"2023-07-18T03:39:57.356339+0000","last_active":"2023-07-18T03:39:58.436623+0000","last_peered":"2023-07-18T03:39:58.436623+0000","last_clean":"2023-07-18T03:39:58.436623+0000","last_became_active":"2023-07-18T03:39:57.356008+0000","last_became_peered":"2023-07-18T03:39:57.356008+0000","last_unstale":"2023-07-18T03:39:58.436623+0000","last_undegraded":"2023-07-18T03:39:58.436623+0000","last_fullsized":"2023-07-18T03:39:58.436623+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:45:50.842355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.437129+0000","last_change":"2023-07-18T03:39:56.352591+0000","last_active":"2023-07-18T03:39:58.437129+0000","last_peered":"2023-07-18T03:39:58.437129+0000","last_clean":"2023-07-18T03:39:58.437129+0000","last_became_active":"2023-07-18T03:39:56.352257+0000","last_became_peered":"2023-07-18T03:39:56.352257+0000","last_unstale":"2023-07-18T03:39:58.437129+0000","last_undegraded":"2023-07-18T03:39:58.437129+0000","last_fullsized":"2023-07-18T03:39:58.437129+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:09.700401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353543+0000","last_change":"2023-07-18T03:39:57.354324+0000","last_active":"2023-07-18T03:39:58.353543+0000","last_peered":"2023-07-18T03:39:58.353543+0000","last_clean":"2023-07-18T03:39:58.353543+0000","last_became_active":"2023-07-18T03:39:57.354028+0000","last_became_peered":"2023-07-18T03:39:57.354028+0000","last_unstale":"2023-07-18T03:39:58.353543+0000","last_undegraded":"2023-07-18T03:39:58.353543+0000","last_fullsized":"2023-07-18T03:39:58.353543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:44:05.319153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436265+0000","last_change":"2023-07-18T03:39:56.353133+0000","last_active":"2023-07-18T03:39:58.436265+0000","last_peered":"2023-07-18T03:39:58.436265+0000","last_clean":"2023-07-18T03:39:58.436265+0000","last_became_active":"2023-07-18T03:39:56.352847+0000","last_became_peered":"2023-07-18T03:39:56.352847+0000","last_unstale":"2023-07-18T03:39:58.436265+0000","last_undegraded":"2023-07-18T03:39:58.436265+0000","last_fullsized":"2023-07-18T03:39:58.436265+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:57:57.797956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436270+0000","last_change":"2023-07-18T03:39:57.350961+0000","last_active":"2023-07-18T03:39:58.436270+0000","last_peered":"2023-07-18T03:39:58.436270+0000","last_clean":"2023-07-18T03:39:58.436270+0000","last_became_active":"2023-07-18T03:39:57.350643+0000","last_became_peered":"2023-07-18T03:39:57.350643+0000","last_unstale":"2023-07-18T03:39:58.436270+0000","last_undegraded":"2023-07-18T03:39:58.436270+0000","last_fullsized":"2023-07-18T03:39:58.436270+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:57:58.276303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'3","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436401+0000","last_change":"2023-07-18T03:39:56.348133+0000","last_active":"2023-07-18T03:39:58.436401+0000","last_peered":"2023-07-18T03:39:58.436401+0000","last_clean":"2023-07-18T03:39:58.436401+0000","last_became_active":"2023-07-18T03:39:56.347930+0000","last_became_peered":"2023-07-18T03:39:56.347930+0000","last_unstale":"2023-07-18T03:39:58.436401+0000","last_undegraded":"2023-07-18T03:39:58.436401+0000","last_fullsized":"2023-07-18T03:39:58.436401+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:57:37.239962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353999+0000","last_change":"2023-07-18T03:39:57.362245+0000","last_active":"2023-07-18T03:39:58.353999+0000","last_peered":"2023-07-18T03:39:58.353999+0000","last_clean":"2023-07-18T03:39:58.353999+0000","last_became_active":"2023-07-18T03:39:57.362019+0000","last_became_peered":"2023-07-18T03:39:57.362019+0000","last_unstale":"2023-07-18T03:39:58.353999+0000","last_undegraded":"2023-07-18T03:39:58.353999+0000","last_fullsized":"2023-07-18T03:39:58.353999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:01:23.093728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.437162+0000","last_change":"2023-07-18T03:39:56.352639+0000","last_active":"2023-07-18T03:39:58.437162+0000","last_peered":"2023-07-18T03:39:58.437162+0000","last_clean":"2023-07-18T03:39:58.437162+0000","last_became_active":"2023-07-18T03:39:56.352134+0000","last_became_peered":"2023-07-18T03:39:56.352134+0000","last_unstale":"2023-07-18T03:39:58.437162+0000","last_undegraded":"2023-07-18T03:39:58.437162+0000","last_fullsized":"2023-07-18T03:39:58.437162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:01.077387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435792+0000","last_change":"2023-07-18T03:39:57.349224+0000","last_active":"2023-07-18T03:39:58.435792+0000","last_peered":"2023-07-18T03:39:58.435792+0000","last_clean":"2023-07-18T03:39:58.435792+0000","last_became_active":"2023-07-18T03:39:57.348944+0000","last_became_peered":"2023-07-18T03:39:57.348944+0000","last_unstale":"2023-07-18T03:39:58.435792+0000","last_undegraded":"2023-07-18T03:39:58.435792+0000","last_fullsized":"2023-07-18T03:39:58.435792+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:23:25.620504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.437125+0000","last_change":"2023-07-18T03:39:56.352532+0000","last_active":"2023-07-18T03:39:58.437125+0000","last_peered":"2023-07-18T03:39:58.437125+0000","last_clean":"2023-07-18T03:39:58.437125+0000","last_became_active":"2023-07-18T03:39:56.352201+0000","last_became_peered":"2023-07-18T03:39:56.352201+0000","last_unstale":"2023-07-18T03:39:58.437125+0000","last_undegraded":"2023-07-18T03:39:58.437125+0000","last_fullsized":"2023-07-18T03:39:58.437125+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:24.138081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436946+0000","last_change":"2023-07-18T03:39:57.360476+0000","last_active":"2023-07-18T03:39:58.436946+0000","last_peered":"2023-07-18T03:39:58.436946+0000","last_clean":"2023-07-18T03:39:58.436946+0000","last_became_active":"2023-07-18T03:39:57.360066+0000","last_became_peered":"2023-07-18T03:39:57.360066+0000","last_unstale":"2023-07-18T03:39:58.436946+0000","last_undegraded":"2023-07-18T03:39:58.436946+0000","last_fullsized":"2023-07-18T03:39:58.436946+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:07:59.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.350785+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:58.350785+0000","last_peered":"2023-07-18T03:39:58.350785+0000","last_clean":"2023-07-18T03:39:58.350785+0000","last_became_active":"2023-07-18T03:39:56.345735+0000","last_became_peered":"2023-07-18T03:39:56.345735+0000","last_unstale":"2023-07-18T03:39:58.350785+0000","last_undegraded":"2023-07-18T03:39:58.350785+0000","last_fullsized":"2023-07-18T03:39:58.350785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:38:06.062193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.354068+0000","last_change":"2023-07-18T03:39:57.347143+0000","last_active":"2023-07-18T03:39:58.354068+0000","last_peered":"2023-07-18T03:39:58.354068+0000","last_clean":"2023-07-18T03:39:58.354068+0000","last_became_active":"2023-07-18T03:39:57.346833+0000","last_became_peered":"2023-07-18T03:39:57.346833+0000","last_unstale":"2023-07-18T03:39:58.354068+0000","last_undegraded":"2023-07-18T03:39:58.354068+0000","last_fullsized":"2023-07-18T03:39:58.354068+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:49:37.330452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435473+0000","last_change":"2023-07-18T03:39:56.339148+0000","last_active":"2023-07-18T03:39:58.435473+0000","last_peered":"2023-07-18T03:39:58.435473+0000","last_clean":"2023-07-18T03:39:58.435473+0000","last_became_active":"2023-07-18T03:39:56.338862+0000","last_became_peered":"2023-07-18T03:39:56.338862+0000","last_unstale":"2023-07-18T03:39:58.435473+0000","last_undegraded":"2023-07-18T03:39:58.435473+0000","last_fullsized":"2023-07-18T03:39:58.435473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:12.877614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436674+0000","last_change":"2023-07-18T03:39:57.363546+0000","last_active":"2023-07-18T03:39:58.436674+0000","last_peered":"2023-07-18T03:39:58.436674+0000","last_clean":"2023-07-18T03:39:58.436674+0000","last_became_active":"2023-07-18T03:39:57.362867+0000","last_became_peered":"2023-07-18T03:39:57.362867+0000","last_unstale":"2023-07-18T03:39:58.436674+0000","last_undegraded":"2023-07-18T03:39:58.436674+0000","last_fullsized":"2023-07-18T03:39:58.436674+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:48:08.031962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":21,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353155+0000","last_change":"2023-07-18T03:39:56.345149+0000","last_active":"2023-07-18T03:39:58.353155+0000","last_peered":"2023-07-18T03:39:58.353155+0000","last_clean":"2023-07-18T03:39:58.353155+0000","last_became_active":"2023-07-18T03:39:56.344848+0000","last_became_peered":"2023-07-18T03:39:56.344848+0000","last_unstale":"2023-07-18T03:39:58.353155+0000","last_undegraded":"2023-07-18T03:39:58.353155+0000","last_fullsized":"2023-07-18T03:39:58.353155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:59:47.055923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.437038+0000","last_change":"2023-07-18T03:39:57.366489+0000","last_active":"2023-07-18T03:39:58.437038+0000","last_peered":"2023-07-18T03:39:58.437038+0000","last_clean":"2023-07-18T03:39:58.437038+0000","last_became_active":"2023-07-18T03:39:57.366243+0000","last_became_peered":"2023-07-18T03:39:57.366243+0000","last_unstale":"2023-07-18T03:39:58.437038+0000","last_undegraded":"2023-07-18T03:39:58.437038+0000","last_fullsized":"2023-07-18T03:39:58.437038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:44:05.319153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436455+0000","last_change":"2023-07-18T03:39:56.346083+0000","last_active":"2023-07-18T03:39:58.436455+0000","last_peered":"2023-07-18T03:39:58.436455+0000","last_clean":"2023-07-18T03:39:58.436455+0000","last_became_active":"2023-07-18T03:39:56.345797+0000","last_became_peered":"2023-07-18T03:39:56.345797+0000","last_unstale":"2023-07-18T03:39:58.436455+0000","last_undegraded":"2023-07-18T03:39:58.436455+0000","last_fullsized":"2023-07-18T03:39:58.436455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:23:36.525406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436125+0000","last_change":"2023-07-18T03:39:57.358675+0000","last_active":"2023-07-18T03:39:58.436125+0000","last_peered":"2023-07-18T03:39:58.436125+0000","last_clean":"2023-07-18T03:39:58.436125+0000","last_became_active":"2023-07-18T03:39:57.358067+0000","last_became_peered":"2023-07-18T03:39:57.358067+0000","last_unstale":"2023-07-18T03:39:58.436125+0000","last_undegraded":"2023-07-18T03:39:58.436125+0000","last_fullsized":"2023-07-18T03:39:58.436125+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:51:10.704943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353183+0000","last_change":"2023-07-18T03:39:56.349196+0000","last_active":"2023-07-18T03:39:58.353183+0000","last_peered":"2023-07-18T03:39:58.353183+0000","last_clean":"2023-07-18T03:39:58.353183+0000","last_became_active":"2023-07-18T03:39:56.347176+0000","last_became_peered":"2023-07-18T03:39:56.347176+0000","last_unstale":"2023-07-18T03:39:58.353183+0000","last_undegraded":"2023-07-18T03:39:58.353183+0000","last_fullsized":"2023-07-18T03:39:58.353183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:02:38.208708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353021+0000","last_change":"2023-07-18T03:39:57.358539+0000","last_active":"2023-07-18T03:39:58.353021+0000","last_peered":"2023-07-18T03:39:58.353021+0000","last_clean":"2023-07-18T03:39:58.353021+0000","last_became_active":"2023-07-18T03:39:57.357578+0000","last_became_peered":"2023-07-18T03:39:57.357578+0000","last_unstale":"2023-07-18T03:39:58.353021+0000","last_undegraded":"2023-07-18T03:39:58.353021+0000","last_fullsized":"2023-07-18T03:39:58.353021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:20:05.683298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436541+0000","last_change":"2023-07-18T03:39:56.344722+0000","last_active":"2023-07-18T03:39:58.436541+0000","last_peered":"2023-07-18T03:39:58.436541+0000","last_clean":"2023-07-18T03:39:58.436541+0000","last_became_active":"2023-07-18T03:39:56.344439+0000","last_became_peered":"2023-07-18T03:39:56.344439+0000","last_unstale":"2023-07-18T03:39:58.436541+0000","last_undegraded":"2023-07-18T03:39:58.436541+0000","last_fullsized":"2023-07-18T03:39:58.436541+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:15:02.889526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435744+0000","last_change":"2023-07-18T03:39:57.363607+0000","last_active":"2023-07-18T03:39:58.435744+0000","last_peered":"2023-07-18T03:39:58.435744+0000","last_clean":"2023-07-18T03:39:58.435744+0000","last_became_active":"2023-07-18T03:39:57.363312+0000","last_became_peered":"2023-07-18T03:39:57.363312+0000","last_unstale":"2023-07-18T03:39:58.435744+0000","last_undegraded":"2023-07-18T03:39:58.435744+0000","last_fullsized":"2023-07-18T03:39:58.435744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:18:53.301445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.350949+0000","last_change":"2023-07-18T03:39:56.336359+0000","last_active":"2023-07-18T03:39:58.350949+0000","last_peered":"2023-07-18T03:39:58.350949+0000","last_clean":"2023-07-18T03:39:58.350949+0000","last_became_active":"2023-07-18T03:39:56.336094+0000","last_became_peered":"2023-07-18T03:39:56.336094+0000","last_unstale":"2023-07-18T03:39:58.350949+0000","last_undegraded":"2023-07-18T03:39:58.350949+0000","last_fullsized":"2023-07-18T03:39:58.350949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:52:09.110549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353005+0000","last_change":"2023-07-18T03:39:57.348592+0000","last_active":"2023-07-18T03:39:58.353005+0000","last_peered":"2023-07-18T03:39:58.353005+0000","last_clean":"2023-07-18T03:39:58.353005+0000","last_became_active":"2023-07-18T03:39:57.348257+0000","last_became_peered":"2023-07-18T03:39:57.348257+0000","last_unstale":"2023-07-18T03:39:58.353005+0000","last_undegraded":"2023-07-18T03:39:58.353005+0000","last_fullsized":"2023-07-18T03:39:58.353005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:02.084523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436171+0000","last_change":"2023-07-18T03:39:56.337863+0000","last_active":"2023-07-18T03:39:58.436171+0000","last_peered":"2023-07-18T03:39:58.436171+0000","last_clean":"2023-07-18T03:39:58.436171+0000","last_became_active":"2023-07-18T03:39:56.337561+0000","last_became_peered":"2023-07-18T03:39:56.337561+0000","last_unstale":"2023-07-18T03:39:58.436171+0000","last_undegraded":"2023-07-18T03:39:58.436171+0000","last_fullsized":"2023-07-18T03:39:58.436171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:12:40.430479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436188+0000","last_change":"2023-07-18T03:39:57.351197+0000","last_active":"2023-07-18T03:39:58.436188+0000","last_peered":"2023-07-18T03:39:58.436188+0000","last_clean":"2023-07-18T03:39:58.436188+0000","last_became_active":"2023-07-18T03:39:57.350814+0000","last_became_peered":"2023-07-18T03:39:57.350814+0000","last_unstale":"2023-07-18T03:39:58.436188+0000","last_undegraded":"2023-07-18T03:39:58.436188+0000","last_fullsized":"2023-07-18T03:39:58.436188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:44:15.826142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436094+0000","last_change":"2023-07-18T03:39:56.336889+0000","last_active":"2023-07-18T03:39:58.436094+0000","last_peered":"2023-07-18T03:39:58.436094+0000","last_clean":"2023-07-18T03:39:58.436094+0000","last_became_active":"2023-07-18T03:39:56.336599+0000","last_became_peered":"2023-07-18T03:39:56.336599+0000","last_unstale":"2023-07-18T03:39:58.436094+0000","last_undegraded":"2023-07-18T03:39:58.436094+0000","last_fullsized":"2023-07-18T03:39:58.436094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:16:35.738099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435889+0000","last_change":"2023-07-18T03:39:57.359396+0000","last_active":"2023-07-18T03:39:58.435889+0000","last_peered":"2023-07-18T03:39:58.435889+0000","last_clean":"2023-07-18T03:39:58.435889+0000","last_became_active":"2023-07-18T03:39:57.359113+0000","last_became_peered":"2023-07-18T03:39:57.359113+0000","last_unstale":"2023-07-18T03:39:58.435889+0000","last_undegraded":"2023-07-18T03:39:58.435889+0000","last_fullsized":"2023-07-18T03:39:58.435889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:22:11.047392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353810+0000","last_change":"2023-07-18T03:39:56.350386+0000","last_active":"2023-07-18T03:39:58.353810+0000","last_peered":"2023-07-18T03:39:58.353810+0000","last_clean":"2023-07-18T03:39:58.353810+0000","last_became_active":"2023-07-18T03:39:56.350046+0000","last_became_peered":"2023-07-18T03:39:56.350046+0000","last_unstale":"2023-07-18T03:39:58.353810+0000","last_undegraded":"2023-07-18T03:39:58.353810+0000","last_fullsized":"2023-07-18T03:39:58.353810+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:06:20.947537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436826+0000","last_change":"2023-07-18T03:39:57.362621+0000","last_active":"2023-07-18T03:39:58.436826+0000","last_peered":"2023-07-18T03:39:58.436826+0000","last_clean":"2023-07-18T03:39:58.436826+0000","last_became_active":"2023-07-18T03:39:57.361838+0000","last_became_peered":"2023-07-18T03:39:57.361838+0000","last_unstale":"2023-07-18T03:39:58.436826+0000","last_undegraded":"2023-07-18T03:39:58.436826+0000","last_fullsized":"2023-07-18T03:39:58.436826+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:40:35.911357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435477+0000","last_change":"2023-07-18T03:39:56.338127+0000","last_active":"2023-07-18T03:39:58.435477+0000","last_peered":"2023-07-18T03:39:58.435477+0000","last_clean":"2023-07-18T03:39:58.435477+0000","last_became_active":"2023-07-18T03:39:56.337747+0000","last_became_peered":"2023-07-18T03:39:56.337747+0000","last_unstale":"2023-07-18T03:39:58.435477+0000","last_undegraded":"2023-07-18T03:39:58.435477+0000","last_fullsized":"2023-07-18T03:39:58.435477+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:28:28.326013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353629+0000","last_change":"2023-07-18T03:39:57.364347+0000","last_active":"2023-07-18T03:39:58.353629+0000","last_peered":"2023-07-18T03:39:58.353629+0000","last_clean":"2023-07-18T03:39:58.353629+0000","last_became_active":"2023-07-18T03:39:57.364145+0000","last_became_peered":"2023-07-18T03:39:57.364145+0000","last_unstale":"2023-07-18T03:39:58.353629+0000","last_undegraded":"2023-07-18T03:39:58.353629+0000","last_fullsized":"2023-07-18T03:39:58.353629+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:20:04.680931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.352539+0000","last_change":"2023-07-18T03:39:56.353087+0000","last_active":"2023-07-18T03:39:58.352539+0000","last_peered":"2023-07-18T03:39:58.352539+0000","last_clean":"2023-07-18T03:39:58.352539+0000","last_became_active":"2023-07-18T03:39:56.352738+0000","last_became_peered":"2023-07-18T03:39:56.352738+0000","last_unstale":"2023-07-18T03:39:58.352539+0000","last_undegraded":"2023-07-18T03:39:58.352539+0000","last_fullsized":"2023-07-18T03:39:58.352539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:18:32.639285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.352905+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:39:58.352905+0000","last_peered":"2023-07-18T03:39:58.352905+0000","last_clean":"2023-07-18T03:39:58.352905+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:39:58.352905+0000","last_undegraded":"2023-07-18T03:39:58.352905+0000","last_fullsized":"2023-07-18T03:39:58.352905+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:58:42.376912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353486+0000","last_change":"2023-07-18T03:39:57.348865+0000","last_active":"2023-07-18T03:39:58.353486+0000","last_peered":"2023-07-18T03:39:58.353486+0000","last_clean":"2023-07-18T03:39:58.353486+0000","last_became_active":"2023-07-18T03:39:57.348246+0000","last_became_peered":"2023-07-18T03:39:57.348246+0000","last_unstale":"2023-07-18T03:39:58.353486+0000","last_undegraded":"2023-07-18T03:39:58.353486+0000","last_fullsized":"2023-07-18T03:39:58.353486+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:10:38.274535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.437202+0000","last_change":"2023-07-18T03:39:56.352671+0000","last_active":"2023-07-18T03:39:58.437202+0000","last_peered":"2023-07-18T03:39:58.437202+0000","last_clean":"2023-07-18T03:39:58.437202+0000","last_became_active":"2023-07-18T03:39:56.351865+0000","last_became_peered":"2023-07-18T03:39:56.351865+0000","last_unstale":"2023-07-18T03:39:58.437202+0000","last_undegraded":"2023-07-18T03:39:58.437202+0000","last_fullsized":"2023-07-18T03:39:58.437202+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:16:54.973371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.437307+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:39:58.437307+0000","last_peered":"2023-07-18T03:39:58.437307+0000","last_clean":"2023-07-18T03:39:58.437307+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:39:58.437307+0000","last_undegraded":"2023-07-18T03:39:58.437307+0000","last_fullsized":"2023-07-18T03:39:58.437307+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:09:10.277610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.350915+0000","last_change":"2023-07-18T03:39:57.342924+0000","last_active":"2023-07-18T03:39:58.350915+0000","last_peered":"2023-07-18T03:39:58.350915+0000","last_clean":"2023-07-18T03:39:58.350915+0000","last_became_active":"2023-07-18T03:39:57.342655+0000","last_became_peered":"2023-07-18T03:39:57.342655+0000","last_unstale":"2023-07-18T03:39:58.350915+0000","last_undegraded":"2023-07-18T03:39:58.350915+0000","last_fullsized":"2023-07-18T03:39:58.350915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:47.799202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436607+0000","last_change":"2023-07-18T03:39:56.347782+0000","last_active":"2023-07-18T03:39:58.436607+0000","last_peered":"2023-07-18T03:39:58.436607+0000","last_clean":"2023-07-18T03:39:58.436607+0000","last_became_active":"2023-07-18T03:39:56.347519+0000","last_became_peered":"2023-07-18T03:39:56.347519+0000","last_unstale":"2023-07-18T03:39:58.436607+0000","last_undegraded":"2023-07-18T03:39:58.436607+0000","last_fullsized":"2023-07-18T03:39:58.436607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:47:41.220141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":41,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.350916+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:39:58.350916+0000","last_peered":"2023-07-18T03:39:58.350916+0000","last_clean":"2023-07-18T03:39:58.350916+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:39:58.350916+0000","last_undegraded":"2023-07-18T03:39:58.350916+0000","last_fullsized":"2023-07-18T03:39:58.350916+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:53.012363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":197,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353360+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:39:58.353360+0000","last_peered":"2023-07-18T03:39:58.353360+0000","last_clean":"2023-07-18T03:39:58.353360+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:39:58.353360+0000","last_undegraded":"2023-07-18T03:39:58.353360+0000","last_fullsized":"2023-07-18T03:39:58.353360+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:25:49.592030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353652+0000","last_change":"2023-07-18T03:39:57.350408+0000","last_active":"2023-07-18T03:39:58.353652+0000","last_peered":"2023-07-18T03:39:58.353652+0000","last_clean":"2023-07-18T03:39:58.353652+0000","last_became_active":"2023-07-18T03:39:57.350103+0000","last_became_peered":"2023-07-18T03:39:57.350103+0000","last_unstale":"2023-07-18T03:39:58.353652+0000","last_undegraded":"2023-07-18T03:39:58.353652+0000","last_fullsized":"2023-07-18T03:39:58.353652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:07:22.601106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"36'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435759+0000","last_change":"2023-07-18T03:39:56.346081+0000","last_active":"2023-07-18T03:39:58.435759+0000","last_peered":"2023-07-18T03:39:58.435759+0000","last_clean":"2023-07-18T03:39:58.435759+0000","last_became_active":"2023-07-18T03:39:56.345762+0000","last_became_peered":"2023-07-18T03:39:56.345762+0000","last_unstale":"2023-07-18T03:39:58.435759+0000","last_undegraded":"2023-07-18T03:39:58.435759+0000","last_fullsized":"2023-07-18T03:39:58.435759+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:27.471564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.354231+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:39:58.354231+0000","last_peered":"2023-07-18T03:39:58.354231+0000","last_clean":"2023-07-18T03:39:58.354231+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:39:58.354231+0000","last_undegraded":"2023-07-18T03:39:58.354231+0000","last_fullsized":"2023-07-18T03:39:58.354231+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:01:35.114046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436298+0000","last_change":"2023-07-18T03:39:57.363717+0000","last_active":"2023-07-18T03:39:58.436298+0000","last_peered":"2023-07-18T03:39:58.436298+0000","last_clean":"2023-07-18T03:39:58.436298+0000","last_became_active":"2023-07-18T03:39:57.363399+0000","last_became_peered":"2023-07-18T03:39:57.363399+0000","last_unstale":"2023-07-18T03:39:58.436298+0000","last_undegraded":"2023-07-18T03:39:58.436298+0000","last_fullsized":"2023-07-18T03:39:58.436298+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:53:22.366395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.352923+0000","last_change":"2023-07-18T03:39:56.348881+0000","last_active":"2023-07-18T03:39:58.352923+0000","last_peered":"2023-07-18T03:39:58.352923+0000","last_clean":"2023-07-18T03:39:58.352923+0000","last_became_active":"2023-07-18T03:39:56.348622+0000","last_became_peered":"2023-07-18T03:39:56.348622+0000","last_unstale":"2023-07-18T03:39:58.352923+0000","last_undegraded":"2023-07-18T03:39:58.352923+0000","last_fullsized":"2023-07-18T03:39:58.352923+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:59:09.700401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.353274+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:39:58.353274+0000","last_peered":"2023-07-18T03:39:58.353274+0000","last_clean":"2023-07-18T03:39:58.353274+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:39:58.353274+0000","last_undegraded":"2023-07-18T03:39:58.353274+0000","last_fullsized":"2023-07-18T03:39:58.353274+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:15:27.983583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.437194+0000","last_change":"2023-07-18T03:39:57.364725+0000","last_active":"2023-07-18T03:39:58.437194+0000","last_peered":"2023-07-18T03:39:58.437194+0000","last_clean":"2023-07-18T03:39:58.437194+0000","last_became_active":"2023-07-18T03:39:57.363441+0000","last_became_peered":"2023-07-18T03:39:57.363441+0000","last_unstale":"2023-07-18T03:39:58.437194+0000","last_undegraded":"2023-07-18T03:39:58.437194+0000","last_fullsized":"2023-07-18T03:39:58.437194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:59:48.063059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436665+0000","last_change":"2023-07-18T03:39:56.337331+0000","last_active":"2023-07-18T03:39:58.436665+0000","last_peered":"2023-07-18T03:39:58.436665+0000","last_clean":"2023-07-18T03:39:58.436665+0000","last_became_active":"2023-07-18T03:39:56.337021+0000","last_became_peered":"2023-07-18T03:39:56.337021+0000","last_unstale":"2023-07-18T03:39:58.436665+0000","last_undegraded":"2023-07-18T03:39:58.436665+0000","last_fullsized":"2023-07-18T03:39:58.436665+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:11:42.662578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435731+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:39:58.435731+0000","last_peered":"2023-07-18T03:39:58.435731+0000","last_clean":"2023-07-18T03:39:58.435731+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:39:58.435731+0000","last_undegraded":"2023-07-18T03:39:58.435731+0000","last_fullsized":"2023-07-18T03:39:58.435731+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:05:37.659726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.350618+0000","last_change":"2023-07-18T03:39:57.356218+0000","last_active":"2023-07-18T03:39:58.350618+0000","last_peered":"2023-07-18T03:39:58.350618+0000","last_clean":"2023-07-18T03:39:58.350618+0000","last_became_active":"2023-07-18T03:39:57.355889+0000","last_became_peered":"2023-07-18T03:39:57.355889+0000","last_unstale":"2023-07-18T03:39:58.350618+0000","last_undegraded":"2023-07-18T03:39:58.350618+0000","last_fullsized":"2023-07-18T03:39:58.350618+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:39:44.062884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435513+0000","last_change":"2023-07-18T03:39:56.341357+0000","last_active":"2023-07-18T03:39:58.435513+0000","last_peered":"2023-07-18T03:39:58.435513+0000","last_clean":"2023-07-18T03:39:58.435513+0000","last_became_active":"2023-07-18T03:39:56.341092+0000","last_became_peered":"2023-07-18T03:39:56.341092+0000","last_unstale":"2023-07-18T03:39:58.435513+0000","last_undegraded":"2023-07-18T03:39:58.435513+0000","last_fullsized":"2023-07-18T03:39:58.435513+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:21:01.203032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436541+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:39:58.436541+0000","last_peered":"2023-07-18T03:39:58.436541+0000","last_clean":"2023-07-18T03:39:58.436541+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:39:58.436541+0000","last_undegraded":"2023-07-18T03:39:58.436541+0000","last_fullsized":"2023-07-18T03:39:58.436541+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:44:55.863256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.435650+0000","last_change":"2023-07-18T03:39:57.360948+0000","last_active":"2023-07-18T03:39:58.435650+0000","last_peered":"2023-07-18T03:39:58.435650+0000","last_clean":"2023-07-18T03:39:58.435650+0000","last_became_active":"2023-07-18T03:39:57.360601+0000","last_became_peered":"2023-07-18T03:39:57.360601+0000","last_unstale":"2023-07-18T03:39:58.435650+0000","last_undegraded":"2023-07-18T03:39:58.435650+0000","last_fullsized":"2023-07-18T03:39:58.435650+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:56.318026+0000","last_clean_scrub_stamp":"2023-07-18T03:39:56.318026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:33:14.114308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436354+0000","last_change":"2023-07-18T03:39:56.342369+0000","last_active":"2023-07-18T03:39:58.436354+0000","last_peered":"2023-07-18T03:39:58.436354+0000","last_clean":"2023-07-18T03:39:58.436354+0000","last_became_active":"2023-07-18T03:39:56.342108+0000","last_became_peered":"2023-07-18T03:39:56.342108+0000","last_unstale":"2023-07-18T03:39:58.436354+0000","last_undegraded":"2023-07-18T03:39:58.436354+0000","last_fullsized":"2023-07-18T03:39:58.436354+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:39:55.310889+0000","last_clean_scrub_stamp":"2023-07-18T03:39:55.310889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:07:56.378262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":41,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-18T03:39:58.436505+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:39:58.436505+0000","last_peered":"2023-07-18T03:39:58.436505+0000","last_clean":"2023-07-18T03:39:58.436505+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:39:58.436505+0000","last_undegraded":"2023-07-18T03:39:58.436505+0000","last_fullsized":"2023-07-18T03:39:58.436505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:55:13.213173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":1011712,"data_stored":758991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.622},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.497}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.2290000000000001,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.69299999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.438},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.65000000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.64000000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.68400000000000005}]}]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":167571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.83099999999999996}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.86299999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.81799999999999995}]},{"osd":3,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.93899999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73099999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.63},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.754}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.88100000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":417792,"data_stored":168587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.433},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.47199999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.628},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.8290000000000002,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.53800000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64300000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70099999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.84199999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":999424,"data_stored":757447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.317},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.443},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.42199999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.80000000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.745}]},{"osd":7,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.85499999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":409600,"data_stored":167593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.61899999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.745},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63700000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.71499999999999997}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.57599999999999996}]},{"osd":5,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:39:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.76000000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":409600,"data_stored":167101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.436},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.5}]},{"osd":3,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.622},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.51500000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:39:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.55200000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:39:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.47199999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":405504,"data_stored":167023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.5}]},{"osd":2,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":8.3040000000000003,"5min":8.3040000000000003,"15min":8.3040000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.61499999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.0229999999999997,"5min":8.0229999999999997,"15min":8.0229999999999997},"last":0.438},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":8.359,"5min":8.359,"15min":8.359},"last":0.52400000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.628}]},{"osd":5,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.69899999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.71599999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:39:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.59399999999999997}]}]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":405504,"data_stored":167059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.45600000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.53300000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.74299999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81799999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:39:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.77300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:40:07.300 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-18T03:40:07.301 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-18T03:40:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.551+0000 7f7bba1ff700 1 Processor -- start 2023-07-18T03:40:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.551+0000 7f7bba1ff700 1 -- start start 2023-07-18T03:40:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.552+0000 7f7bba1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7bb40d0f90 0x7f7bb40b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.552+0000 7f7bba1ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb40b8e30 con 0x7f7bb40d0b10 2023-07-18T03:40:07.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.552+0000 7f7bba1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb40afaa0 con 0x7f7bb40b8a60 2023-07-18T03:40:07.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.552+0000 7f7bba1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb40afd80 con 0x7f7bb40d0f90 2023-07-18T03:40:07.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.552+0000 7f7bb2ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7bb40d0b10 0x7f7bb40b0e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56870/0 (socket says 172.21.15.49:56870) 2023-07-18T03:40:07.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.552+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/3175577615 learned_addr learned my addr 172.21.15.49:0/3175577615 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:07.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.553+0000 7f7bb3fff700 1 --2- 172.21.15.49:0/3175577615 >> v2:172.21.15.107:3300/0 conn(0x7f7bb40d0f90 0x7f7bb40b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.553+0000 7f7bb27fc700 1 -- 172.21.15.49:0/3175577615 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509958053 0 0) 0x7f7bb40b8e30 con 0x7f7bb40d0b10 2023-07-18T03:40:07.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.553+0000 7f7bb27fc700 1 -- 172.21.15.49:0/3175577615 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b9c0035b0 con 0x7f7bb40d0b10 2023-07-18T03:40:07.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.553+0000 7f7bb27fc700 1 -- 172.21.15.49:0/3175577615 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293701707 0 0) 0x7f7bb40afaa0 con 0x7f7bb40b8a60 2023-07-18T03:40:07.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.553+0000 7f7bb27fc700 1 -- 172.21.15.49:0/3175577615 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb40b8e30 con 0x7f7bb40b8a60 2023-07-18T03:40:07.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.553+0000 7f7bb3fff700 1 -- 172.21.15.49:0/3175577615 >> v1:172.21.15.107:6789/0 conn(0x7f7bb40b8a60 legacy=0x7f7bb40b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.553+0000 7f7bb3fff700 1 -- 172.21.15.49:0/3175577615 >> v1:172.21.15.49:6789/0 conn(0x7f7bb40d0b10 legacy=0x7f7bb40b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.553+0000 7f7bb27fc700 1 -- 172.21.15.49:0/3175577615 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3353338869 0 0) 0x7f7b9c0035b0 con 0x7f7bb40d0b10 2023-07-18T03:40:07.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.553+0000 7f7bb3fff700 1 -- 172.21.15.49:0/3175577615 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb40b0060 con 0x7f7bb40d0f90 2023-07-18T03:40:07.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.554+0000 7f7bb3fff700 1 --2- 172.21.15.49:0/3175577615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb40d0f90 0x7f7bb40b1530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009d90 tx=0x7f7bac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ac760b6642b91a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:07.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.554+0000 7f7bb27fc700 1 -- 172.21.15.49:0/3175577615 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00e070 con 0x7f7bb40d0f90 2023-07-18T03:40:07.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.554+0000 7f7bb27fc700 1 -- 172.21.15.49:0/3175577615 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac003a20 con 0x7f7bb40d0f90 2023-07-18T03:40:07.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.554+0000 7f7bb27fc700 1 -- 172.21.15.49:0/3175577615 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8001670 con 0x7f7bb40d0f90 2023-07-18T03:40:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.554+0000 7f7bba1ff700 1 -- 172.21.15.49:0/3175577615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb40d0f90 msgr2=0x7f7bb40b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.554+0000 7f7bba1ff700 1 --2- 172.21.15.49:0/3175577615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb40d0f90 0x7f7bb40b1530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009d90 tx=0x7f7bac00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.554+0000 7f7bba1ff700 1 -- 172.21.15.49:0/3175577615 shutdown_connections 2023-07-18T03:40:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.554+0000 7f7bba1ff700 1 --2- 172.21.15.49:0/3175577615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb40d0f90 0x7f7bb40b1530 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.554+0000 7f7bba1ff700 1 -- 172.21.15.49:0/3175577615 >> 172.21.15.49:0/3175577615 conn(0x7f7bb409c1b0 msgr2=0x7f7bb40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.555+0000 7f7bba1ff700 1 -- 172.21.15.49:0/3175577615 shutdown_connections 2023-07-18T03:40:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.555+0000 7f7bba1ff700 1 -- 172.21.15.49:0/3175577615 wait complete. 2023-07-18T03:40:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.556+0000 7f7bba1ff700 1 Processor -- start 2023-07-18T03:40:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.557+0000 7f7bba1ff700 1 -- start start 2023-07-18T03:40:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.557+0000 7f7bba1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb40b8a60 0x7f7bb40c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bba1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb40d0b10 0x7f7bb413fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bba1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bb40d0f90 0x7f7bb4140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bba1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb415f210 con 0x7f7bb40d0b10 2023-07-18T03:40:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bba1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40cda30 con 0x7f7bb40b8a60 2023-07-18T03:40:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bba1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb415ebc0 con 0x7f7bb40d0f90 2023-07-18T03:40:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb40d0b10 0x7f7bb413fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb40d0b10 0x7f7bb413fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38524/0 (socket says 172.21.15.49:38524) 2023-07-18T03:40:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/4064257981 learned_addr learned my addr 172.21.15.49:0/4064257981 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb40b8a60 0x7f7bb40c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bb3fff700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bb40d0f90 0x7f7bb4140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bb40d0f90 msgr2=0x7f7bb4140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bb40d0f90 0x7f7bb4140000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb40b8a60 msgr2=0x7f7bb40c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb40b8a60 0x7f7bb40c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.558+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/4064257981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac009a70 con 0x7f7bb40d0b10 2023-07-18T03:40:07.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.559+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb40d0b10 0x7f7bb413fac0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800c750 tx=0x7f7ba800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:07.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.559+0000 7f7b9bfff700 1 -- 172.21.15.49:0/4064257981 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8004c80 con 0x7f7bb40d0b10 2023-07-18T03:40:07.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.559+0000 7f7b9bfff700 1 -- 172.21.15.49:0/4064257981 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8004e00 con 0x7f7bb40d0b10 2023-07-18T03:40:07.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.559+0000 7f7bba1ff700 1 -- 172.21.15.49:0/4064257981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb415eea0 con 0x7f7bb40d0b10 2023-07-18T03:40:07.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.559+0000 7f7b9bfff700 1 -- 172.21.15.49:0/4064257981 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8007310 con 0x7f7bb40d0b10 2023-07-18T03:40:07.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.559+0000 7f7bba1ff700 1 -- 172.21.15.49:0/4064257981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb4153310 con 0x7f7bb40d0b10 2023-07-18T03:40:07.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.560+0000 7f7bba1ff700 1 -- 172.21.15.49:0/4064257981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb415f380 con 0x7f7bb40d0b10 2023-07-18T03:40:07.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.561+0000 7f7b9bfff700 1 -- 172.21.15.49:0/4064257981 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ba8007470 con 0x7f7bb40d0b10 2023-07-18T03:40:07.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.562+0000 7f7b9bfff700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7ba006ace0 0x7f7ba006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.562+0000 7f7b9bfff700 1 -- 172.21.15.49:0/4064257981 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7ba8089230 con 0x7f7bb40d0b10 2023-07-18T03:40:07.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.563+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7ba006ace0 0x7f7ba006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.564+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7ba006ace0 0x7f7ba006d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4001220 tx=0x7f7ba4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:07.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.567+0000 7f7b9bfff700 1 -- 172.21.15.49:0/4064257981 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7ba80541c0 con 0x7f7bb40d0b10 2023-07-18T03:40:07.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.863+0000 7f7bba1ff700 1 -- 172.21.15.49:0/4064257981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7bb4074350 con 0x7f7bb40d0b10 2023-07-18T03:40:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.867+0000 7f7b9bfff700 1 -- 172.21.15.49:0/4064257981 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7ba80577e0 con 0x7f7bb40d0b10 2023-07-18T03:40:07.861 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2023-07-18T03:40:07.862 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQD+CLZkI5fMMhAAisfdJO7kEPCDAngS1ywP+g== 2023-07-18T03:40:07.863 INFO:teuthology.orchestra.run.smithi049.stdout: caps mds = "allow" 2023-07-18T03:40:07.863 INFO:teuthology.orchestra.run.smithi049.stdout: caps mon = "allow r" 2023-07-18T03:40:07.863 INFO:teuthology.orchestra.run.smithi049.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-18T03:40:07.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.869+0000 7f7bba1ff700 1 -- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7ba006ace0 msgr2=0x7f7ba006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.869+0000 7f7bba1ff700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7ba006ace0 0x7f7ba006d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4001220 tx=0x7f7ba4009040 comp rx=0 tx=0).stop 2023-07-18T03:40:07.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.869+0000 7f7bba1ff700 1 -- 172.21.15.49:0/4064257981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb40d0b10 msgr2=0x7f7bb413fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.869+0000 7f7bba1ff700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb40d0b10 0x7f7bb413fac0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800c750 tx=0x7f7ba800ca60 comp rx=0 tx=0).stop 2023-07-18T03:40:07.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.869+0000 7f7bba1ff700 1 -- 172.21.15.49:0/4064257981 shutdown_connections 2023-07-18T03:40:07.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.869+0000 7f7bba1ff700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bb40d0f90 0x7f7bb4140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.869+0000 7f7bba1ff700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb40b8a60 0x7f7bb40c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.869+0000 7f7bba1ff700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7ba006ace0 0x7f7ba006d190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.869+0000 7f7bba1ff700 1 --2- 172.21.15.49:0/4064257981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb40d0b10 0x7f7bb413fac0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.869+0000 7f7bba1ff700 1 -- 172.21.15.49:0/4064257981 >> 172.21.15.49:0/4064257981 conn(0x7f7bb409c1b0 msgr2=0x7f7bb406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:07.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.870+0000 7f7bba1ff700 1 -- 172.21.15.49:0/4064257981 shutdown_connections 2023-07-18T03:40:07.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:07.870+0000 7f7bba1ff700 1 -- 172.21.15.49:0/4064257981 wait complete. 2023-07-18T03:40:07.868 INFO:teuthology.orchestra.run.smithi049.stderr:updated caps for client.0 2023-07-18T03:40:07.883 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:40:08.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.095+0000 7fc265dff700 1 Processor -- start 2023-07-18T03:40:08.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.096+0000 7fc265dff700 1 -- start start 2023-07-18T03:40:08.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.097+0000 7fc265dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc2600b3e90 0x7fc2600cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:08.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.097+0000 7fc265dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc26015a8c0 0x7fc26015ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:08.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.097+0000 7fc265dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2600ce110 con 0x7fc26015e890 2023-07-18T03:40:08.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.097+0000 7fc265dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2600cf380 con 0x7fc2600b3e90 2023-07-18T03:40:08.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.097+0000 7fc265dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26015b660 con 0x7fc26015a8c0 2023-07-18T03:40:08.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.097+0000 7fc25ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc26015a8c0 0x7fc26015ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:08.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.097+0000 7fc257fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc2600b3e90 0x7fc2600cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:08.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.097+0000 7fc25ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc26015a8c0 0x7fc26015ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:57402/0 (socket says 172.21.15.49:57402) 2023-07-18T03:40:08.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.097+0000 7fc257fff700 1 --2- 172.21.15.49:0/1783520514 >> v2:172.21.15.107:3300/0 conn(0x7fc2600b3e90 0x7fc2600cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:58042/0 (socket says 172.21.15.49:58042) 2023-07-18T03:40:08.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.097+0000 7fc25ffff700 1 -- 172.21.15.49:0/1783520514 learned_addr learned my addr 172.21.15.49:0/1783520514 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:08.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.098+0000 7fc257fff700 1 -- 172.21.15.49:0/1783520514 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc26015a8c0 msgr2=0x7fc26015ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:08.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.098+0000 7fc257fff700 1 --2- 172.21.15.49:0/1783520514 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc26015a8c0 0x7fc26015ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.098+0000 7fc257fff700 1 -- 172.21.15.49:0/1783520514 >> v1:172.21.15.107:6789/0 conn(0x7fc26015e890 legacy=0x7fc2600b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:08.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.098+0000 7fc257fff700 1 -- 172.21.15.49:0/1783520514 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2600ce360 con 0x7fc2600b3e90 2023-07-18T03:40:08.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.098+0000 7fc257fff700 1 --2- 172.21.15.49:0/1783520514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2600b3e90 0x7fc2600cee40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc248009840 tx=0x7fc248009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3aecc7fa32fc4fa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:08.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.098+0000 7fc25effd700 1 -- 172.21.15.49:0/1783520514 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248011830 con 0x7fc2600b3e90 2023-07-18T03:40:08.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.099+0000 7fc25effd700 1 -- 172.21.15.49:0/1783520514 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc248011990 con 0x7fc2600b3e90 2023-07-18T03:40:08.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.099+0000 7fc25effd700 1 -- 172.21.15.49:0/1783520514 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248011d20 con 0x7fc2600b3e90 2023-07-18T03:40:08.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.099+0000 7fc265dff700 1 -- 172.21.15.49:0/1783520514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2600b3e90 msgr2=0x7fc2600cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:08.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.099+0000 7fc265dff700 1 --2- 172.21.15.49:0/1783520514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2600b3e90 0x7fc2600cee40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc248009840 tx=0x7fc248009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:08.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.099+0000 7fc265dff700 1 -- 172.21.15.49:0/1783520514 shutdown_connections 2023-07-18T03:40:08.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.099+0000 7fc265dff700 1 --2- 172.21.15.49:0/1783520514 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc26015a8c0 0x7fc26015ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.099+0000 7fc265dff700 1 --2- 172.21.15.49:0/1783520514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2600b3e90 0x7fc2600cee40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.099+0000 7fc265dff700 1 -- 172.21.15.49:0/1783520514 >> 172.21.15.49:0/1783520514 conn(0x7fc260069f30 msgr2=0x7fc26006a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:08.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.099+0000 7fc265dff700 1 -- 172.21.15.49:0/1783520514 shutdown_connections 2023-07-18T03:40:08.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.099+0000 7fc265dff700 1 -- 172.21.15.49:0/1783520514 wait complete. 2023-07-18T03:40:08.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.101+0000 7fc265dff700 1 Processor -- start 2023-07-18T03:40:08.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.101+0000 7fc265dff700 1 -- start start 2023-07-18T03:40:08.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.102+0000 7fc265dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2600b3e90 0x7fc2601505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:08.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.102+0000 7fc265dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc26015a8c0 0x7fc2600b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:08.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.102+0000 7fc265dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc26015e890 0x7fc2601e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:08.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.102+0000 7fc265dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2601f53e0 con 0x7fc2600b3e90 2023-07-18T03:40:08.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.102+0000 7fc265dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc260150af0 con 0x7fc26015e890 2023-07-18T03:40:08.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.102+0000 7fc265dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2601f22b0 con 0x7fc26015a8c0 2023-07-18T03:40:08.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.103+0000 7fc25f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2600b3e90 0x7fc2601505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:08.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.103+0000 7fc25f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2600b3e90 0x7fc2601505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38536/0 (socket says 172.21.15.49:38536) 2023-07-18T03:40:08.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.103+0000 7fc25f7fe700 1 -- 172.21.15.49:0/2338157309 learned_addr learned my addr 172.21.15.49:0/2338157309 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:08.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.103+0000 7fc257fff700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc26015a8c0 0x7fc2600b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:08.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.103+0000 7fc25ffff700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc26015e890 0x7fc2601e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:08.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.103+0000 7fc25f7fe700 1 -- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc26015a8c0 msgr2=0x7fc2600b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.103+0000 7fc25f7fe700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc26015a8c0 0x7fc2600b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.103+0000 7fc25f7fe700 1 -- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc26015e890 msgr2=0x7fc2601e5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.103+0000 7fc25f7fe700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc26015e890 0x7fc2601e5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.103+0000 7fc25f7fe700 1 -- 172.21.15.49:0/2338157309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2480094a0 con 0x7fc2600b3e90 2023-07-18T03:40:08.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.103+0000 7fc25f7fe700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2600b3e90 0x7fc2601505b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc25000c450 tx=0x7fc25000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:08.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.104+0000 7fc25cff9700 1 -- 172.21.15.49:0/2338157309 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2500050d0 con 0x7fc2600b3e90 2023-07-18T03:40:08.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.104+0000 7fc25cff9700 1 -- 172.21.15.49:0/2338157309 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc250005230 con 0x7fc2600b3e90 2023-07-18T03:40:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.104+0000 7fc25cff9700 1 -- 172.21.15.49:0/2338157309 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250003680 con 0x7fc2600b3e90 2023-07-18T03:40:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.104+0000 7fc265dff700 1 -- 172.21.15.49:0/2338157309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2601f2590 con 0x7fc2600b3e90 2023-07-18T03:40:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.104+0000 7fc265dff700 1 -- 172.21.15.49:0/2338157309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2601f2b50 con 0x7fc2600b3e90 2023-07-18T03:40:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.105+0000 7fc265dff700 1 -- 172.21.15.49:0/2338157309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2601f5550 con 0x7fc2600b3e90 2023-07-18T03:40:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.106+0000 7fc25cff9700 1 -- 172.21.15.49:0/2338157309 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2500037e0 con 0x7fc2600b3e90 2023-07-18T03:40:08.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.107+0000 7fc25cff9700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc24006ace0 0x7fc24006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:08.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.107+0000 7fc25cff9700 1 -- 172.21.15.49:0/2338157309 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc250088750 con 0x7fc2600b3e90 2023-07-18T03:40:08.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.107+0000 7fc257fff700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc24006ace0 0x7fc24006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:08.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.108+0000 7fc257fff700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc24006ace0 0x7fc24006d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc2600b4280 tx=0x7fc248008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:08.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.111+0000 7fc25cff9700 1 -- 172.21.15.49:0/2338157309 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc2601f5550 con 0x7fc2600b3e90 2023-07-18T03:40:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.364+0000 7fc265dff700 1 -- 172.21.15.49:0/2338157309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2601f5520 con 0x7fc2600b3e90 2023-07-18T03:40:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.365+0000 7fc25cff9700 1 -- 172.21.15.49:0/2338157309 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fc250053800 con 0x7fc2600b3e90 2023-07-18T03:40:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:08.360 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1095518881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1095518881},{"type":"v1","addr":"172.21.15.49:6834","nonce":1095518881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/843626338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":843626338},{"type":"v1","addr":"172.21.15.107:6835","nonce":843626338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3238680863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3238680863},{"type":"v1","addr":"172.21.15.107:6837","nonce":3238680863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:39:56.844870+0000","modified":"2023-07-18T03:40:00.356383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5172":{"gid":5172,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6835/3161575771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":3161575771},{"type":"v1","addr":"172.21.15.49:6835","nonce":3161575771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-18T03:40:08.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.367+0000 7fc265dff700 1 -- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc24006ace0 msgr2=0x7fc24006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:08.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.367+0000 7fc265dff700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc24006ace0 0x7fc24006d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc2600b4280 tx=0x7fc248008030 comp rx=0 tx=0).stop 2023-07-18T03:40:08.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.367+0000 7fc265dff700 1 -- 172.21.15.49:0/2338157309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2600b3e90 msgr2=0x7fc2601505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:08.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.367+0000 7fc265dff700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2600b3e90 0x7fc2601505b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc25000c450 tx=0x7fc25000c760 comp rx=0 tx=0).stop 2023-07-18T03:40:08.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.367+0000 7fc265dff700 1 -- 172.21.15.49:0/2338157309 shutdown_connections 2023-07-18T03:40:08.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.367+0000 7fc265dff700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc26015e890 0x7fc2601e5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.367+0000 7fc265dff700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc24006ace0 0x7fc24006d190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.367+0000 7fc265dff700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2600b3e90 0x7fc2601505b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.367+0000 7fc265dff700 1 --2- 172.21.15.49:0/2338157309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc26015a8c0 0x7fc2600b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.367+0000 7fc265dff700 1 -- 172.21.15.49:0/2338157309 >> 172.21.15.49:0/2338157309 conn(0x7fc260069f30 msgr2=0x7fc26006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:08.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.368+0000 7fc265dff700 1 -- 172.21.15.49:0/2338157309 shutdown_connections 2023-07-18T03:40:08.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:08.368+0000 7fc265dff700 1 -- 172.21.15.49:0/2338157309 wait complete. 2023-07-18T03:40:08.364 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 18 2023-07-18T03:40:08.784 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-18T03:39:56.844870+0000', 'modified': '2023-07-18T03:40:00.356383+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5172}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5172': {'gid': 5172, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.49:6835/3161575771', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6833', 'nonce': 3161575771}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 3161575771}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-18T03:40:08.784 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-18T03:40:08.785 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-18T03:40:08.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.000+0000 7f2b6d7ff700 1 Processor -- start 2023-07-18T03:40:08.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.000+0000 7f2b6d7ff700 1 -- start start 2023-07-18T03:40:08.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.001+0000 7f2b6d7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2b6806c9f0 0x7f2b680b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:08.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.001+0000 7f2b6d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b680b1e50 0x7f2b68071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:08.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.001+0000 7f2b6d7ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6806bcc0 con 0x7f2b6815d9d0 2023-07-18T03:40:08.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.001+0000 7f2b6d7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6806c4e0 con 0x7f2b6806c9f0 2023-07-18T03:40:08.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.001+0000 7f2b6d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6815dda0 con 0x7f2b680b1e50 2023-07-18T03:40:08.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.001+0000 7f2b66ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2b6815d9d0 0x7f2b6806b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49154/0 (socket says 172.21.15.49:49154) 2023-07-18T03:40:08.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.001+0000 7f2b66ffd700 1 -- 172.21.15.49:0/2854306100 learned_addr learned my addr 172.21.15.49:0/2854306100 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:08.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.002+0000 7f2b667fc700 1 --2- 172.21.15.49:0/2854306100 >> v2:172.21.15.107:3300/0 conn(0x7f2b6806c9f0 0x7f2b680b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:08.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.002+0000 7f2b677fe700 1 --2- 172.21.15.49:0/2854306100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b680b1e50 0x7f2b68071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:08.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.002+0000 7f2b65ffb700 1 -- 172.21.15.49:0/2854306100 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409250284 0 0) 0x7f2b6806bcc0 con 0x7f2b6815d9d0 2023-07-18T03:40:08.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.002+0000 7f2b65ffb700 1 -- 172.21.15.49:0/2854306100 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b500035b0 con 0x7f2b6815d9d0 2023-07-18T03:40:09.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.002+0000 7f2b667fc700 1 -- 172.21.15.49:0/2854306100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b680b1e50 msgr2=0x7f2b68071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:09.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.002+0000 7f2b667fc700 1 --2- 172.21.15.49:0/2854306100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b680b1e50 0x7f2b68071170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.002+0000 7f2b667fc700 1 -- 172.21.15.49:0/2854306100 >> v1:172.21.15.49:6789/0 conn(0x7f2b6815d9d0 legacy=0x7f2b6806b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.002+0000 7f2b667fc700 1 -- 172.21.15.49:0/2854306100 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6806bf20 con 0x7f2b6806c9f0 2023-07-18T03:40:09.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.002+0000 7f2b677fe700 1 --2- 172.21.15.49:0/2854306100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b680b1e50 0x7f2b68071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:09.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.002+0000 7f2b667fc700 1 --2- 172.21.15.49:0/2854306100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b6806c9f0 0x7f2b680b1910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009840 tx=0x7f2b5c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=868b35aa20b5d6b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:09.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.003+0000 7f2b65ffb700 1 -- 172.21.15.49:0/2854306100 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c011830 con 0x7f2b6806c9f0 2023-07-18T03:40:09.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.003+0000 7f2b65ffb700 1 -- 172.21.15.49:0/2854306100 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c011990 con 0x7f2b6806c9f0 2023-07-18T03:40:09.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.003+0000 7f2b65ffb700 1 -- 172.21.15.49:0/2854306100 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c011d20 con 0x7f2b6806c9f0 2023-07-18T03:40:09.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.003+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2854306100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b6806c9f0 msgr2=0x7f2b680b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.003+0000 7f2b6d7ff700 1 --2- 172.21.15.49:0/2854306100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b6806c9f0 0x7f2b680b1910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009840 tx=0x7f2b5c009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:09.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.003+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2854306100 shutdown_connections 2023-07-18T03:40:09.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.003+0000 7f2b6d7ff700 1 --2- 172.21.15.49:0/2854306100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b680b1e50 0x7f2b68071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.003+0000 7f2b6d7ff700 1 --2- 172.21.15.49:0/2854306100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b6806c9f0 0x7f2b680b1910 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.003+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2854306100 >> 172.21.15.49:0/2854306100 conn(0x7f2b680b05d0 msgr2=0x7f2b680b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:09.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.003+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2854306100 shutdown_connections 2023-07-18T03:40:09.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.003+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2854306100 wait complete. 2023-07-18T03:40:09.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.005+0000 7f2b6d7ff700 1 Processor -- start 2023-07-18T03:40:09.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.005+0000 7f2b6d7ff700 1 -- start start 2023-07-18T03:40:09.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.006+0000 7f2b6d7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6806c9f0 0x7f2b681efac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.006+0000 7f2b6d7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b680b1e50 0x7f2b681f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.006+0000 7f2b6d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b6815d9d0 0x7f2b680c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.006+0000 7f2b6d7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680c7410 con 0x7f2b6806c9f0 2023-07-18T03:40:09.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.006+0000 7f2b6d7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680c5440 con 0x7f2b680b1e50 2023-07-18T03:40:09.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.006+0000 7f2b6d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680c6ae0 con 0x7f2b6815d9d0 2023-07-18T03:40:09.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.006+0000 7f2b66ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6806c9f0 0x7f2b681efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.007+0000 7f2b66ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6806c9f0 0x7f2b681efac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48930/0 (socket says 172.21.15.49:48930) 2023-07-18T03:40:09.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.007+0000 7f2b66ffd700 1 -- 172.21.15.49:0/2787499168 learned_addr learned my addr 172.21.15.49:0/2787499168 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.007+0000 7f2b667fc700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b680b1e50 0x7f2b681f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.007+0000 7f2b677fe700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b6815d9d0 0x7f2b680c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.007+0000 7f2b66ffd700 1 -- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b6815d9d0 msgr2=0x7f2b680c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.007+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b6815d9d0 0x7f2b680c4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.007+0000 7f2b66ffd700 1 -- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b680b1e50 msgr2=0x7f2b681f2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.007+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b680b1e50 0x7f2b681f2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.007+0000 7f2b66ffd700 1 -- 172.21.15.49:0/2787499168 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b60009a70 con 0x7f2b6806c9f0 2023-07-18T03:40:09.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.007+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6806c9f0 0x7f2b681efac0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2b58001eb0 tx=0x7f2b5800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:09.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.007+0000 7f2b4f7fe700 1 -- 172.21.15.49:0/2787499168 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5800cea0 con 0x7f2b6806c9f0 2023-07-18T03:40:09.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.008+0000 7f2b4f7fe700 1 -- 172.21.15.49:0/2787499168 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b58005d70 con 0x7f2b6806c9f0 2023-07-18T03:40:09.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.008+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2787499168 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5c0094a0 con 0x7f2b6806c9f0 2023-07-18T03:40:09.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.008+0000 7f2b4f7fe700 1 -- 172.21.15.49:0/2787499168 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b580072f0 con 0x7f2b6806c9f0 2023-07-18T03:40:09.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.008+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2787499168 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b680c70a0 con 0x7f2b6806c9f0 2023-07-18T03:40:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.010+0000 7f2b4f7fe700 1 -- 172.21.15.49:0/2787499168 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b58007450 con 0x7f2b6806c9f0 2023-07-18T03:40:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.011+0000 7f2b4f7fe700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2b5406ace0 0x7f2b5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.012+0000 7f2b4f7fe700 1 -- 172.21.15.49:0/2787499168 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b580888a0 con 0x7f2b6806c9f0 2023-07-18T03:40:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.012+0000 7f2b667fc700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2b5406ace0 0x7f2b5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.013+0000 7f2b667fc700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2b5406ace0 0x7f2b5406d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2b6806cdc0 tx=0x7f2b5c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.013+0000 7f2b81101b80 1 Processor -- start 2023-07-18T03:40:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.014+0000 7f2b81101b80 1 -- start start 2023-07-18T03:40:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.014+0000 7f2b81101b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e1c07f0 0x55852e199420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.014+0000 7f2b81101b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c0bc0 0x55852e1de010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b81101b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e1de550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b81101b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852e09dc60 con 0x55852e1c0bc0 2023-07-18T03:40:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b81101b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852e0f09f0 con 0x55852e1c07f0 2023-07-18T03:40:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b81101b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55852e0e9e80 con 0x55852e193840 2023-07-18T03:40:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b667fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c0bc0 0x55852e1de010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b66ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e1c07f0 0x55852e199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b667fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c0bc0 0x55852e1de010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48944/0 (socket says 172.21.15.49:48944) 2023-07-18T03:40:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b667fc700 1 -- 172.21.15.49:0/2821296713 learned_addr learned my addr 172.21.15.49:0/2821296713 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b677fe700 1 --2- 172.21.15.49:0/2821296713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e1de550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b66ffd700 1 -- 172.21.15.49:0/2821296713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 msgr2=0x55852e1de550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/2821296713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e1de550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b66ffd700 1 -- 172.21.15.49:0/2821296713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c0bc0 msgr2=0x55852e1de010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/2821296713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c0bc0 0x55852e1de010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b66ffd700 1 -- 172.21.15.49:0/2821296713 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55852df17d90 con 0x55852e1c07f0 2023-07-18T03:40:09.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.015+0000 7f2b667fc700 1 --2- 172.21.15.49:0/2821296713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c0bc0 0x55852e1de010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:09.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.016+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/2821296713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e1c07f0 0x55852e199420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2b58086c30 tx=0x7f2b58057050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11bc6f988d939f6d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:09.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.016+0000 7f2b4cff9700 1 -- 172.21.15.49:0/2821296713 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5805f950 con 0x55852e1c07f0 2023-07-18T03:40:09.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.016+0000 7f2b4cff9700 1 -- 172.21.15.49:0/2821296713 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b580559a0 con 0x55852e1c07f0 2023-07-18T03:40:09.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.016+0000 7f2b4cff9700 1 -- 172.21.15.49:0/2821296713 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5805f4c0 con 0x55852e1c07f0 2023-07-18T03:40:09.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.016+0000 7f2b81101b80 1 -- 172.21.15.49:0/2821296713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e1c07f0 msgr2=0x55852e199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.016+0000 7f2b81101b80 1 --2- 172.21.15.49:0/2821296713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e1c07f0 0x55852e199420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2b58086c30 tx=0x7f2b58057050 comp rx=0 tx=0).stop 2023-07-18T03:40:09.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.017+0000 7f2b81101b80 1 -- 172.21.15.49:0/2821296713 shutdown_connections 2023-07-18T03:40:09.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.017+0000 7f2b81101b80 1 --2- 172.21.15.49:0/2821296713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e1de550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.017+0000 7f2b81101b80 1 --2- 172.21.15.49:0/2821296713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e1c07f0 0x55852e199420 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.017+0000 7f2b81101b80 1 --2- 172.21.15.49:0/2821296713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c0bc0 0x55852e1de010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.017+0000 7f2b81101b80 1 -- 172.21.15.49:0/2821296713 >> 172.21.15.49:0/2821296713 conn(0x55852e18db90 msgr2=0x55852e193130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:09.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.017+0000 7f2b81101b80 1 -- 172.21.15.49:0/2821296713 shutdown_connections 2023-07-18T03:40:09.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.017+0000 7f2b81101b80 1 -- 172.21.15.49:0/2821296713 wait complete. 2023-07-18T03:40:09.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.018+0000 7f2b81101b80 1 -- start start 2023-07-18T03:40:09.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.018+0000 7f2b81101b80 1 Processor -- start 2023-07-18T03:40:09.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b81101b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e193840 0x55852e1c4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b81101b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e1c07f0 0x55852e1c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b81101b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e1c0bc0 0x55852e1c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b81101b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852e089a70 con 0x55852e193840 2023-07-18T03:40:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b81101b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852e089770 con 0x55852e1c0bc0 2023-07-18T03:40:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b81101b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55852df6d8d0 con 0x55852e1c07f0 2023-07-18T03:40:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b81101b80 10 client.0 _ll_register_callbacks cb 0x55852e087710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b66ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e193840 0x55852e1c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b66ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e193840 0x55852e1c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48954/0 (socket says 172.21.15.49:48954) 2023-07-18T03:40:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b66ffd700 1 -- 172.21.15.49:0/1073899361 learned_addr learned my addr 172.21.15.49:0/1073899361 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b677fe700 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e1c0bc0 0x55852e1c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b667fc700 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e1c07f0 0x55852e1c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.020+0000 7f2b66ffd700 1 -- 172.21.15.49:0/1073899361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e1c07f0 msgr2=0x55852e1c19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:09.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e1c07f0 0x55852e1c19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b66ffd700 1 -- 172.21.15.49:0/1073899361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e1c0bc0 msgr2=0x55852e1c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:09.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e1c0bc0 0x55852e1c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b66ffd700 1 -- 172.21.15.49:0/1073899361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6806bf20 con 0x55852e193840 2023-07-18T03:40:09.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b667fc700 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e1c07f0 0x55852e1c19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:09.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e193840 0x55852e1c4510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2b580011e0 tx=0x7f2b5805b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eca72493f016974 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-07-18T03:40:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b45ffb700 1 -- 172.21.15.49:0/1073899361 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58072070 con 0x55852e193840 2023-07-18T03:40:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b45ffb700 1 -- 172.21.15.49:0/1073899361 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b5805b0b0 con 0x55852e193840 2023-07-18T03:40:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b81101b80 1 -- 172.21.15.49:0/1073899361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55852e18b1a0 con 0x55852e193840 2023-07-18T03:40:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b45ffb700 1 -- 172.21.15.49:0/1073899361 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b58076080 con 0x55852e193840 2023-07-18T03:40:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b81101b80 10 client.5379 fetch_fsmap learned FSMap version 18 2023-07-18T03:40:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b81101b80 1 -- 172.21.15.49:0/1073899361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x7f2b680c70a0 con 0x55852e193840 2023-07-18T03:40:09.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.021+0000 7f2b45ffb700 1 -- 172.21.15.49:0/1073899361 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55852e18b1a0 con 0x55852e193840 2023-07-18T03:40:09.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.022+0000 7f2b45ffb700 1 -- 172.21.15.49:0/1073899361 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 18) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2b5805b270 con 0x55852e193840 2023-07-18T03:40:09.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.022+0000 7f2b81101b80 10 client.5379 fetch_fsmap finished waiting for FSMap version 18 2023-07-18T03:40:09.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.022+0000 7f2b81101b80 10 client.5379 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-18T03:40:09.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.022+0000 7f2b81101b80 20 client.5379 populate_metadata read hostname 'smithi049' 2023-07-18T03:40:09.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.022+0000 7f2b81101b80 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.49:6833/3161575771,v1:172.21.15.49:6835/3161575771] conn(0x55852e1b7690 0x55852e1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.022+0000 7f2b81101b80 4 client.5379 mds_command: new command op to 5172 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:40:09.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.022+0000 7f2b81101b80 1 -- 172.21.15.49:0/1073899361 --> [v2:172.21.15.49:6833/3161575771,v1:172.21.15.49:6835/3161575771] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55852e0e2e60 con 0x55852e1b7690 2023-07-18T03:40:09.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.022+0000 7f2b667fc700 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.49:6833/3161575771,v1:172.21.15.49:6835/3161575771] conn(0x55852e1b7690 0x55852e1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.023+0000 7f2b45ffb700 10 client.5379 ms_handle_connect on v2:172.21.15.49:6833/3161575771 2023-07-18T03:40:09.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.023+0000 7f2b667fc700 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.49:6833/3161575771,v1:172.21.15.49:6835/3161575771] conn(0x55852e1b7690 0x55852e1970f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d088eddc594ffda2 server_cookie=22ba9ef30948f22a in_seq=0 out_seq=0 2023-07-18T03:40:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.023+0000 7f2b45ffb700 1 -- 172.21.15.49:0/1073899361 <== mds.0 v2:172.21.15.49:6833/3161575771 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55852e0e2e60 con 0x55852e1b7690 2023-07-18T03:40:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.023+0000 7f2b45ffb700 10 client.5379 handle_command_reply: tid=0 2023-07-18T03:40:09.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.023+0000 7f2b81101b80 1 client.5379 shutdown 2023-07-18T03:40:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.024+0000 7f2b81101b80 1 -- 172.21.15.49:0/1073899361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e193840 msgr2=0x55852e1c4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.024+0000 7f2b81101b80 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e193840 0x55852e1c4510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2b580011e0 tx=0x7f2b5805b040 comp rx=0 tx=0).stop 2023-07-18T03:40:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.024+0000 7f2b81101b80 1 -- 172.21.15.49:0/1073899361 shutdown_connections 2023-07-18T03:40:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.024+0000 7f2b81101b80 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.49:6833/3161575771,v1:172.21.15.49:6835/3161575771] conn(0x55852e1b7690 0x55852e1970f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.024+0000 7f2b81101b80 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e1c0bc0 0x55852e1c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.024+0000 7f2b81101b80 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e193840 0x55852e1c4510 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.024+0000 7f2b45ffb700 0 client.5379 ms_handle_reset on v2:172.21.15.49:6833/3161575771 2023-07-18T03:40:09.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.024+0000 7f2b81101b80 1 --2- 172.21.15.49:0/1073899361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e1c07f0 0x55852e1c19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.024+0000 7f2b81101b80 1 -- 172.21.15.49:0/1073899361 >> 172.21.15.49:0/1073899361 conn(0x55852e18db90 msgr2=0x55852e198140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:09.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.024+0000 7f2b81101b80 1 -- 172.21.15.49:0/1073899361 shutdown_connections 2023-07-18T03:40:09.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.024+0000 7f2b81101b80 1 -- 172.21.15.49:0/1073899361 wait complete. 2023-07-18T03:40:09.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.024+0000 7f2b81101b80 20 client.5379 trim_cache size 0 max 16384 2023-07-18T03:40:09.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.042+0000 7f2b81101b80 1 Processor -- start 2023-07-18T03:40:09.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.043+0000 7f2b81101b80 1 -- start start 2023-07-18T03:40:09.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.043+0000 7f2b81101b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e18db90 0x55852e1c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.043+0000 7f2b81101b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e1bf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.043+0000 7f2b81101b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 0x55852e1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.043+0000 7f2b81101b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852e198f30 con 0x55852e1c07f0 2023-07-18T03:40:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.043+0000 7f2b81101b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852e1c1770 con 0x55852e18db90 2023-07-18T03:40:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.043+0000 7f2b81101b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55852e1d7f00 con 0x55852e193840 2023-07-18T03:40:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.044+0000 7f2b677fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 0x55852e1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.044+0000 7f2b677fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 0x55852e1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48956/0 (socket says 172.21.15.49:48956) 2023-07-18T03:40:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.044+0000 7f2b677fe700 1 -- 172.21.15.49:0/825080503 learned_addr learned my addr 172.21.15.49:0/825080503 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:09.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.044+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/825080503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e18db90 0x55852e1c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.044+0000 7f2b667fc700 1 --2- 172.21.15.49:0/825080503 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e1bf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.044+0000 7f2b677fe700 1 -- 172.21.15.49:0/825080503 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 msgr2=0x55852e1bf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:09.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.044+0000 7f2b677fe700 1 --2- 172.21.15.49:0/825080503 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e1bf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.044+0000 7f2b677fe700 1 -- 172.21.15.49:0/825080503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e18db90 msgr2=0x55852e1c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:09.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.044+0000 7f2b677fe700 1 --2- 172.21.15.49:0/825080503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e18db90 0x55852e1c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.044+0000 7f2b677fe700 1 -- 172.21.15.49:0/825080503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55852df00c10 con 0x55852e1c07f0 2023-07-18T03:40:09.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.044+0000 7f2b677fe700 1 --2- 172.21.15.49:0/825080503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 0x55852e1989f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2b600177f0 tx=0x7f2b60009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e2fc8fc4dd17f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:09.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.045+0000 7f2b4d7fa700 1 -- 172.21.15.49:0/825080503 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000e040 con 0x55852e1c07f0 2023-07-18T03:40:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.045+0000 7f2b4d7fa700 1 -- 172.21.15.49:0/825080503 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6000f040 con 0x55852e1c07f0 2023-07-18T03:40:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.045+0000 7f2b4d7fa700 1 -- 172.21.15.49:0/825080503 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b600117c0 con 0x55852e1c07f0 2023-07-18T03:40:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.045+0000 7f2b81101b80 1 -- 172.21.15.49:0/825080503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 msgr2=0x55852e1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.045+0000 7f2b81101b80 1 --2- 172.21.15.49:0/825080503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 0x55852e1989f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2b600177f0 tx=0x7f2b60009750 comp rx=0 tx=0).stop 2023-07-18T03:40:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.045+0000 7f2b81101b80 1 -- 172.21.15.49:0/825080503 shutdown_connections 2023-07-18T03:40:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.045+0000 7f2b81101b80 1 --2- 172.21.15.49:0/825080503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 0x55852e1989f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.045+0000 7f2b81101b80 1 --2- 172.21.15.49:0/825080503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e18db90 0x55852e1c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.045+0000 7f2b81101b80 1 --2- 172.21.15.49:0/825080503 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e1bf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.045+0000 7f2b81101b80 1 -- 172.21.15.49:0/825080503 >> 172.21.15.49:0/825080503 conn(0x55852e197d40 msgr2=0x55852e1c6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.045+0000 7f2b81101b80 1 -- 172.21.15.49:0/825080503 shutdown_connections 2023-07-18T03:40:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.045+0000 7f2b81101b80 1 -- 172.21.15.49:0/825080503 wait complete. 2023-07-18T03:40:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.046+0000 7f2b81101b80 1 -- start start 2023-07-18T03:40:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.046+0000 7f2b81101b80 1 Processor -- start 2023-07-18T03:40:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.047+0000 7f2b81101b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e18db90 0x55852e1dc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.047+0000 7f2b81101b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e232240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.047+0000 7f2b81101b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 0x55852e1c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.047+0000 7f2b81101b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852e232940 con 0x55852e1c07f0 2023-07-18T03:40:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.047+0000 7f2b81101b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852e1c1cd0 con 0x55852e18db90 2023-07-18T03:40:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.047+0000 7f2b81101b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55852e1de890 con 0x55852e193840 2023-07-18T03:40:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.047+0000 7f2b81101b80 10 client.0 _ll_register_callbacks cb 0x7f2b5c0093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.048+0000 7f2b677fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 0x55852e1c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.048+0000 7f2b677fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 0x55852e1c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48968/0 (socket says 172.21.15.49:48968) 2023-07-18T03:40:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.048+0000 7f2b677fe700 1 -- 172.21.15.49:0/1929042966 learned_addr learned my addr 172.21.15.49:0/1929042966 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:09.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.048+0000 7f2b667fc700 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e232240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.048+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e18db90 0x55852e1dc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.048+0000 7f2b677fe700 1 -- 172.21.15.49:0/1929042966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 msgr2=0x55852e232240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:09.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.048+0000 7f2b677fe700 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e232240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.048+0000 7f2b677fe700 1 -- 172.21.15.49:0/1929042966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e18db90 msgr2=0x55852e1dc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:09.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.048+0000 7f2b677fe700 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e18db90 0x55852e1dc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.048+0000 7f2b677fe700 1 -- 172.21.15.49:0/1929042966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b60017470 con 0x55852e1c07f0 2023-07-18T03:40:09.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.048+0000 7f2b667fc700 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e232240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:09.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.048+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e18db90 0x55852e1dc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:09.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b677fe700 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 0x55852e1c69a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009790 tx=0x7f2b60004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9377c9676cb2913a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:09.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-07-18T03:40:09.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b45ffb700 1 -- 172.21.15.49:0/1929042966 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000e040 con 0x55852e1c07f0 2023-07-18T03:40:09.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b45ffb700 1 -- 172.21.15.49:0/1929042966 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b60003c20 con 0x55852e1c07f0 2023-07-18T03:40:09.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b81101b80 1 -- 172.21.15.49:0/1929042966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b580559a0 con 0x55852e1c07f0 2023-07-18T03:40:09.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b45ffb700 1 -- 172.21.15.49:0/1929042966 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b60017300 con 0x55852e1c07f0 2023-07-18T03:40:09.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b81101b80 10 client.5385 fetch_fsmap learned FSMap version 18 2023-07-18T03:40:09.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b81101b80 1 -- 172.21.15.49:0/1929042966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55852df3e200 con 0x55852e1c07f0 2023-07-18T03:40:09.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b45ffb700 1 -- 172.21.15.49:0/1929042966 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b580559a0 con 0x55852e1c07f0 2023-07-18T03:40:09.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b45ffb700 1 -- 172.21.15.49:0/1929042966 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 18) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2b60030060 con 0x55852e1c07f0 2023-07-18T03:40:09.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b81101b80 10 client.5385 fetch_fsmap finished waiting for FSMap version 18 2023-07-18T03:40:09.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b81101b80 10 client.5385 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-18T03:40:09.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.049+0000 7f2b81101b80 20 client.5385 populate_metadata read hostname 'smithi049' 2023-07-18T03:40:09.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.050+0000 7f2b81101b80 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.49:6833/3161575771,v1:172.21.15.49:6835/3161575771] conn(0x55852e1c0bc0 0x55852e1da990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.050+0000 7f2b81101b80 4 client.5385 mds_command: new command op to 5172 tid=0[{"prefix": "status"}] 2023-07-18T03:40:09.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.050+0000 7f2b81101b80 1 -- 172.21.15.49:0/1929042966 --> [v2:172.21.15.49:6833/3161575771,v1:172.21.15.49:6835/3161575771] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55852e0a5000 con 0x55852e1c0bc0 2023-07-18T03:40:09.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.050+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.49:6833/3161575771,v1:172.21.15.49:6835/3161575771] conn(0x55852e1c0bc0 0x55852e1da990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.050+0000 7f2b66ffd700 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.49:6833/3161575771,v1:172.21.15.49:6835/3161575771] conn(0x55852e1c0bc0 0x55852e1da990 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1926e44bac113d6c server_cookie=8da679bba26c7f8a in_seq=0 out_seq=0 2023-07-18T03:40:09.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.050+0000 7f2b45ffb700 10 client.5385 ms_handle_connect on v2:172.21.15.49:6833/3161575771 2023-07-18T03:40:09.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.050+0000 7f2b45ffb700 1 -- 172.21.15.49:0/1929042966 <== mds.0 v2:172.21.15.49:6833/3161575771 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55852e0a5000 con 0x55852e1c0bc0 2023-07-18T03:40:09.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.050+0000 7f2b45ffb700 10 client.5385 handle_command_reply: tid=0 2023-07-18T03:40:09.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.050+0000 7f2b81101b80 1 client.5385 shutdown 2023-07-18T03:40:09.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.051+0000 7f2b81101b80 1 -- 172.21.15.49:0/1929042966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 msgr2=0x55852e1c69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.051+0000 7f2b81101b80 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 0x55852e1c69a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009790 tx=0x7f2b60004b40 comp rx=0 tx=0).stop 2023-07-18T03:40:09.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.051+0000 7f2b81101b80 1 -- 172.21.15.49:0/1929042966 shutdown_connections 2023-07-18T03:40:09.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.051+0000 7f2b81101b80 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.49:6833/3161575771,v1:172.21.15.49:6835/3161575771] conn(0x55852e1c0bc0 0x55852e1da990 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.051+0000 7f2b81101b80 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55852e1c07f0 0x55852e1c69a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.051+0000 7f2b81101b80 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852e18db90 0x55852e1dc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.051+0000 7f2b45ffb700 0 client.5385 ms_handle_reset on v2:172.21.15.49:6833/3161575771 2023-07-18T03:40:09.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.051+0000 7f2b81101b80 1 --2- 172.21.15.49:0/1929042966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55852e193840 0x55852e232240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.051+0000 7f2b81101b80 1 -- 172.21.15.49:0/1929042966 >> 172.21.15.49:0/1929042966 conn(0x55852e197d40 msgr2=0x55852e1c5410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:09.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.051+0000 7f2b81101b80 1 -- 172.21.15.49:0/1929042966 shutdown_connections 2023-07-18T03:40:09.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.051+0000 7f2b81101b80 1 -- 172.21.15.49:0/1929042966 wait complete. 2023-07-18T03:40:09.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.051+0000 7f2b81101b80 20 client.5385 trim_cache size 0 max 16384 2023-07-18T03:40:09.055 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:40:09.055 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "e859a751-7613-40d7-973c-3e47a5e301cb", 2023-07-18T03:40:09.055 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-07-18T03:40:09.055 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 5172, 2023-07-18T03:40:09.056 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-07-18T03:40:09.056 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-07-18T03:40:09.056 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-07-18T03:40:09.056 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 11.725898869, 2023-07-18T03:40:09.056 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 18, 2023-07-18T03:40:09.056 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 36, 2023-07-18T03:40:09.056 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 36, 2023-07-18T03:40:09.057 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 27.785816104999999 2023-07-18T03:40:09.057 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:40:09.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.052+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2b5406ace0 msgr2=0x7f2b5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.052+0000 7f2b6d7ff700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2b5406ace0 0x7f2b5406d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2b6806cdc0 tx=0x7f2b5c008030 comp rx=0 tx=0).stop 2023-07-18T03:40:09.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.052+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2787499168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6806c9f0 msgr2=0x7f2b681efac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.052+0000 7f2b6d7ff700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6806c9f0 0x7f2b681efac0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2b58001eb0 tx=0x7f2b5800c7b0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.052+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2787499168 shutdown_connections 2023-07-18T03:40:09.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.052+0000 7f2b6d7ff700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b6815d9d0 0x7f2b680c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.052+0000 7f2b6d7ff700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2b5406ace0 0x7f2b5406d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.052+0000 7f2b6d7ff700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6806c9f0 0x7f2b681efac0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.052+0000 7f2b6d7ff700 1 --2- 172.21.15.49:0/2787499168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b680b1e50 0x7f2b681f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.052+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2787499168 >> 172.21.15.49:0/2787499168 conn(0x7f2b680b05d0 msgr2=0x7f2b680af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:09.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.052+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2787499168 shutdown_connections 2023-07-18T03:40:09.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.052+0000 7f2b6d7ff700 1 -- 172.21.15.49:0/2787499168 wait complete. 2023-07-18T03:40:09.065 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-18T03:40:09.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.276+0000 7f6854dff700 1 Processor -- start 2023-07-18T03:40:09.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.276+0000 7f6854dff700 1 -- start start 2023-07-18T03:40:09.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.277+0000 7f6854dff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f685015c950 0x7f68500b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.277+0000 7f6854dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f68500b1e00 0x7f68500d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.277+0000 7f6854dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68500d2ce0 con 0x7f68500b2350 2023-07-18T03:40:09.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.277+0000 7f6854dff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500b3350 con 0x7f685015c950 2023-07-18T03:40:09.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.277+0000 7f6854dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500b3630 con 0x7f68500b1e00 2023-07-18T03:40:09.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.277+0000 7f684e59c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f685015c950 0x7f68500b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.277+0000 7f684e59c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f685015c950 0x7f68500b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48974/0 (socket says 172.21.15.49:48974) 2023-07-18T03:40:09.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.277+0000 7f684e59c700 1 -- 172.21.15.49:0/4237062093 learned_addr learned my addr 172.21.15.49:0/4237062093 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:09.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.277+0000 7f6847fff700 1 --2- 172.21.15.49:0/4237062093 >> v2:172.21.15.107:3300/0 conn(0x7f68500b1e00 0x7f68500d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.278+0000 7f684e59c700 1 -- 172.21.15.49:0/4237062093 >> v2:172.21.15.107:3300/0 conn(0x7f68500b1e00 msgr2=0x7f68500d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:09.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.278+0000 7f684e59c700 1 --2- 172.21.15.49:0/4237062093 >> v2:172.21.15.107:3300/0 conn(0x7f68500b1e00 0x7f68500d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.278+0000 7f684e59c700 1 -- 172.21.15.49:0/4237062093 >> v1:172.21.15.107:6789/0 conn(0x7f68500b2350 legacy=0x7f68500afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:09.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.278+0000 7f684e59c700 1 -- 172.21.15.49:0/4237062093 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500b3910 con 0x7f685015c950 2023-07-18T03:40:09.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.278+0000 7f684e59c700 1 --2- 172.21.15.49:0/4237062093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f685015c950 0x7f68500b43f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6850072140 tx=0x7f6840009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ced74831f7ce9f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:09.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.278+0000 7f684dd9b700 1 -- 172.21.15.49:0/4237062093 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840005e30 con 0x7f685015c950 2023-07-18T03:40:09.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.278+0000 7f684dd9b700 1 -- 172.21.15.49:0/4237062093 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6840006030 con 0x7f685015c950 2023-07-18T03:40:09.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.278+0000 7f684dd9b700 1 -- 172.21.15.49:0/4237062093 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400063e0 con 0x7f685015c950 2023-07-18T03:40:09.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.279+0000 7f6854dff700 1 -- 172.21.15.49:0/4237062093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f685015c950 msgr2=0x7f68500b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.279+0000 7f6854dff700 1 --2- 172.21.15.49:0/4237062093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f685015c950 0x7f68500b43f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6850072140 tx=0x7f6840009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.279+0000 7f6854dff700 1 -- 172.21.15.49:0/4237062093 shutdown_connections 2023-07-18T03:40:09.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.279+0000 7f6854dff700 1 --2- 172.21.15.49:0/4237062093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f685015c950 0x7f68500b43f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.279+0000 7f6854dff700 1 --2- 172.21.15.49:0/4237062093 >> v2:172.21.15.107:3300/0 conn(0x7f68500b1e00 0x7f68500d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.279+0000 7f6854dff700 1 -- 172.21.15.49:0/4237062093 >> 172.21.15.49:0/4237062093 conn(0x7f685009c1d0 msgr2=0x7f68500d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:09.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.279+0000 7f6854dff700 1 -- 172.21.15.49:0/4237062093 shutdown_connections 2023-07-18T03:40:09.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.279+0000 7f6854dff700 1 -- 172.21.15.49:0/4237062093 wait complete. 2023-07-18T03:40:09.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.281+0000 7f6854dff700 1 Processor -- start 2023-07-18T03:40:09.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.281+0000 7f6854dff700 1 -- start start 2023-07-18T03:40:09.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f6854dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68500b1e00 0x7f68500c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f6854dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68500b2350 0x7f68500cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f6854dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f685015c950 0x7f68500cf790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f6854dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c7d10 con 0x7f685015c950 2023-07-18T03:40:09.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f6854dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500cc840 con 0x7f68500b2350 2023-07-18T03:40:09.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f6854dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68501f25e0 con 0x7f68500b1e00 2023-07-18T03:40:09.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f684ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f685015c950 0x7f68500cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f684ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f685015c950 0x7f68500cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48986/0 (socket says 172.21.15.49:48986) 2023-07-18T03:40:09.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f684ed9d700 1 -- 172.21.15.49:0/3596628978 learned_addr learned my addr 172.21.15.49:0/3596628978 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:09.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f684e59c700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68500b1e00 0x7f68500c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f6847fff700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68500b2350 0x7f68500cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f684ed9d700 1 -- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68500b1e00 msgr2=0x7f68500c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f684ed9d700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68500b1e00 0x7f68500c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f684ed9d700 1 -- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68500b2350 msgr2=0x7f68500cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f684ed9d700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68500b2350 0x7f68500cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f684ed9d700 1 -- 172.21.15.49:0/3596628978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6840009500 con 0x7f685015c950 2023-07-18T03:40:09.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.282+0000 7f684e59c700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68500b1e00 0x7f68500c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:09.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.283+0000 7f684ed9d700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f685015c950 0x7f68500cf790 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f684800cc10 tx=0x7f684800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:09.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.283+0000 7f6846ffd700 1 -- 172.21.15.49:0/3596628978 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684800abb0 con 0x7f685015c950 2023-07-18T03:40:09.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.283+0000 7f6846ffd700 1 -- 172.21.15.49:0/3596628978 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684800ad10 con 0x7f685015c950 2023-07-18T03:40:09.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.283+0000 7f6846ffd700 1 -- 172.21.15.49:0/3596628978 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848012660 con 0x7f685015c950 2023-07-18T03:40:09.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.283+0000 7f6854dff700 1 -- 172.21.15.49:0/3596628978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68500c7100 con 0x7f685015c950 2023-07-18T03:40:09.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.283+0000 7f6854dff700 1 -- 172.21.15.49:0/3596628978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68500c76c0 con 0x7f685015c950 2023-07-18T03:40:09.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.284+0000 7f6854dff700 1 -- 172.21.15.49:0/3596628978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68500c7ec0 con 0x7f685015c950 2023-07-18T03:40:09.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.285+0000 7f6846ffd700 1 -- 172.21.15.49:0/3596628978 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f68480127c0 con 0x7f685015c950 2023-07-18T03:40:09.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.286+0000 7f6846ffd700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f683006ace0 0x7f683006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:09.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.286+0000 7f6846ffd700 1 -- 172.21.15.49:0/3596628978 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6848089d00 con 0x7f685015c950 2023-07-18T03:40:09.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.287+0000 7f684e59c700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f683006ace0 0x7f683006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:09.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.287+0000 7f684e59c700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f683006ace0 0x7f683006d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6850072140 tx=0x7f6840008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:09.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.291+0000 7f6846ffd700 1 -- 172.21.15.49:0/3596628978 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f68500c7ec0 con 0x7f685015c950 2023-07-18T03:40:09.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.545+0000 7f6854dff700 1 -- 172.21.15.49:0/3596628978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f68500c7ec0 con 0x7f685015c950 2023-07-18T03:40:09.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.545+0000 7f6846ffd700 1 -- 172.21.15.49:0/3596628978 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f68500c7ec0 con 0x7f685015c950 2023-07-18T03:40:09.541 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-18T03:40:09.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.547+0000 7f6854dff700 1 -- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f683006ace0 msgr2=0x7f683006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.547+0000 7f6854dff700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f683006ace0 0x7f683006d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6850072140 tx=0x7f6840008070 comp rx=0 tx=0).stop 2023-07-18T03:40:09.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.547+0000 7f6854dff700 1 -- 172.21.15.49:0/3596628978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f685015c950 msgr2=0x7f68500cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.547+0000 7f6854dff700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f685015c950 0x7f68500cf790 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f684800cc10 tx=0x7f684800a060 comp rx=0 tx=0).stop 2023-07-18T03:40:09.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.547+0000 7f6854dff700 1 -- 172.21.15.49:0/3596628978 shutdown_connections 2023-07-18T03:40:09.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.547+0000 7f6854dff700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f683006ace0 0x7f683006d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.547+0000 7f6854dff700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f685015c950 0x7f68500cf790 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.547+0000 7f6854dff700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68500b1e00 0x7f68500c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.547+0000 7f6854dff700 1 --2- 172.21.15.49:0/3596628978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68500b2350 0x7f68500cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.547+0000 7f6854dff700 1 -- 172.21.15.49:0/3596628978 >> 172.21.15.49:0/3596628978 conn(0x7f685009c1d0 msgr2=0x7f68500b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:09.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.548+0000 7f6854dff700 1 -- 172.21.15.49:0/3596628978 shutdown_connections 2023-07-18T03:40:09.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:09.548+0000 7f6854dff700 1 -- 172.21.15.49:0/3596628978 wait complete. 2023-07-18T03:40:09.561 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-18T03:40:09.561 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-18T03:40:09.561 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-07-18T03:40:09.562 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-18T03:40:09.562 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-18T03:40:09.562 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-18T03:40:09.563 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-07-18T03:40:09.563 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-18T03:40:09.563 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-18T03:40:09.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:09.564 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-07-18T03:40:09.610 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:40:09.610 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:40:09.610 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:40:09.610 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:09.610 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-07-18T03:40:09.610 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:09.611 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:40:09.611 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:09.611 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-07-18T03:40:09.611 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:40:09.611 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:09.611 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-07-18T03:40:09.611 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-18T03:40:09.612 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:09.612 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-07-18T03:40:09.612 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-18T03:40:09.612 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 41898sec preferred_lft 41898sec 2023-07-18T03:40:09.612 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-07-18T03:40:09.612 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:09.612 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:40:09.613 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:09.613 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-07-18T03:40:09.613 INFO:teuthology.orchestra.run.smithi049.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-18T03:40:09.613 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 16:94:d3:1e:3e:91 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:09.613 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-18T03:40:09.613 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:09.613 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::463:84ff:fe79:c51d/64 scope link 2023-07-18T03:40:09.613 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:09.614 INFO:teuthology.orchestra.run.smithi049.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-18T03:40:09.614 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 16:94:d3:1e:3e:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-18T03:40:09.614 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::1494:d3ff:fe1e:3e91/64 scope link 2023-07-18T03:40:09.614 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:09.614 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-18T03:40:09.614 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether ca:3a:27:0d:fe:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-18T03:40:09.614 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c83a:27ff:fe0d:fe50/64 scope link 2023-07-18T03:40:09.614 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:09.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:09.615 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-18T03:40:09.718 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-07-18T03:40:09.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:09.722 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-07-18T03:40:09.770 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-07-18T03:40:09.771 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-18T03:40:09.771 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-18T03:40:09.771 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-18T03:40:09.771 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-18T03:40:09.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:09.773 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-07-18T03:40:09.773 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-07-18T03:40:09.773 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-18T03:40:09.774 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-18T03:40:09.774 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-18T03:40:09.774 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-07-18T03:40:09.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:09.954 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-07-18T03:40:10.000 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-18T03:40:10.001 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-18T03:40:10.001 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-18T03:40:10.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:10.002 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-07-18T03:40:10.113 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:40:10.114 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-18T03:40:10.158 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-18T03:40:10.158 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:40:10.159 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-18T03:40:10.242 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-07-18T03:40:10.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:10.243 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:40:10.390 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:40:10.390 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:40:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:40:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:40:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2023-07-18T03:40:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with new device syntax: 0@e859a751-7613-40d7-973c-3e47a5e301cb.cephfs=/ 2023-07-18T03:40:10.423 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.49:3300/172.21.15.107:3300/172.21.15.107:3301" will pass to kernel 2023-07-18T03:40:10.427 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-18T03:40:10.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:10.428 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-18T03:40:10.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:10.501 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:40:10.604 INFO:teuthology.orchestra.run.smithi049.stdout:client5233 2023-07-18T03:40:10.604 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:40:10.605 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:10.650 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-18T03:40:10.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.900+0000 7f79875ff700 1 Processor -- start 2023-07-18T03:40:10.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.900+0000 7f79875ff700 1 -- start start 2023-07-18T03:40:10.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.901+0000 7f79875ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f79800ce840 0x7f79800cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:10.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.901+0000 7f79875ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79800cf1d0 con 0x7f798015ca30 2023-07-18T03:40:10.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.901+0000 7f79875ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f798006c360 con 0x7f79800d0830 2023-07-18T03:40:10.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.901+0000 7f79875ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798006cf00 con 0x7f79800ce840 2023-07-18T03:40:10.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.902+0000 7f7984b9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f79800ce840 0x7f79800cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:10.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.902+0000 7f798539b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f798015ca30 0x7f798006c7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49166/0 (socket says 172.21.15.49:49166) 2023-07-18T03:40:10.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.902+0000 7f798539b700 1 -- 172.21.15.49:0/1415067509 learned_addr learned my addr 172.21.15.49:0/1415067509 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:10.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.902+0000 7f7984b9a700 1 -- 172.21.15.49:0/1415067509 >> v1:172.21.15.107:6789/0 conn(0x7f79800d0830 legacy=0x7f79800d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:10.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.902+0000 7f7984b9a700 1 -- 172.21.15.49:0/1415067509 >> v1:172.21.15.49:6789/0 conn(0x7f798015ca30 legacy=0x7f798006c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:10.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.902+0000 7f7984b9a700 1 -- 172.21.15.49:0/1415067509 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798006ba50 con 0x7f79800ce840 2023-07-18T03:40:10.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.902+0000 7f7977fff700 1 -- 172.21.15.49:0/1415067509 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050318865 0 0) 0x7f797c001110 con 0x7f798015ca30 2023-07-18T03:40:10.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.902+0000 7f7984b9a700 1 --2- 172.21.15.49:0/1415067509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800ce840 0x7f79800cec90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7970009840 tx=0x7f7970009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4d2c5a4c04a489 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:10.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.903+0000 7f7977fff700 1 -- 172.21.15.49:0/1415067509 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970011830 con 0x7f79800ce840 2023-07-18T03:40:10.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.903+0000 7f7977fff700 1 -- 172.21.15.49:0/1415067509 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970011990 con 0x7f79800ce840 2023-07-18T03:40:10.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.903+0000 7f7977fff700 1 -- 172.21.15.49:0/1415067509 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970011d20 con 0x7f79800ce840 2023-07-18T03:40:10.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.903+0000 7f79875ff700 1 -- 172.21.15.49:0/1415067509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800ce840 msgr2=0x7f79800cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:10.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.903+0000 7f79875ff700 1 --2- 172.21.15.49:0/1415067509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800ce840 0x7f79800cec90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7970009840 tx=0x7f7970009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:10.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.903+0000 7f79875ff700 1 -- 172.21.15.49:0/1415067509 shutdown_connections 2023-07-18T03:40:10.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.903+0000 7f79875ff700 1 --2- 172.21.15.49:0/1415067509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800ce840 0x7f79800cec90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.903+0000 7f79875ff700 1 -- 172.21.15.49:0/1415067509 >> 172.21.15.49:0/1415067509 conn(0x7f79800664f0 msgr2=0x7f7980099fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:10.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.904+0000 7f79875ff700 1 -- 172.21.15.49:0/1415067509 shutdown_connections 2023-07-18T03:40:10.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.904+0000 7f79875ff700 1 -- 172.21.15.49:0/1415067509 wait complete. 2023-07-18T03:40:10.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.905+0000 7f79875ff700 1 Processor -- start 2023-07-18T03:40:10.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.905+0000 7f79875ff700 1 -- start start 2023-07-18T03:40:10.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.906+0000 7f79875ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79800ce840 0x7f79801f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:10.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.906+0000 7f79875ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79800d0830 0x7f79800b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:10.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.906+0000 7f79875ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798015ca30 0x7f79800b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:10.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f79875ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79801f2cf0 con 0x7f798015ca30 2023-07-18T03:40:10.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f79875ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798015b000 con 0x7f79800d0830 2023-07-18T03:40:10.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f79875ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798015dd00 con 0x7f79800ce840 2023-07-18T03:40:10.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f7985b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798015ca30 0x7f79800b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:10.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f7985b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798015ca30 0x7f79800b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49002/0 (socket says 172.21.15.49:49002) 2023-07-18T03:40:10.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f7985b9c700 1 -- 172.21.15.49:0/1046240235 learned_addr learned my addr 172.21.15.49:0/1046240235 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:10.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f798539b700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79800ce840 0x7f79801f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:10.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f7984b9a700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79800d0830 0x7f79800b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:10.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f7985b9c700 1 -- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79800ce840 msgr2=0x7f79801f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:10.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f7985b9c700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79800ce840 0x7f79801f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f7985b9c700 1 -- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79800d0830 msgr2=0x7f79800b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:10.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f7985b9c700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79800d0830 0x7f79800b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.907+0000 7f7985b9c700 1 -- 172.21.15.49:0/1046240235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79700094a0 con 0x7f798015ca30 2023-07-18T03:40:10.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.908+0000 7f7985b9c700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798015ca30 0x7f79800b5ff0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f797800cc50 tx=0x7f797800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:10.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.908+0000 7f7975ffb700 1 -- 172.21.15.49:0/1046240235 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978003c00 con 0x7f798015ca30 2023-07-18T03:40:10.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.908+0000 7f7975ffb700 1 -- 172.21.15.49:0/1046240235 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7978003d60 con 0x7f798015ca30 2023-07-18T03:40:10.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.908+0000 7f79875ff700 1 -- 172.21.15.49:0/1046240235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798015dfe0 con 0x7f798015ca30 2023-07-18T03:40:10.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.908+0000 7f7975ffb700 1 -- 172.21.15.49:0/1046240235 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797800aab0 con 0x7f798015ca30 2023-07-18T03:40:10.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.908+0000 7f79875ff700 1 -- 172.21.15.49:0/1046240235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798015e5a0 con 0x7f798015ca30 2023-07-18T03:40:10.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.909+0000 7f79875ff700 1 -- 172.21.15.49:0/1046240235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79800cfc20 con 0x7f798015ca30 2023-07-18T03:40:10.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.910+0000 7f7975ffb700 1 -- 172.21.15.49:0/1046240235 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f797800ac10 con 0x7f798015ca30 2023-07-18T03:40:10.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.911+0000 7f7975ffb700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f796406ace0 0x7f796406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:10.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.911+0000 7f7975ffb700 1 -- 172.21.15.49:0/1046240235 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7978089410 con 0x7f798015ca30 2023-07-18T03:40:10.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.911+0000 7f798539b700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f796406ace0 0x7f796406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:10.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.912+0000 7f798539b700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f796406ace0 0x7f796406d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f797c007c40 tx=0x7f797c007b90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:10.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:10.915+0000 7f7975ffb700 1 -- 172.21.15.49:0/1046240235 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f79780544c0 con 0x7f798015ca30 2023-07-18T03:40:11.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.177+0000 7f79875ff700 1 -- 172.21.15.49:0/1046240235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f79800b3aa0 con 0x7f798015ca30 2023-07-18T03:40:11.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.177+0000 7f7975ffb700 1 -- 172.21.15.49:0/1046240235 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f79800b3aa0 con 0x7f798015ca30 2023-07-18T03:40:11.173 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-18T03:40:11.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.179+0000 7f79875ff700 1 -- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f796406ace0 msgr2=0x7f796406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:11.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.179+0000 7f79875ff700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f796406ace0 0x7f796406d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f797c007c40 tx=0x7f797c007b90 comp rx=0 tx=0).stop 2023-07-18T03:40:11.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.179+0000 7f79875ff700 1 -- 172.21.15.49:0/1046240235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798015ca30 msgr2=0x7f79800b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.180+0000 7f79875ff700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798015ca30 0x7f79800b5ff0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f797800cc50 tx=0x7f797800cf60 comp rx=0 tx=0).stop 2023-07-18T03:40:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.180+0000 7f79875ff700 1 -- 172.21.15.49:0/1046240235 shutdown_connections 2023-07-18T03:40:11.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.180+0000 7f79875ff700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f796406ace0 0x7f796406d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.180+0000 7f79875ff700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f798015ca30 0x7f79800b5ff0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.180+0000 7f79875ff700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79800ce840 0x7f79801f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.180+0000 7f79875ff700 1 --2- 172.21.15.49:0/1046240235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79800d0830 0x7f79800b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.180+0000 7f79875ff700 1 -- 172.21.15.49:0/1046240235 >> 172.21.15.49:0/1046240235 conn(0x7f79800664f0 msgr2=0x7f798006eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.180+0000 7f79875ff700 1 -- 172.21.15.49:0/1046240235 shutdown_connections 2023-07-18T03:40:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:11.180+0000 7f79875ff700 1 -- 172.21.15.49:0/1046240235 wait complete. 2023-07-18T03:40:11.193 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-18T03:40:11.194 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-18T03:40:11.194 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2023-07-18T03:40:11.195 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-18T03:40:11.195 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-18T03:40:11.195 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-18T03:40:11.195 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-07-18T03:40:11.196 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-18T03:40:11.196 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-18T03:40:11.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:11.197 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2023-07-18T03:40:11.246 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:40:11.247 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:40:11.247 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:40:11.247 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:11.247 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2023-07-18T03:40:11.247 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:11.247 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:40:11.248 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:11.248 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2023-07-18T03:40:11.248 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:40:11.248 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:11.248 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2023-07-18T03:40:11.248 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:40:11.249 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:11.249 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-18T03:40:11.249 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:11.249 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-18T03:40:11.249 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 41882sec preferred_lft 41882sec 2023-07-18T03:40:11.249 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::9059:9611:e3b2:9f28/64 scope link noprefixroute 2023-07-18T03:40:11.249 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:11.249 INFO:teuthology.orchestra.run.smithi107.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-18T03:40:11.250 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 3a:61:9e:d0:ce:0b brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:11.250 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-18T03:40:11.250 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:11.250 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::60b5:e0ff:fe5c:981b/64 scope link 2023-07-18T03:40:11.250 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:11.250 INFO:teuthology.orchestra.run.smithi107.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-18T03:40:11.250 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether d2:89:6b:8e:5e:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-18T03:40:11.251 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::d089:6bff:fe8e:5eaf/64 scope link 2023-07-18T03:40:11.251 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:11.251 INFO:teuthology.orchestra.run.smithi107.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-18T03:40:11.251 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 3a:61:9e:d0:ce:0b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-18T03:40:11.251 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::3861:9eff:fed0:ce0b/64 scope link 2023-07-18T03:40:11.251 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:11.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:11.253 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-18T03:40:11.323 INFO:teuthology.orchestra.run.smithi107.stdout:1 2023-07-18T03:40:11.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:11.327 DEBUG:teuthology.orchestra.run.smithi107:> route 2023-07-18T03:40:11.374 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2023-07-18T03:40:11.374 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-18T03:40:11.374 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-18T03:40:11.375 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-18T03:40:11.375 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-18T03:40:11.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:11.376 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-18T03:40:11.376 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-18T03:40:11.377 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-18T03:40:11.378 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-18T03:40:11.380 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-18T03:40:11.380 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-18T03:40:11.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:11.584 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-18T03:40:11.627 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-18T03:40:11.627 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-18T03:40:11.628 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-18T03:40:11.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:11.629 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2023-07-18T03:40:11.736 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:40:11.737 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-18T03:40:11.780 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-18T03:40:11.780 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:40:11.780 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-18T03:40:11.862 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-07-18T03:40:11.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:11.865 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:40:12.006 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:40:12.006 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:40:12.036 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:40:12.037 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:40:12.037 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2023-07-18T03:40:12.037 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with new device syntax: 1@e859a751-7613-40d7-973c-3e47a5e301cb.cephfs=/ 2023-07-18T03:40:12.037 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.49:3300/172.21.15.107:3300/172.21.15.107:3301" will pass to kernel 2023-07-18T03:40:12.041 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-18T03:40:12.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:12.042 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-18T03:40:12.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:12.114 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:40:12.199 INFO:teuthology.orchestra.run.smithi107.stdout:client4895 2023-07-18T03:40:12.200 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:40:12.200 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:12.243 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-07-18T03:40:12.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.470+0000 7f77d77ff700 1 Processor -- start 2023-07-18T03:40:12.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.471+0000 7f77d77ff700 1 -- start start 2023-07-18T03:40:12.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.472+0000 7f77d77ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00ca1e0 0x7f77d00ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:12.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.472+0000 7f77d77ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d015b530 con 0x7f77d0070520 2023-07-18T03:40:12.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.472+0000 7f77d77ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d00caaf0 con 0x7f77d00b0480 2023-07-18T03:40:12.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.472+0000 7f77d77ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d015aa10 con 0x7f77d00ca1e0 2023-07-18T03:40:12.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.472+0000 7f77d5d9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f77d0070520 0x7f77d00709a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49170/0 (socket says 172.21.15.49:49170) 2023-07-18T03:40:12.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.472+0000 7f77d5d9c700 1 -- 172.21.15.49:0/3230675062 learned_addr learned my addr 172.21.15.49:0/3230675062 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:12.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.472+0000 7f77d559b700 1 --2- 172.21.15.49:0/3230675062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00ca1e0 0x7f77d00ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:12.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.473+0000 7f77c7fff700 1 -- 172.21.15.49:0/3230675062 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229296124 0 0) 0x7f77d015b530 con 0x7f77d0070520 2023-07-18T03:40:12.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.473+0000 7f77c7fff700 1 -- 172.21.15.49:0/3230675062 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77bc0035b0 con 0x7f77d0070520 2023-07-18T03:40:12.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.473+0000 7f77d559b700 1 -- 172.21.15.49:0/3230675062 >> v1:172.21.15.107:6789/0 conn(0x7f77d00b0480 legacy=0x7f77d00b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:12.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.473+0000 7f77c7fff700 1 -- 172.21.15.49:0/3230675062 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665440480 0 0) 0x7f77d00caaf0 con 0x7f77d00b0480 2023-07-18T03:40:12.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.473+0000 7f77d559b700 1 -- 172.21.15.49:0/3230675062 >> v1:172.21.15.49:6789/0 conn(0x7f77d0070520 legacy=0x7f77d00709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:12.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.473+0000 7f77d559b700 1 -- 172.21.15.49:0/3230675062 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d015ac00 con 0x7f77d00ca1e0 2023-07-18T03:40:12.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.473+0000 7f77d559b700 1 --2- 172.21.15.49:0/3230675062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00ca1e0 0x7f77d00ca5b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009870 tx=0x7f77cc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf60ec8d6b139ae6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:12.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.473+0000 7f77c7fff700 1 -- 172.21.15.49:0/3230675062 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc011830 con 0x7f77d00ca1e0 2023-07-18T03:40:12.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.473+0000 7f77c7fff700 1 -- 172.21.15.49:0/3230675062 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc011990 con 0x7f77d00ca1e0 2023-07-18T03:40:12.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.473+0000 7f77c7fff700 1 -- 172.21.15.49:0/3230675062 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc011d20 con 0x7f77d00ca1e0 2023-07-18T03:40:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.474+0000 7f77d77ff700 1 -- 172.21.15.49:0/3230675062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00ca1e0 msgr2=0x7f77d00ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.474+0000 7f77d77ff700 1 --2- 172.21.15.49:0/3230675062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00ca1e0 0x7f77d00ca5b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009870 tx=0x7f77cc009b80 comp rx=0 tx=0).stop 2023-07-18T03:40:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.474+0000 7f77d77ff700 1 -- 172.21.15.49:0/3230675062 shutdown_connections 2023-07-18T03:40:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.474+0000 7f77d77ff700 1 --2- 172.21.15.49:0/3230675062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00ca1e0 0x7f77d00ca5b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.474+0000 7f77d77ff700 1 -- 172.21.15.49:0/3230675062 >> 172.21.15.49:0/3230675062 conn(0x7f77d00cea00 msgr2=0x7f77d00cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.474+0000 7f77d77ff700 1 -- 172.21.15.49:0/3230675062 shutdown_connections 2023-07-18T03:40:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.474+0000 7f77d77ff700 1 -- 172.21.15.49:0/3230675062 wait complete. 2023-07-18T03:40:12.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.476+0000 7f77d77ff700 1 Processor -- start 2023-07-18T03:40:12.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.476+0000 7f77d77ff700 1 -- start start 2023-07-18T03:40:12.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.477+0000 7f77d77ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d0070520 0x7f77d00c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:12.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.477+0000 7f77d77ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b0480 0x7f77d00c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:12.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.477+0000 7f77d77ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00ca1e0 0x7f77d0149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:12.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.477+0000 7f77d77ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00c3a80 con 0x7f77d00ca1e0 2023-07-18T03:40:12.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.477+0000 7f77d77ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00c24a0 con 0x7f77d0070520 2023-07-18T03:40:12.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.477+0000 7f77d77ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00c2780 con 0x7f77d00b0480 2023-07-18T03:40:12.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.477+0000 7f77d5d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00ca1e0 0x7f77d0149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:12.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.477+0000 7f77d5d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00ca1e0 0x7f77d0149fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49006/0 (socket says 172.21.15.49:49006) 2023-07-18T03:40:12.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.477+0000 7f77d5d9c700 1 -- 172.21.15.49:0/2282141785 learned_addr learned my addr 172.21.15.49:0/2282141785 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:12.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.477+0000 7f77d559b700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d0070520 0x7f77d00c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:12.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.477+0000 7f77d4d9a700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b0480 0x7f77d00c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:12.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.478+0000 7f77d5d9c700 1 -- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b0480 msgr2=0x7f77d00c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:12.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.478+0000 7f77d5d9c700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b0480 0x7f77d00c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:12.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.478+0000 7f77d5d9c700 1 -- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d0070520 msgr2=0x7f77d00c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:12.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.478+0000 7f77d5d9c700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d0070520 0x7f77d00c3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:12.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.478+0000 7f77d5d9c700 1 -- 172.21.15.49:0/2282141785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc0094a0 con 0x7f77d00ca1e0 2023-07-18T03:40:12.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.478+0000 7f77d5d9c700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00ca1e0 0x7f77d0149fe0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f77c800caa0 tx=0x7f77c800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:12.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.478+0000 7f77c5ffb700 1 -- 172.21.15.49:0/2282141785 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c800a9e0 con 0x7f77d00ca1e0 2023-07-18T03:40:12.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.478+0000 7f77c5ffb700 1 -- 172.21.15.49:0/2282141785 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c800ab40 con 0x7f77d00ca1e0 2023-07-18T03:40:12.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.478+0000 7f77c5ffb700 1 -- 172.21.15.49:0/2282141785 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8012400 con 0x7f77d00ca1e0 2023-07-18T03:40:12.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.478+0000 7f77d77ff700 1 -- 172.21.15.49:0/2282141785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d00c2a60 con 0x7f77d00ca1e0 2023-07-18T03:40:12.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.478+0000 7f77d77ff700 1 -- 172.21.15.49:0/2282141785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d00c3020 con 0x7f77d00ca1e0 2023-07-18T03:40:12.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.480+0000 7f77d77ff700 1 -- 172.21.15.49:0/2282141785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d00c3210 con 0x7f77d00ca1e0 2023-07-18T03:40:12.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.481+0000 7f77c5ffb700 1 -- 172.21.15.49:0/2282141785 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f77c801b3b0 con 0x7f77d00ca1e0 2023-07-18T03:40:12.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.481+0000 7f77c5ffb700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f77b406ace0 0x7f77b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:12.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.482+0000 7f77c5ffb700 1 -- 172.21.15.49:0/2282141785 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77c80890a0 con 0x7f77d00ca1e0 2023-07-18T03:40:12.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.482+0000 7f77d559b700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f77b406ace0 0x7f77b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:12.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.483+0000 7f77d559b700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f77b406ace0 0x7f77b406d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0097c0 tx=0x7f77cc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:12.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.486+0000 7f77c5ffb700 1 -- 172.21.15.49:0/2282141785 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f77c8054210 con 0x7f77d00ca1e0 2023-07-18T03:40:12.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.738+0000 7f77d77ff700 1 -- 172.21.15.49:0/2282141785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f77d0074350 con 0x7f77d00ca1e0 2023-07-18T03:40:12.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.935+0000 7f77c5ffb700 1 -- 172.21.15.49:0/2282141785 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 136+0+0 (secure 0 0 0) 0x7f77c8057830 con 0x7f77d00ca1e0 2023-07-18T03:40:12.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.938+0000 7f77d77ff700 1 -- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f77b406ace0 msgr2=0x7f77b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:12.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.938+0000 7f77d77ff700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f77b406ace0 0x7f77b406d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0097c0 tx=0x7f77cc01c040 comp rx=0 tx=0).stop 2023-07-18T03:40:12.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.938+0000 7f77d77ff700 1 -- 172.21.15.49:0/2282141785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00ca1e0 msgr2=0x7f77d0149fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:12.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.938+0000 7f77d77ff700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00ca1e0 0x7f77d0149fe0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f77c800caa0 tx=0x7f77c800cdb0 comp rx=0 tx=0).stop 2023-07-18T03:40:12.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.938+0000 7f77d77ff700 1 -- 172.21.15.49:0/2282141785 shutdown_connections 2023-07-18T03:40:12.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.938+0000 7f77d77ff700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f77b406ace0 0x7f77b406d190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:12.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.938+0000 7f77d77ff700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d00ca1e0 0x7f77d0149fe0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:12.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.938+0000 7f77d77ff700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d0070520 0x7f77d00c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:12.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.938+0000 7f77d77ff700 1 --2- 172.21.15.49:0/2282141785 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b0480 0x7f77d00c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:12.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.938+0000 7f77d77ff700 1 -- 172.21.15.49:0/2282141785 >> 172.21.15.49:0/2282141785 conn(0x7f77d00cea00 msgr2=0x7f77d006c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:12.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.938+0000 7f77d77ff700 1 -- 172.21.15.49:0/2282141785 shutdown_connections 2023-07-18T03:40:12.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:12.938+0000 7f77d77ff700 1 -- 172.21.15.49:0/2282141785 wait complete. 2023-07-18T03:40:12.951 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-07-18T03:40:13.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.161+0000 7eff4e7ff700 1 Processor -- start 2023-07-18T03:40:13.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.162+0000 7eff4e7ff700 1 -- start start 2023-07-18T03:40:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.163+0000 7eff4e7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4815d9d0 0x7eff4806b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.163+0000 7eff4e7ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4806c4e0 con 0x7eff4806c9f0 2023-07-18T03:40:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.163+0000 7eff4e7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff480b2020 con 0x7eff48071170 2023-07-18T03:40:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.163+0000 7eff4e7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4815dda0 con 0x7eff4815d9d0 2023-07-18T03:40:13.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.163+0000 7eff477fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7eff4806c9f0 0x7eff480b1910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49184/0 (socket says 172.21.15.49:49184) 2023-07-18T03:40:13.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.163+0000 7eff477fe700 1 -- 172.21.15.49:0/2777354790 learned_addr learned my addr 172.21.15.49:0/2777354790 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:13.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.163+0000 7eff47fff700 1 --2- 172.21.15.49:0/2777354790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4815d9d0 0x7eff4806b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:13.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.164+0000 7eff46ffd700 1 -- 172.21.15.49:0/2777354790 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959566175 0 0) 0x7eff4806c4e0 con 0x7eff4806c9f0 2023-07-18T03:40:13.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.164+0000 7eff46ffd700 1 -- 172.21.15.49:0/2777354790 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff300035b0 con 0x7eff4806c9f0 2023-07-18T03:40:13.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.164+0000 7eff46ffd700 1 -- 172.21.15.49:0/2777354790 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284813196 0 0) 0x7eff480b2020 con 0x7eff48071170 2023-07-18T03:40:13.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.164+0000 7eff46ffd700 1 -- 172.21.15.49:0/2777354790 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff4806c4e0 con 0x7eff48071170 2023-07-18T03:40:13.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.164+0000 7eff47fff700 1 -- 172.21.15.49:0/2777354790 >> v1:172.21.15.107:6789/0 conn(0x7eff48071170 legacy=0x7eff480715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.164+0000 7eff47fff700 1 -- 172.21.15.49:0/2777354790 >> v1:172.21.15.49:6789/0 conn(0x7eff4806c9f0 legacy=0x7eff480b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.164+0000 7eff47fff700 1 -- 172.21.15.49:0/2777354790 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4806bf20 con 0x7eff4815d9d0 2023-07-18T03:40:13.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.164+0000 7eff47fff700 1 --2- 172.21.15.49:0/2777354790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4815d9d0 0x7eff4806b5b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7eff38009870 tx=0x7eff38009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8593b90cb46737ac server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:13.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.164+0000 7eff46ffd700 1 -- 172.21.15.49:0/2777354790 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38011830 con 0x7eff4815d9d0 2023-07-18T03:40:13.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.164+0000 7eff46ffd700 1 -- 172.21.15.49:0/2777354790 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff38011990 con 0x7eff4815d9d0 2023-07-18T03:40:13.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.164+0000 7eff46ffd700 1 -- 172.21.15.49:0/2777354790 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38011d00 con 0x7eff4815d9d0 2023-07-18T03:40:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.165+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2777354790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4815d9d0 msgr2=0x7eff4806b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.165+0000 7eff4e7ff700 1 --2- 172.21.15.49:0/2777354790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4815d9d0 0x7eff4806b5b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7eff38009870 tx=0x7eff38009b80 comp rx=0 tx=0).stop 2023-07-18T03:40:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.165+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2777354790 shutdown_connections 2023-07-18T03:40:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.165+0000 7eff4e7ff700 1 --2- 172.21.15.49:0/2777354790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4815d9d0 0x7eff4806b5b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.165+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2777354790 >> 172.21.15.49:0/2777354790 conn(0x7eff480b05d0 msgr2=0x7eff480b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.165+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2777354790 shutdown_connections 2023-07-18T03:40:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.165+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2777354790 wait complete. 2023-07-18T03:40:13.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.167+0000 7eff4e7ff700 1 Processor -- start 2023-07-18T03:40:13.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.167+0000 7eff4e7ff700 1 -- start start 2023-07-18T03:40:13.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.168+0000 7eff4e7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4806c9f0 0x7eff48152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:13.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.168+0000 7eff4e7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff48071170 0x7eff480c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:13.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.168+0000 7eff4e7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4815d9d0 0x7eff480ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:13.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.168+0000 7eff4e7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff48156590 con 0x7eff48071170 2023-07-18T03:40:13.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.168+0000 7eff4e7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4815b3f0 con 0x7eff4815d9d0 2023-07-18T03:40:13.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.168+0000 7eff4e7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4815a7e0 con 0x7eff4806c9f0 2023-07-18T03:40:13.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.168+0000 7eff477fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff48071170 0x7eff480c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:13.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.168+0000 7eff477fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff48071170 0x7eff480c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49014/0 (socket says 172.21.15.49:49014) 2023-07-18T03:40:13.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.168+0000 7eff477fe700 1 -- 172.21.15.49:0/2541447428 learned_addr learned my addr 172.21.15.49:0/2541447428 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:13.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff47fff700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4806c9f0 0x7eff48152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:13.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff4cd9c700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4815d9d0 0x7eff480ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:13.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff477fe700 1 -- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4806c9f0 msgr2=0x7eff48152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff477fe700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4806c9f0 0x7eff48152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff477fe700 1 -- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4815d9d0 msgr2=0x7eff480ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff477fe700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4815d9d0 0x7eff480ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff477fe700 1 -- 172.21.15.49:0/2541447428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff380094a0 con 0x7eff48071170 2023-07-18T03:40:13.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff4cd9c700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4815d9d0 0x7eff480ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:13.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff477fe700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff48071170 0x7eff480c9fa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7eff3c005910 tx=0x7eff3c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:13.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff44ff9700 1 -- 172.21.15.49:0/2541447428 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c005d70 con 0x7eff48071170 2023-07-18T03:40:13.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff44ff9700 1 -- 172.21.15.49:0/2541447428 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c0070d0 con 0x7eff48071170 2023-07-18T03:40:13.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2541447428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4815aac0 con 0x7eff48071170 2023-07-18T03:40:13.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff44ff9700 1 -- 172.21.15.49:0/2541447428 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c007480 con 0x7eff48071170 2023-07-18T03:40:13.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.169+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2541447428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff4815b080 con 0x7eff48071170 2023-07-18T03:40:13.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.171+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2541447428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4815b210 con 0x7eff48071170 2023-07-18T03:40:13.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.172+0000 7eff44ff9700 1 -- 172.21.15.49:0/2541447428 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff3c0075e0 con 0x7eff48071170 2023-07-18T03:40:13.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.172+0000 7eff44ff9700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7eff3406ace0 0x7eff3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:13.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.173+0000 7eff44ff9700 1 -- 172.21.15.49:0/2541447428 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff3c088a00 con 0x7eff48071170 2023-07-18T03:40:13.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.173+0000 7eff47fff700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7eff3406ace0 0x7eff3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:13.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.174+0000 7eff47fff700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7eff3406ace0 0x7eff3406d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff380090b0 tx=0x7eff38008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:13.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.177+0000 7eff44ff9700 1 -- 172.21.15.49:0/2541447428 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7eff3c053b40 con 0x7eff48071170 2023-07-18T03:40:13.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.420+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2541447428 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7eff48070970 con 0x7eff3406ace0 2023-07-18T03:40:13.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.422+0000 7eff44ff9700 1 -- 172.21.15.49:0/2541447428 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7eff38011b00 con 0x7eff3406ace0 2023-07-18T03:40:13.417 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2023-07-18T03:40:13.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:13.417 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephfs" 2023-07-18T03:40:13.418 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:13.418 INFO:teuthology.orchestra.run.smithi049.stdout:] 2023-07-18T03:40:13.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.424+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7eff3406ace0 msgr2=0x7eff3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.424+0000 7eff4e7ff700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7eff3406ace0 0x7eff3406d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff380090b0 tx=0x7eff38008030 comp rx=0 tx=0).stop 2023-07-18T03:40:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.424+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2541447428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff48071170 msgr2=0x7eff480c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.424+0000 7eff4e7ff700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff48071170 0x7eff480c9fa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7eff3c005910 tx=0x7eff3c00ca90 comp rx=0 tx=0).stop 2023-07-18T03:40:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.424+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2541447428 shutdown_connections 2023-07-18T03:40:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.424+0000 7eff4e7ff700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4815d9d0 0x7eff480ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.424+0000 7eff4e7ff700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4806c9f0 0x7eff48152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.424+0000 7eff4e7ff700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7eff3406ace0 0x7eff3406d190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.424+0000 7eff4e7ff700 1 --2- 172.21.15.49:0/2541447428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff48071170 0x7eff480c9fa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.424+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2541447428 >> 172.21.15.49:0/2541447428 conn(0x7eff480b05d0 msgr2=0x7eff480b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.425+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2541447428 shutdown_connections 2023-07-18T03:40:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.425+0000 7eff4e7ff700 1 -- 172.21.15.49:0/2541447428 wait complete. 2023-07-18T03:40:13.438 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-18T03:40:13.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.651+0000 7f08981ff700 1 Processor -- start 2023-07-18T03:40:13.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.651+0000 7f08981ff700 1 -- start start 2023-07-18T03:40:13.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.652+0000 7f08981ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08900d0b10 0x7f08900d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.652+0000 7f08981ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f08900b0e20 0x7f08900b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.652+0000 7f08981ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08900d14a0 con 0x7f08900b8a60 2023-07-18T03:40:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.652+0000 7f08981ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08900b8e30 con 0x7f08900b0e20 2023-07-18T03:40:13.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.652+0000 7f08981ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08900afaa0 con 0x7f08900d0b10 2023-07-18T03:40:13.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.653+0000 7f0895f9b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f08900b8a60 0x7f08900b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49196/0 (socket says 172.21.15.49:49196) 2023-07-18T03:40:13.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.653+0000 7f0895f9b700 1 -- 172.21.15.49:0/3130788207 learned_addr learned my addr 172.21.15.49:0/3130788207 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:13.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.653+0000 7f089579a700 1 --2- 172.21.15.49:0/3130788207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08900d0b10 0x7f08900d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:13.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.653+0000 7f089679c700 1 --2- 172.21.15.49:0/3130788207 >> v2:172.21.15.107:3300/0 conn(0x7f08900b0e20 0x7f08900b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:13.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.653+0000 7f0894f99700 1 -- 172.21.15.49:0/3130788207 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025810185 0 0) 0x7f088c001110 con 0x7f08900b8a60 2023-07-18T03:40:13.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.653+0000 7f0894f99700 1 -- 172.21.15.49:0/3130788207 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f087c0035b0 con 0x7f08900b8a60 2023-07-18T03:40:13.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.653+0000 7f089579a700 1 -- 172.21.15.49:0/3130788207 >> v2:172.21.15.107:3300/0 conn(0x7f08900b0e20 msgr2=0x7f08900b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:13.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.653+0000 7f089579a700 1 --2- 172.21.15.49:0/3130788207 >> v2:172.21.15.107:3300/0 conn(0x7f08900b0e20 0x7f08900b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.653+0000 7f089579a700 1 -- 172.21.15.49:0/3130788207 >> v1:172.21.15.49:6789/0 conn(0x7f08900b8a60 legacy=0x7f08900b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.653+0000 7f089579a700 1 -- 172.21.15.49:0/3130788207 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08900afd80 con 0x7f08900d0b10 2023-07-18T03:40:13.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.654+0000 7f089579a700 1 --2- 172.21.15.49:0/3130788207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08900d0b10 0x7f08900d0f60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0880009840 tx=0x7f0880009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f72bc2a2d7640d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:13.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.654+0000 7f0894f99700 1 -- 172.21.15.49:0/3130788207 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880011830 con 0x7f08900d0b10 2023-07-18T03:40:13.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.654+0000 7f0894f99700 1 -- 172.21.15.49:0/3130788207 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0880011990 con 0x7f08900d0b10 2023-07-18T03:40:13.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.654+0000 7f0894f99700 1 -- 172.21.15.49:0/3130788207 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880011d00 con 0x7f08900d0b10 2023-07-18T03:40:13.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.655+0000 7f08981ff700 1 -- 172.21.15.49:0/3130788207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08900d0b10 msgr2=0x7f08900d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.655+0000 7f08981ff700 1 --2- 172.21.15.49:0/3130788207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08900d0b10 0x7f08900d0f60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0880009840 tx=0x7f0880009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.655+0000 7f08981ff700 1 -- 172.21.15.49:0/3130788207 shutdown_connections 2023-07-18T03:40:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.655+0000 7f08981ff700 1 --2- 172.21.15.49:0/3130788207 >> v2:172.21.15.107:3300/0 conn(0x7f08900b0e20 0x7f08900b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.655+0000 7f08981ff700 1 --2- 172.21.15.49:0/3130788207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08900d0b10 0x7f08900d0f60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.655+0000 7f08981ff700 1 -- 172.21.15.49:0/3130788207 >> 172.21.15.49:0/3130788207 conn(0x7f089009c1b0 msgr2=0x7f08900d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.655+0000 7f08981ff700 1 -- 172.21.15.49:0/3130788207 shutdown_connections 2023-07-18T03:40:13.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.655+0000 7f08981ff700 1 -- 172.21.15.49:0/3130788207 wait complete. 2023-07-18T03:40:13.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.657+0000 7f08981ff700 1 Processor -- start 2023-07-18T03:40:13.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.657+0000 7f08981ff700 1 -- start start 2023-07-18T03:40:13.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.658+0000 7f08981ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08900b0e20 0x7f08900cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:13.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.658+0000 7f08981ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08900b8a60 0x7f08900cf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.658+0000 7f08981ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08900d0b10 0x7f08900cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.658+0000 7f08981ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08900cfb60 con 0x7f08900b8a60 2023-07-18T03:40:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.658+0000 7f08981ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0890151f80 con 0x7f08900d0b10 2023-07-18T03:40:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.658+0000 7f08981ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08900cfe10 con 0x7f08900b0e20 2023-07-18T03:40:13.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.658+0000 7f089579a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08900b8a60 0x7f08900cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.658+0000 7f089579a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08900b8a60 0x7f08900cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49030/0 (socket says 172.21.15.49:49030) 2023-07-18T03:40:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.658+0000 7f089579a700 1 -- 172.21.15.49:0/3308339325 learned_addr learned my addr 172.21.15.49:0/3308339325 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.658+0000 7f0895f9b700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08900b0e20 0x7f08900cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.658+0000 7f089679c700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08900d0b10 0x7f08900cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:13.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.658+0000 7f089579a700 1 -- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08900b0e20 msgr2=0x7f08900cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.659+0000 7f089579a700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08900b0e20 0x7f08900cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.659+0000 7f089579a700 1 -- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08900d0b10 msgr2=0x7f08900cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.659+0000 7f089579a700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08900d0b10 0x7f08900cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.659+0000 7f089579a700 1 -- 172.21.15.49:0/3308339325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800094a0 con 0x7f08900b8a60 2023-07-18T03:40:13.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.659+0000 7f089679c700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08900d0b10 0x7f08900cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:13.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.659+0000 7f089579a700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08900b8a60 0x7f08900cf620 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0880009fd0 tx=0x7f0880004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:13.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.659+0000 7f08867fc700 1 -- 172.21.15.49:0/3308339325 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880004550 con 0x7f08900b8a60 2023-07-18T03:40:13.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.659+0000 7f08867fc700 1 -- 172.21.15.49:0/3308339325 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088001e070 con 0x7f08900b8a60 2023-07-18T03:40:13.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.659+0000 7f08867fc700 1 -- 172.21.15.49:0/3308339325 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08800038c0 con 0x7f08900b8a60 2023-07-18T03:40:13.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.659+0000 7f08981ff700 1 -- 172.21.15.49:0/3308339325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08900d00f0 con 0x7f08900b8a60 2023-07-18T03:40:13.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.659+0000 7f08981ff700 1 -- 172.21.15.49:0/3308339325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089015e4e0 con 0x7f08900b8a60 2023-07-18T03:40:13.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.660+0000 7f0873fff700 1 -- 172.21.15.49:0/3308339325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089015e6d0 con 0x7f08900b8a60 2023-07-18T03:40:13.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.661+0000 7f08867fc700 1 -- 172.21.15.49:0/3308339325 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0880003a20 con 0x7f08900b8a60 2023-07-18T03:40:13.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.662+0000 7f08867fc700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f087406ace0 0x7f087406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:13.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.662+0000 7f08867fc700 1 -- 172.21.15.49:0/3308339325 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f088008a4f0 con 0x7f08900b8a60 2023-07-18T03:40:13.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.663+0000 7f0895f9b700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f087406ace0 0x7f087406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:13.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.664+0000 7f0895f9b700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f087406ace0 0x7f087406d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f088c007fd0 tx=0x7f088c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:13.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.667+0000 7f08867fc700 1 -- 172.21.15.49:0/3308339325 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0880055500 con 0x7f08900b8a60 2023-07-18T03:40:13.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.876+0000 7f0873fff700 1 -- 172.21.15.49:0/3308339325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0890074350 con 0x7f08900b8a60 2023-07-18T03:40:13.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.876+0000 7f08867fc700 1 -- 172.21.15.49:0/3308339325 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0880058b20 con 0x7f08900b8a60 2023-07-18T03:40:13.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.878+0000 7f0873fff700 1 -- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f087406ace0 msgr2=0x7f087406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.878+0000 7f0873fff700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f087406ace0 0x7f087406d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f088c007fd0 tx=0x7f088c009040 comp rx=0 tx=0).stop 2023-07-18T03:40:13.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.878+0000 7f0873fff700 1 -- 172.21.15.49:0/3308339325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08900b8a60 msgr2=0x7f08900cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:13.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.878+0000 7f0873fff700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08900b8a60 0x7f08900cf620 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0880009fd0 tx=0x7f0880004750 comp rx=0 tx=0).stop 2023-07-18T03:40:13.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.878+0000 7f0873fff700 1 -- 172.21.15.49:0/3308339325 shutdown_connections 2023-07-18T03:40:13.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.878+0000 7f0873fff700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08900d0b10 0x7f08900cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.878+0000 7f0873fff700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08900b0e20 0x7f08900cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.878+0000 7f0873fff700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f087406ace0 0x7f087406d190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.878+0000 7f0873fff700 1 --2- 172.21.15.49:0/3308339325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08900b8a60 0x7f08900cf620 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:13.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.878+0000 7f0873fff700 1 -- 172.21.15.49:0/3308339325 >> 172.21.15.49:0/3308339325 conn(0x7f089009c1b0 msgr2=0x7f08900b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:13.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.878+0000 7f0873fff700 1 -- 172.21.15.49:0/3308339325 shutdown_connections 2023-07-18T03:40:13.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:13.878+0000 7f0873fff700 1 -- 172.21.15.49:0/3308339325 wait complete. 2023-07-18T03:40:13.892 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000041737 2023-07-18T03:40:14.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.106+0000 7fbb2b3ff700 1 Processor -- start 2023-07-18T03:40:14.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.106+0000 7fbb2b3ff700 1 -- start start 2023-07-18T03:40:14.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.108+0000 7fbb2b3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbb240d0b10 0x7fbb240d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.108+0000 7fbb2b3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb240b0e20 0x7fbb240b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.108+0000 7fbb2b3ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb240b8e30 con 0x7fbb240b8a60 2023-07-18T03:40:14.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.108+0000 7fbb2b3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb240d14d0 con 0x7fbb240d0b10 2023-07-18T03:40:14.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.108+0000 7fbb2b3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb240afaa0 con 0x7fbb240b0e20 2023-07-18T03:40:14.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.108+0000 7fbb2919b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fbb240b8a60 0x7fbb240b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49206/0 (socket says 172.21.15.49:49206) 2023-07-18T03:40:14.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.108+0000 7fbb2919b700 1 -- 172.21.15.49:0/1872410473 learned_addr learned my addr 172.21.15.49:0/1872410473 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:14.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.108+0000 7fbb2899a700 1 --2- 172.21.15.49:0/1872410473 >> v2:172.21.15.107:3300/0 conn(0x7fbb240d0b10 0x7fbb240d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.108+0000 7fbb2999c700 1 --2- 172.21.15.49:0/1872410473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb240b0e20 0x7fbb240b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.108+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201155467 0 0) 0x7fbb240b8e30 con 0x7fbb240b8a60 2023-07-18T03:40:14.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.108+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb100035b0 con 0x7fbb240b8a60 2023-07-18T03:40:14.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2915417505 0 0) 0x7fbb100035b0 con 0x7fbb240b8a60 2023-07-18T03:40:14.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb240b0e20 msgr2=0x7fbb240b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 --2- 172.21.15.49:0/1872410473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb240b0e20 0x7fbb240b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 >> v2:172.21.15.107:3300/0 conn(0x7fbb240d0b10 msgr2=0x7fbb240d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 --2- 172.21.15.49:0/1872410473 >> v2:172.21.15.107:3300/0 conn(0x7fbb240d0b10 0x7fbb240d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb240afd80 con 0x7fbb240b8a60 2023-07-18T03:40:14.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7fbb20001670 con 0x7fbb240b8a60 2023-07-18T03:40:14.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 >> v1:172.21.15.49:6789/0 conn(0x7fbb240b8a60 legacy=0x7fbb240b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 --2- 172.21.15.49:0/1872410473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb10003a00 0x7fbb10003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 --2- 172.21.15.49:0/1872410473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb240b0e20 0x7fbb10004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 --2- 172.21.15.49:0/1872410473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb10004d90 0x7fbb10005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb240d14d0 con 0x7fbb10004d90 2023-07-18T03:40:14.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb240afaa0 con 0x7fbb10003a00 2023-07-18T03:40:14.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1000ba40 con 0x7fbb240b0e20 2023-07-18T03:40:14.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb2899a700 1 --2- 172.21.15.49:0/1872410473 >> v2:172.21.15.107:3300/0 conn(0x7fbb240d0b10 0x7fbb240d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:14.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb2999c700 1 --2- 172.21.15.49:0/1872410473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb10004d90 0x7fbb10005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb2919b700 1 --2- 172.21.15.49:0/1872410473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb10003a00 0x7fbb10003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.109+0000 7fbb2899a700 1 --2- 172.21.15.49:0/1872410473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb240b0e20 0x7fbb10004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.110+0000 7fbb2999c700 1 -- 172.21.15.49:0/1872410473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb240b0e20 msgr2=0x7fbb10004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.110+0000 7fbb2999c700 1 --2- 172.21.15.49:0/1872410473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb240b0e20 0x7fbb10004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.110+0000 7fbb2999c700 1 -- 172.21.15.49:0/1872410473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb10003a00 msgr2=0x7fbb10003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.110+0000 7fbb2999c700 1 --2- 172.21.15.49:0/1872410473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb10003a00 0x7fbb10003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.110+0000 7fbb2999c700 1 -- 172.21.15.49:0/1872410473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100035b0 con 0x7fbb10004d90 2023-07-18T03:40:14.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.110+0000 7fbb2999c700 1 --2- 172.21.15.49:0/1872410473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb10004d90 0x7fbb10005200 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009fc0 tx=0x7fbb10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d394ee2d5ef18b2c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:14.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.110+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c00ebd0 con 0x7fbb10004d90 2023-07-18T03:40:14.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.110+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c00f040 con 0x7fbb10004d90 2023-07-18T03:40:14.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.110+0000 7fbb1bfff700 1 -- 172.21.15.49:0/1872410473 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c014480 con 0x7fbb10004d90 2023-07-18T03:40:14.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.111+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/1872410473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb10004d90 msgr2=0x7fbb10005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.111+0000 7fbb2b3ff700 1 --2- 172.21.15.49:0/1872410473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb10004d90 0x7fbb10005200 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009fc0 tx=0x7fbb10005880 comp rx=0 tx=0).stop 2023-07-18T03:40:14.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.111+0000 7fbb2999c700 1 -- 172.21.15.49:0/1872410473 reap_dead start 2023-07-18T03:40:14.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.111+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/1872410473 shutdown_connections 2023-07-18T03:40:14.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.111+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/1872410473 >> 172.21.15.49:0/1872410473 conn(0x7fbb2409c1b0 msgr2=0x7fbb240d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:14.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.111+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/1872410473 shutdown_connections 2023-07-18T03:40:14.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.111+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/1872410473 wait complete. 2023-07-18T03:40:14.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.113+0000 7fbb2b3ff700 1 Processor -- start 2023-07-18T03:40:14.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.113+0000 7fbb2b3ff700 1 -- start start 2023-07-18T03:40:14.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2b3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb10004d90 0x7fbb24151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2b3ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb240cc7e0 0x7fbb2415f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2b3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb240ccc50 0x7fbb2415f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2b3ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb24152170 con 0x7fbb240cc7e0 2023-07-18T03:40:14.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2b3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb240cab50 con 0x7fbb240ccc50 2023-07-18T03:40:14.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2b3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb240cae30 con 0x7fbb10004d90 2023-07-18T03:40:14.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2899a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb240cc7e0 0x7fbb2415f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2899a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb240cc7e0 0x7fbb2415f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49050/0 (socket says 172.21.15.49:49050) 2023-07-18T03:40:14.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2899a700 1 -- 172.21.15.49:0/4100942955 learned_addr learned my addr 172.21.15.49:0/4100942955 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:14.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2999c700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb240ccc50 0x7fbb2415f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2919b700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb10004d90 0x7fbb24151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2899a700 1 -- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb10004d90 msgr2=0x7fbb24151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2899a700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb10004d90 0x7fbb24151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2899a700 1 -- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb240ccc50 msgr2=0x7fbb2415f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2899a700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb240ccc50 0x7fbb2415f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.114+0000 7fbb2899a700 1 -- 172.21.15.49:0/4100942955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb240cb110 con 0x7fbb240cc7e0 2023-07-18T03:40:14.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.115+0000 7fbb2919b700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb10004d90 0x7fbb24151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:14.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.115+0000 7fbb2899a700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb240cc7e0 0x7fbb2415f100 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbb14011900 tx=0x7fbb14011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:14.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.115+0000 7fbb19ffb700 1 -- 172.21.15.49:0/4100942955 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400f8d0 con 0x7fbb240cc7e0 2023-07-18T03:40:14.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.115+0000 7fbb19ffb700 1 -- 172.21.15.49:0/4100942955 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1400fa30 con 0x7fbb240cc7e0 2023-07-18T03:40:14.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.115+0000 7fbb19ffb700 1 -- 172.21.15.49:0/4100942955 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400fda0 con 0x7fbb240cc7e0 2023-07-18T03:40:14.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.115+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/4100942955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb240cb3f0 con 0x7fbb240cc7e0 2023-07-18T03:40:14.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.115+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/4100942955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb240cbbf0 con 0x7fbb240cc7e0 2023-07-18T03:40:14.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.116+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/4100942955 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb240cb910 con 0x7fbb240cc7e0 2023-07-18T03:40:14.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.117+0000 7fbb19ffb700 1 -- 172.21.15.49:0/4100942955 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbb14015460 con 0x7fbb240cc7e0 2023-07-18T03:40:14.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.118+0000 7fbb19ffb700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb0806ace0 0x7fbb0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.118+0000 7fbb19ffb700 1 -- 172.21.15.49:0/4100942955 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbb14089a30 con 0x7fbb240cc7e0 2023-07-18T03:40:14.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.119+0000 7fbb2919b700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb0806ace0 0x7fbb0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.119+0000 7fbb2919b700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb0806ace0 0x7fbb0806d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbb20000c00 tx=0x7fbb20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:14.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.123+0000 7fbb19ffb700 1 -- 172.21.15.49:0/4100942955 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbb240cb910 con 0x7fbb240cc7e0 2023-07-18T03:40:14.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.362+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/4100942955 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000000041737", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb240b0340 con 0x7fbb0806ace0 2023-07-18T03:40:14.365 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:40:14.370+0000 7f45ce8fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:40:14.365 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:40:14.370+0000 7f45ce8fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:40:14.365 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:40:14.370+0000 7f45ce8fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:40:14.365 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:40:14.370+0000 7f45ce8fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:40:14.366 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:40:14.370+0000 7f45ce8fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:40:14.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.390+0000 7fbb19ffb700 1 -- 172.21.15.49:0/4100942955 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbb20004960 con 0x7fbb0806ace0 2023-07-18T03:40:14.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.392+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb0806ace0 msgr2=0x7fbb0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.392+0000 7fbb2b3ff700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb0806ace0 0x7fbb0806d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbb20000c00 tx=0x7fbb20009040 comp rx=0 tx=0).stop 2023-07-18T03:40:14.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.392+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/4100942955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb240cc7e0 msgr2=0x7fbb2415f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.392+0000 7fbb2b3ff700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb240cc7e0 0x7fbb2415f100 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbb14011900 tx=0x7fbb14011c10 comp rx=0 tx=0).stop 2023-07-18T03:40:14.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.392+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/4100942955 shutdown_connections 2023-07-18T03:40:14.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.392+0000 7fbb2b3ff700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb240ccc50 0x7fbb2415f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.392+0000 7fbb2b3ff700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb10004d90 0x7fbb24151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.392+0000 7fbb2b3ff700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbb0806ace0 0x7fbb0806d190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.392+0000 7fbb2b3ff700 1 --2- 172.21.15.49:0/4100942955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb240cc7e0 0x7fbb2415f100 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.392+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/4100942955 >> 172.21.15.49:0/4100942955 conn(0x7fbb2409c1b0 msgr2=0x7fbb240d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:14.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.393+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/4100942955 shutdown_connections 2023-07-18T03:40:14.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.393+0000 7fbb2b3ff700 1 -- 172.21.15.49:0/4100942955 wait complete. 2023-07-18T03:40:14.407 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create cephfs group_0000000000041737 snapshot_0000000000452388 2023-07-18T03:40:14.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.622+0000 7ff7a6bff700 1 Processor -- start 2023-07-18T03:40:14.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.622+0000 7ff7a6bff700 1 -- start start 2023-07-18T03:40:14.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.624+0000 7ff7a6bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff7a00ce190 0x7ff7a006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.624+0000 7ff7a6bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff7a00cc540 0x7ff7a00cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.624+0000 7ff7a6bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a0071220 con 0x7ff7a0158af0 2023-07-18T03:40:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.624+0000 7ff7a6bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a00cce50 con 0x7ff7a00cc540 2023-07-18T03:40:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.624+0000 7ff7a6bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a00ce7c0 con 0x7ff7a00ce190 2023-07-18T03:40:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.624+0000 7ff7a519c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff7a00cc540 0x7ff7a00cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.624+0000 7ff7a519c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff7a00cc540 0x7ff7a00cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49062/0 (socket says 172.21.15.49:49062) 2023-07-18T03:40:14.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.624+0000 7ff7a519c700 1 -- 172.21.15.49:0/3405279301 learned_addr learned my addr 172.21.15.49:0/3405279301 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:14.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.624+0000 7ff7a519c700 1 -- 172.21.15.49:0/3405279301 >> v2:172.21.15.107:3300/0 conn(0x7ff7a00ce190 msgr2=0x7ff7a006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.624+0000 7ff7a519c700 1 --2- 172.21.15.49:0/3405279301 >> v2:172.21.15.107:3300/0 conn(0x7ff7a00ce190 0x7ff7a006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.624+0000 7ff7a519c700 1 -- 172.21.15.49:0/3405279301 >> v1:172.21.15.107:6789/0 conn(0x7ff7a0158af0 legacy=0x7ff7a00b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.625+0000 7ff7a519c700 1 -- 172.21.15.49:0/3405279301 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a00caa50 con 0x7ff7a00cc540 2023-07-18T03:40:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.625+0000 7ff7a519c700 1 --2- 172.21.15.49:0/3405279301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7a00cc540 0x7ff7a00cc910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00713b0 tx=0x7ff79800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74207fbc32f0c03 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.625+0000 7ff79f7fe700 1 -- 172.21.15.49:0/3405279301 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798010070 con 0x7ff7a00cc540 2023-07-18T03:40:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.625+0000 7ff79f7fe700 1 -- 172.21.15.49:0/3405279301 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79800dd20 con 0x7ff7a00cc540 2023-07-18T03:40:14.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.625+0000 7ff79f7fe700 1 -- 172.21.15.49:0/3405279301 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79800b310 con 0x7ff7a00cc540 2023-07-18T03:40:14.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.626+0000 7ff7a6bff700 1 -- 172.21.15.49:0/3405279301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7a00cc540 msgr2=0x7ff7a00cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.626+0000 7ff7a6bff700 1 --2- 172.21.15.49:0/3405279301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7a00cc540 0x7ff7a00cc910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00713b0 tx=0x7ff79800d430 comp rx=0 tx=0).stop 2023-07-18T03:40:14.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.626+0000 7ff7a6bff700 1 -- 172.21.15.49:0/3405279301 shutdown_connections 2023-07-18T03:40:14.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.626+0000 7ff7a6bff700 1 --2- 172.21.15.49:0/3405279301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7a00cc540 0x7ff7a00cc910 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.626+0000 7ff7a6bff700 1 --2- 172.21.15.49:0/3405279301 >> v2:172.21.15.107:3300/0 conn(0x7ff7a00ce190 0x7ff7a006b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.626+0000 7ff7a6bff700 1 -- 172.21.15.49:0/3405279301 >> 172.21.15.49:0/3405279301 conn(0x7ff7a00cea00 msgr2=0x7ff7a00cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:14.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.626+0000 7ff7a6bff700 1 -- 172.21.15.49:0/3405279301 shutdown_connections 2023-07-18T03:40:14.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.626+0000 7ff7a6bff700 1 -- 172.21.15.49:0/3405279301 wait complete. 2023-07-18T03:40:14.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.628+0000 7ff7a6bff700 1 Processor -- start 2023-07-18T03:40:14.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.628+0000 7ff7a6bff700 1 -- start start 2023-07-18T03:40:14.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff7a6bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a00cc540 0x7ff7a014df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff7a6bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7a00ce190 0x7ff7a00b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff7a6bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7a0158af0 0x7ff7a01ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff7a6bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00c6b20 con 0x7ff7a00ce190 2023-07-18T03:40:14.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff7a6bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00c6e00 con 0x7ff7a00cc540 2023-07-18T03:40:14.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff7a6bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00c70e0 con 0x7ff7a0158af0 2023-07-18T03:40:14.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff79ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7a00ce190 0x7ff7a00b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff79ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7a00ce190 0x7ff7a00b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49076/0 (socket says 172.21.15.49:49076) 2023-07-18T03:40:14.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff79ffff700 1 -- 172.21.15.49:0/181513895 learned_addr learned my addr 172.21.15.49:0/181513895 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:14.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff79ffff700 1 -- 172.21.15.49:0/181513895 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7a0158af0 msgr2=0x7ff7a01ef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff7a499b700 1 --2- 172.21.15.49:0/181513895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a00cc540 0x7ff7a014df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff7a519c700 1 --2- 172.21.15.49:0/181513895 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7a0158af0 0x7ff7a01ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff79ffff700 1 --2- 172.21.15.49:0/181513895 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7a0158af0 0x7ff7a01ef380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.629+0000 7ff79ffff700 1 -- 172.21.15.49:0/181513895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a00cc540 msgr2=0x7ff7a014df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.630+0000 7ff79ffff700 1 --2- 172.21.15.49:0/181513895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a00cc540 0x7ff7a014df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.630+0000 7ff79ffff700 1 -- 172.21.15.49:0/181513895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79800bd40 con 0x7ff7a00ce190 2023-07-18T03:40:14.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.630+0000 7ff7a499b700 1 --2- 172.21.15.49:0/181513895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a00cc540 0x7ff7a014df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:14.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.630+0000 7ff7a519c700 1 --2- 172.21.15.49:0/181513895 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7a0158af0 0x7ff7a01ef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:14.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.630+0000 7ff79ffff700 1 --2- 172.21.15.49:0/181513895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7a00ce190 0x7ff7a00b4110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff78c00b530 tx=0x7ff78c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:14.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.630+0000 7ff79d7fa700 1 -- 172.21.15.49:0/181513895 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c011830 con 0x7ff7a00ce190 2023-07-18T03:40:14.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.630+0000 7ff79d7fa700 1 -- 172.21.15.49:0/181513895 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c0119b0 con 0x7ff7a00ce190 2023-07-18T03:40:14.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.630+0000 7ff7a6bff700 1 -- 172.21.15.49:0/181513895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a00c73c0 con 0x7ff7a00ce190 2023-07-18T03:40:14.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.630+0000 7ff79d7fa700 1 -- 172.21.15.49:0/181513895 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c011d60 con 0x7ff7a00ce190 2023-07-18T03:40:14.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.630+0000 7ff7a6bff700 1 -- 172.21.15.49:0/181513895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a014e440 con 0x7ff7a00ce190 2023-07-18T03:40:14.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.631+0000 7ff7a6bff700 1 -- 172.21.15.49:0/181513895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a00c78e0 con 0x7ff7a00ce190 2023-07-18T03:40:14.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.632+0000 7ff79d7fa700 1 -- 172.21.15.49:0/181513895 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff78c01a400 con 0x7ff7a00ce190 2023-07-18T03:40:14.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.633+0000 7ff79d7fa700 1 --2- 172.21.15.49:0/181513895 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff78406ace0 0x7ff78406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.634+0000 7ff79d7fa700 1 -- 172.21.15.49:0/181513895 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff78c0895e0 con 0x7ff7a00ce190 2023-07-18T03:40:14.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.634+0000 7ff7a499b700 1 --2- 172.21.15.49:0/181513895 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff78406ace0 0x7ff78406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.635+0000 7ff7a499b700 1 --2- 172.21.15.49:0/181513895 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff78406ace0 0x7ff78406d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff794006fd0 tx=0x7ff794008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.638+0000 7ff79d7fa700 1 -- 172.21.15.49:0/181513895 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff7a00c78e0 con 0x7ff7a00ce190 2023-07-18T03:40:14.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.878+0000 7ff7a6bff700 1 -- 172.21.15.49:0/181513895 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "cephfs", "group_name": "group_0000000000041737", "snap_name": "snapshot_0000000000452388", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7a00cb010 con 0x7ff78406ace0 2023-07-18T03:40:14.875 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:40:14.880+0000 7f45ce8fe700 -1 mgr.server reply reply (38) Function not implemented subvolume group snapshots are not supported 2023-07-18T03:40:14.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:14.880+0000 7ff79d7fa700 1 -- 172.21.15.49:0/181513895 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: -38 subvolume group snapshots are not supported) v1 ==== 51+0+0 (secure 0 0 0) 0x7ff79400ad30 con 0x7ff78406ace0 2023-07-18T03:40:14.877 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENOSYS: subvolume group snapshots are not supported 2023-07-18T03:40:14.885 DEBUG:teuthology.orchestra.run:got remote process result: 38 2023-07-18T03:40:14.886 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000041737 2023-07-18T03:40:15.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.101+0000 7f4a4d3ff700 1 Processor -- start 2023-07-18T03:40:15.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.102+0000 7f4a4d3ff700 1 -- start start 2023-07-18T03:40:15.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.103+0000 7f4a4d3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4a480d0b10 0x7f4a480d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.103+0000 7f4a4d3ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4a480b0e20 0x7f4a480b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.103+0000 7f4a4d3ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a480b8e30 con 0x7f4a480b8a60 2023-07-18T03:40:15.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.103+0000 7f4a4d3ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a480d14d0 con 0x7f4a480b0e20 2023-07-18T03:40:15.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.103+0000 7f4a4d3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a480afaa0 con 0x7f4a480d0b10 2023-07-18T03:40:15.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.103+0000 7f4a477fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4a480b0e20 0x7f4a480b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.103+0000 7f4a477fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4a480b0e20 0x7f4a480b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49092/0 (socket says 172.21.15.49:49092) 2023-07-18T03:40:15.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.103+0000 7f4a477fe700 1 -- 172.21.15.49:0/2311675018 learned_addr learned my addr 172.21.15.49:0/2311675018 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:15.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.103+0000 7f4a46ffd700 1 --1- 172.21.15.49:0/2311675018 >> v1:172.21.15.49:6789/0 conn(0x7f4a480b8a60 0x7f4a480b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49210/0 (socket says 172.21.15.49:49210) 2023-07-18T03:40:15.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.103+0000 7f4a467fc700 1 --2- 172.21.15.49:0/2311675018 >> v2:172.21.15.107:3300/0 conn(0x7f4a480d0b10 0x7f4a480d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.104+0000 7f4a477fe700 1 -- 172.21.15.49:0/2311675018 >> v2:172.21.15.107:3300/0 conn(0x7f4a480d0b10 msgr2=0x7f4a480d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.104+0000 7f4a477fe700 1 --2- 172.21.15.49:0/2311675018 >> v2:172.21.15.107:3300/0 conn(0x7f4a480d0b10 0x7f4a480d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.104+0000 7f4a477fe700 1 -- 172.21.15.49:0/2311675018 >> v1:172.21.15.49:6789/0 conn(0x7f4a480b8a60 legacy=0x7f4a480b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.104+0000 7f4a477fe700 1 -- 172.21.15.49:0/2311675018 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480afd80 con 0x7f4a480b0e20 2023-07-18T03:40:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.104+0000 7f4a45ffb700 1 -- 172.21.15.49:0/2311675018 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1867954999 0 0) 0x7f4a480b8e30 con 0x7f4a480b8a60 2023-07-18T03:40:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.104+0000 7f4a477fe700 1 --2- 172.21.15.49:0/2311675018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a480b0e20 0x7f4a480b1260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4a4809c8e0 tx=0x7f4a4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a9437c8a7993c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.104+0000 7f4a45ffb700 1 -- 172.21.15.49:0/2311675018 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000e070 con 0x7f4a480b0e20 2023-07-18T03:40:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.104+0000 7f4a45ffb700 1 -- 172.21.15.49:0/2311675018 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4000b910 con 0x7f4a480b0e20 2023-07-18T03:40:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.104+0000 7f4a45ffb700 1 -- 172.21.15.49:0/2311675018 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000bcc0 con 0x7f4a480b0e20 2023-07-18T03:40:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.105+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2311675018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a480b0e20 msgr2=0x7f4a480b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.105+0000 7f4a4d3ff700 1 --2- 172.21.15.49:0/2311675018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a480b0e20 0x7f4a480b1260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4a4809c8e0 tx=0x7f4a4000b330 comp rx=0 tx=0).stop 2023-07-18T03:40:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.105+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2311675018 shutdown_connections 2023-07-18T03:40:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.105+0000 7f4a4d3ff700 1 --2- 172.21.15.49:0/2311675018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a480b0e20 0x7f4a480b1260 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.105+0000 7f4a4d3ff700 1 --2- 172.21.15.49:0/2311675018 >> v2:172.21.15.107:3300/0 conn(0x7f4a480d0b10 0x7f4a480d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.105+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2311675018 >> 172.21.15.49:0/2311675018 conn(0x7f4a4809c1b0 msgr2=0x7f4a480d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.105+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2311675018 shutdown_connections 2023-07-18T03:40:15.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.105+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2311675018 wait complete. 2023-07-18T03:40:15.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.107+0000 7f4a4d3ff700 1 Processor -- start 2023-07-18T03:40:15.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.108+0000 7f4a4d3ff700 1 -- start start 2023-07-18T03:40:15.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.109+0000 7f4a4d3ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a480b0e20 0x7f4a480cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.109+0000 7f4a4d3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a480b8a60 0x7f4a4815f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.109+0000 7f4a4d3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a480d0b10 0x7f4a480ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.109+0000 7f4a4d3ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a481f1c60 con 0x7f4a480b0e20 2023-07-18T03:40:15.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.109+0000 7f4a4d3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a481f2c30 con 0x7f4a480d0b10 2023-07-18T03:40:15.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.109+0000 7f4a4d3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a481f1e70 con 0x7f4a480b8a60 2023-07-18T03:40:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.109+0000 7f4a46ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a480b0e20 0x7f4a480cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.109+0000 7f4a46ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a480b0e20 0x7f4a480cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49104/0 (socket says 172.21.15.49:49104) 2023-07-18T03:40:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.109+0000 7f4a46ffd700 1 -- 172.21.15.49:0/2999437018 learned_addr learned my addr 172.21.15.49:0/2999437018 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.109+0000 7f4a467fc700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a480b8a60 0x7f4a4815f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.110+0000 7f4a477fe700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a480d0b10 0x7f4a480ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.110+0000 7f4a46ffd700 1 -- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a480b8a60 msgr2=0x7f4a4815f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.110+0000 7f4a46ffd700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a480b8a60 0x7f4a4815f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.110+0000 7f4a46ffd700 1 -- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a480d0b10 msgr2=0x7f4a480ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.110+0000 7f4a46ffd700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a480d0b10 0x7f4a480ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.110+0000 7f4a46ffd700 1 -- 172.21.15.49:0/2999437018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40009b30 con 0x7f4a480b0e20 2023-07-18T03:40:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.110+0000 7f4a46ffd700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a480b0e20 0x7f4a480cc2e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c750 tx=0x7f4a3800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.110+0000 7f4a2f7fe700 1 -- 172.21.15.49:0/2999437018 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38004c80 con 0x7f4a480b0e20 2023-07-18T03:40:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.110+0000 7f4a2f7fe700 1 -- 172.21.15.49:0/2999437018 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a38004e00 con 0x7f4a480b0e20 2023-07-18T03:40:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.110+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2999437018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a481f2150 con 0x7f4a480b0e20 2023-07-18T03:40:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.110+0000 7f4a2f7fe700 1 -- 172.21.15.49:0/2999437018 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38007310 con 0x7f4a480b0e20 2023-07-18T03:40:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.110+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2999437018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a481f2710 con 0x7f4a480b0e20 2023-07-18T03:40:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.111+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2999437018 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4806b7e0 con 0x7f4a480b0e20 2023-07-18T03:40:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.112+0000 7f4a2f7fe700 1 -- 172.21.15.49:0/2999437018 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a38007470 con 0x7f4a480b0e20 2023-07-18T03:40:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.113+0000 7f4a2f7fe700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4a3406ace0 0x7f4a3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.113+0000 7f4a2f7fe700 1 -- 172.21.15.49:0/2999437018 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4a380891a0 con 0x7f4a480b0e20 2023-07-18T03:40:15.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.114+0000 7f4a467fc700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4a3406ace0 0x7f4a3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.115+0000 7f4a467fc700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4a3406ace0 0x7f4a3406d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a4815fe90 tx=0x7f4a3c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:15.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.118+0000 7f4a2f7fe700 1 -- 172.21.15.49:0/2999437018 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4a38054240 con 0x7f4a480b0e20 2023-07-18T03:40:15.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.358+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2999437018 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "group_0000000000041737", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a480b0340 con 0x7f4a3406ace0 2023-07-18T03:40:15.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.364+0000 7f4a2f7fe700 1 -- 172.21.15.49:0/2999437018 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4a3c011880 con 0x7f4a3406ace0 2023-07-18T03:40:15.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.366+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4a3406ace0 msgr2=0x7f4a3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:15.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.366+0000 7f4a4d3ff700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4a3406ace0 0x7f4a3406d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a4815fe90 tx=0x7f4a3c009210 comp rx=0 tx=0).stop 2023-07-18T03:40:15.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.366+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2999437018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a480b0e20 msgr2=0x7f4a480cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:15.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.366+0000 7f4a4d3ff700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a480b0e20 0x7f4a480cc2e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c750 tx=0x7f4a3800ca60 comp rx=0 tx=0).stop 2023-07-18T03:40:15.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.366+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2999437018 shutdown_connections 2023-07-18T03:40:15.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.366+0000 7f4a4d3ff700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a480d0b10 0x7f4a480ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.366+0000 7f4a4d3ff700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4a3406ace0 0x7f4a3406d190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.366+0000 7f4a4d3ff700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a480b0e20 0x7f4a480cc2e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.366+0000 7f4a4d3ff700 1 --2- 172.21.15.49:0/2999437018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a480b8a60 0x7f4a4815f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.366+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2999437018 >> 172.21.15.49:0/2999437018 conn(0x7f4a4809c1b0 msgr2=0x7f4a4806b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:15.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.366+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2999437018 shutdown_connections 2023-07-18T03:40:15.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.366+0000 7f4a4d3ff700 1 -- 172.21.15.49:0/2999437018 wait complete. 2023-07-18T03:40:15.380 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-18T03:40:15.381 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-18T03:40:15.489 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-18T03:40:15.594 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-18T03:40:15.594 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:40:15.595 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:15.638 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:40:15.639 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:15.722 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:15.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.971+0000 7facfd1ff700 1 Processor -- start 2023-07-18T03:40:15.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.971+0000 7facfd1ff700 1 -- start start 2023-07-18T03:40:15.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.972+0000 7facfd1ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7facf815c950 0x7facf80b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.972+0000 7facfd1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7facf80b1e00 0x7facf80d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facfd1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facf80b2350 0x7facf80d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facfd1ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf80d2ce0 con 0x7facf815c950 2023-07-18T03:40:15.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facfd1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf80b3350 con 0x7facf80b1e00 2023-07-18T03:40:15.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facfd1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf80b3630 con 0x7facf80b2350 2023-07-18T03:40:15.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facf6d9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7facf815c950 0x7facf80b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facf6d9d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7facf815c950 0x7facf80b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49116/0 (socket says 172.21.15.49:49116) 2023-07-18T03:40:15.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facf6d9d700 1 -- 172.21.15.49:0/2976639333 learned_addr learned my addr 172.21.15.49:0/2976639333 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:15.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facf759e700 1 --2- 172.21.15.49:0/2976639333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facf80b2350 0x7facf80d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7faceffff700 1 --2- 172.21.15.49:0/2976639333 >> v2:172.21.15.107:3300/0 conn(0x7facf80b1e00 0x7facf80d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facf6d9d700 1 -- 172.21.15.49:0/2976639333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facf80b2350 msgr2=0x7facf80d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:15.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facf6d9d700 1 --2- 172.21.15.49:0/2976639333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facf80b2350 0x7facf80d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facf6d9d700 1 -- 172.21.15.49:0/2976639333 >> v2:172.21.15.107:3300/0 conn(0x7facf80b1e00 msgr2=0x7facf80d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:15.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facf6d9d700 1 --2- 172.21.15.49:0/2976639333 >> v2:172.21.15.107:3300/0 conn(0x7facf80b1e00 0x7facf80d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facf6d9d700 1 -- 172.21.15.49:0/2976639333 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf80b3910 con 0x7facf815c950 2023-07-18T03:40:15.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7faceffff700 1 --2- 172.21.15.49:0/2976639333 >> v2:172.21.15.107:3300/0 conn(0x7facf80b1e00 0x7facf80d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:15.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.973+0000 7facf759e700 1 --2- 172.21.15.49:0/2976639333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facf80b2350 0x7facf80d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:15.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.974+0000 7facf6d9d700 1 --2- 172.21.15.49:0/2976639333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facf815c950 0x7facf80b43f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7facf8072140 tx=0x7face8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7014fde523ff4e68 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:15.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.974+0000 7facf659c700 1 -- 172.21.15.49:0/2976639333 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8005e30 con 0x7facf815c950 2023-07-18T03:40:15.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.974+0000 7facf659c700 1 -- 172.21.15.49:0/2976639333 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face8004030 con 0x7facf815c950 2023-07-18T03:40:15.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.974+0000 7facf659c700 1 -- 172.21.15.49:0/2976639333 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face80043e0 con 0x7facf815c950 2023-07-18T03:40:15.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.974+0000 7facfd1ff700 1 -- 172.21.15.49:0/2976639333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facf815c950 msgr2=0x7facf80b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:15.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.974+0000 7facfd1ff700 1 --2- 172.21.15.49:0/2976639333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facf815c950 0x7facf80b43f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7facf8072140 tx=0x7face8009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.975+0000 7facfd1ff700 1 -- 172.21.15.49:0/2976639333 shutdown_connections 2023-07-18T03:40:15.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.975+0000 7facfd1ff700 1 --2- 172.21.15.49:0/2976639333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facf80b2350 0x7facf80d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.975+0000 7facfd1ff700 1 --2- 172.21.15.49:0/2976639333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facf815c950 0x7facf80b43f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.975+0000 7facfd1ff700 1 --2- 172.21.15.49:0/2976639333 >> v2:172.21.15.107:3300/0 conn(0x7facf80b1e00 0x7facf80d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.975+0000 7facfd1ff700 1 -- 172.21.15.49:0/2976639333 >> 172.21.15.49:0/2976639333 conn(0x7facf809c1d0 msgr2=0x7facf80d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:15.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.975+0000 7facfd1ff700 1 -- 172.21.15.49:0/2976639333 shutdown_connections 2023-07-18T03:40:15.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.975+0000 7facfd1ff700 1 -- 172.21.15.49:0/2976639333 wait complete. 2023-07-18T03:40:15.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.977+0000 7facfd1ff700 1 Processor -- start 2023-07-18T03:40:15.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.977+0000 7facfd1ff700 1 -- start start 2023-07-18T03:40:15.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.978+0000 7facfd1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facf80b1e00 0x7facf81f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.978+0000 7facfd1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facf80b2350 0x7facf80ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.978+0000 7facfd1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf815c950 0x7facf80b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.978+0000 7facfd1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf81f2ee0 con 0x7facf80b1e00 2023-07-18T03:40:15.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.978+0000 7facfd1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf815dea0 con 0x7facf815c950 2023-07-18T03:40:15.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.978+0000 7facfd1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf815e740 con 0x7facf80b2350 2023-07-18T03:40:15.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.978+0000 7facf6d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facf80b1e00 0x7facf81f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.978+0000 7facf6d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facf80b1e00 0x7facf81f29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49130/0 (socket says 172.21.15.49:49130) 2023-07-18T03:40:15.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.978+0000 7facf6d9d700 1 -- 172.21.15.49:0/2154050303 learned_addr learned my addr 172.21.15.49:0/2154050303 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:15.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.978+0000 7facf759e700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf815c950 0x7facf80b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.978+0000 7faceffff700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facf80b2350 0x7facf80ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.979+0000 7facf6d9d700 1 -- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facf80b2350 msgr2=0x7facf80ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:15.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.979+0000 7facf6d9d700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facf80b2350 0x7facf80ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.979+0000 7facf6d9d700 1 -- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf815c950 msgr2=0x7facf80b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:15.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.979+0000 7facf6d9d700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf815c950 0x7facf80b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:15.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.979+0000 7facf6d9d700 1 -- 172.21.15.49:0/2154050303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face8009500 con 0x7facf80b1e00 2023-07-18T03:40:15.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.979+0000 7faceffff700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facf80b2350 0x7facf80ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:15.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.979+0000 7facf6d9d700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facf80b1e00 0x7facf81f29a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7face8000c00 tx=0x7face8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:15.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.979+0000 7facef7fe700 1 -- 172.21.15.49:0/2154050303 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8003dc0 con 0x7facf80b1e00 2023-07-18T03:40:15.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.979+0000 7facef7fe700 1 -- 172.21.15.49:0/2154050303 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face8011420 con 0x7facf80b1e00 2023-07-18T03:40:15.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.979+0000 7facef7fe700 1 -- 172.21.15.49:0/2154050303 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face80117d0 con 0x7facf80b1e00 2023-07-18T03:40:15.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.979+0000 7facfd1ff700 1 -- 172.21.15.49:0/2154050303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf80ce6d0 con 0x7facf80b1e00 2023-07-18T03:40:15.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.979+0000 7facfd1ff700 1 -- 172.21.15.49:0/2154050303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf80b8b60 con 0x7facf80b1e00 2023-07-18T03:40:15.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.980+0000 7facfd1ff700 1 -- 172.21.15.49:0/2154050303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf804e280 con 0x7facf80b1e00 2023-07-18T03:40:15.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.981+0000 7facef7fe700 1 -- 172.21.15.49:0/2154050303 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7face8011930 con 0x7facf80b1e00 2023-07-18T03:40:15.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.982+0000 7facef7fe700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7facd806ace0 0x7facd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:15.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.982+0000 7facef7fe700 1 -- 172.21.15.49:0/2154050303 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7face808a480 con 0x7facf80b1e00 2023-07-18T03:40:15.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.983+0000 7faceffff700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7facd806ace0 0x7facd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:15.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.984+0000 7faceffff700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7facd806ace0 0x7facd806d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facf815f9b0 tx=0x7face0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:15.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:15.987+0000 7facef7fe700 1 -- 172.21.15.49:0/2154050303 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7face8055520 con 0x7facf80b1e00 2023-07-18T03:40:16.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.195+0000 7facfd1ff700 1 -- 172.21.15.49:0/2154050303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facf80d2ac0 con 0x7facf80b1e00 2023-07-18T03:40:16.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.196+0000 7facef7fe700 1 -- 172.21.15.49:0/2154050303 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18257 (secure 0 0 0) 0x7face8058b40 con 0x7facf80b1e00 2023-07-18T03:40:16.192 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:16.192 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:58.343119+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:39:54.412334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-18T03:39:55.841218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:16.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.199+0000 7facfd1ff700 1 -- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7facd806ace0 msgr2=0x7facd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:16.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.199+0000 7facfd1ff700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7facd806ace0 0x7facd806d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facf815f9b0 tx=0x7face0008040 comp rx=0 tx=0).stop 2023-07-18T03:40:16.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.199+0000 7facfd1ff700 1 -- 172.21.15.49:0/2154050303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facf80b1e00 msgr2=0x7facf81f29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:16.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.199+0000 7facfd1ff700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facf80b1e00 0x7facf81f29a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7face8000c00 tx=0x7face8004750 comp rx=0 tx=0).stop 2023-07-18T03:40:16.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.199+0000 7facfd1ff700 1 -- 172.21.15.49:0/2154050303 shutdown_connections 2023-07-18T03:40:16.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.199+0000 7facfd1ff700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facf815c950 0x7facf80b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.199+0000 7facfd1ff700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7facd806ace0 0x7facd806d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.199+0000 7facfd1ff700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facf80b1e00 0x7facf81f29a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.199+0000 7facfd1ff700 1 --2- 172.21.15.49:0/2154050303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facf80b2350 0x7facf80ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.199+0000 7facfd1ff700 1 -- 172.21.15.49:0/2154050303 >> 172.21.15.49:0/2154050303 conn(0x7facf809c1d0 msgr2=0x7facf806b120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:16.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.199+0000 7facfd1ff700 1 -- 172.21.15.49:0/2154050303 shutdown_connections 2023-07-18T03:40:16.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.199+0000 7facfd1ff700 1 -- 172.21.15.49:0/2154050303 wait complete. 2023-07-18T03:40:16.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:39:54.412334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:39:55.841218+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:40:16.214 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:40:16.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.427+0000 7f3d931ff700 1 Processor -- start 2023-07-18T03:40:16.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.428+0000 7f3d931ff700 1 -- start start 2023-07-18T03:40:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.429+0000 7f3d931ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d8c0b4860 0x7f3d8c0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.429+0000 7f3d931ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3d8c0d12c0 0x7f3d8c0d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.429+0000 7f3d931ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d8c0d02a0 con 0x7f3d8c0b8660 2023-07-18T03:40:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.429+0000 7f3d931ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8c0b5210 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.429+0000 7f3d931ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c0cfce0 con 0x7f3d8c0b4860 2023-07-18T03:40:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.429+0000 7f3d9179c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3d8c0d12c0 0x7f3d8c0d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.429+0000 7f3d9179c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3d8c0d12c0 0x7f3d8c0d1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49146/0 (socket says 172.21.15.49:49146) 2023-07-18T03:40:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.429+0000 7f3d9179c700 1 -- 172.21.15.49:0/2330885972 learned_addr learned my addr 172.21.15.49:0/2330885972 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.429+0000 7f3d8bfff700 1 --2- 172.21.15.49:0/2330885972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d8c0b4860 0x7f3d8c0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.430+0000 7f3d9179c700 1 -- 172.21.15.49:0/2330885972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d8c0b4860 msgr2=0x7f3d8c0b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.430+0000 7f3d9179c700 1 --2- 172.21.15.49:0/2330885972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d8c0b4860 0x7f3d8c0b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.430+0000 7f3d9179c700 1 -- 172.21.15.49:0/2330885972 >> v1:172.21.15.107:6789/0 conn(0x7f3d8c0b8660 legacy=0x7f3d8c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:16.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.430+0000 7f3d9179c700 1 -- 172.21.15.49:0/2330885972 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c0cffc0 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.430+0000 7f3d9179c700 1 --2- 172.21.15.49:0/2330885972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d8c0d12c0 0x7f3d8c0d1700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009e50 tx=0x7f3d8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d90542206cf21f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:16.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.430+0000 7f3d8b7fe700 1 -- 172.21.15.49:0/2330885972 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400e070 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.430+0000 7f3d8b7fe700 1 -- 172.21.15.49:0/2330885972 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8400b910 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.430+0000 7f3d8b7fe700 1 -- 172.21.15.49:0/2330885972 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400bcc0 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.431+0000 7f3d931ff700 1 -- 172.21.15.49:0/2330885972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d8c0d12c0 msgr2=0x7f3d8c0d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:16.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.431+0000 7f3d931ff700 1 --2- 172.21.15.49:0/2330885972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d8c0d12c0 0x7f3d8c0d1700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009e50 tx=0x7f3d8400b330 comp rx=0 tx=0).stop 2023-07-18T03:40:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.431+0000 7f3d931ff700 1 -- 172.21.15.49:0/2330885972 shutdown_connections 2023-07-18T03:40:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.431+0000 7f3d931ff700 1 --2- 172.21.15.49:0/2330885972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d8c0d12c0 0x7f3d8c0d1700 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.431+0000 7f3d931ff700 1 --2- 172.21.15.49:0/2330885972 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d8c0b4860 0x7f3d8c0b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.431+0000 7f3d931ff700 1 -- 172.21.15.49:0/2330885972 >> 172.21.15.49:0/2330885972 conn(0x7f3d8c066860 msgr2=0x7f3d8c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.431+0000 7f3d931ff700 1 -- 172.21.15.49:0/2330885972 shutdown_connections 2023-07-18T03:40:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.431+0000 7f3d931ff700 1 -- 172.21.15.49:0/2330885972 wait complete. 2023-07-18T03:40:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.433+0000 7f3d931ff700 1 Processor -- start 2023-07-18T03:40:16.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.433+0000 7f3d931ff700 1 -- start start 2023-07-18T03:40:16.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.434+0000 7f3d931ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d8c0b4860 0x7f3d8c144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.434+0000 7f3d931ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d8c0b8660 0x7f3d8c145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.434+0000 7f3d931ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d8c0d12c0 0x7f3d8c1f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.434+0000 7f3d931ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c145750 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.434+0000 7f3d931ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c144b40 con 0x7f3d8c0b8660 2023-07-18T03:40:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.434+0000 7f3d931ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c144e20 con 0x7f3d8c0b4860 2023-07-18T03:40:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.434+0000 7f3d9179c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d8c0d12c0 0x7f3d8c1f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.434+0000 7f3d9179c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d8c0d12c0 0x7f3d8c1f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49152/0 (socket says 172.21.15.49:49152) 2023-07-18T03:40:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.434+0000 7f3d9179c700 1 -- 172.21.15.49:0/680516049 learned_addr learned my addr 172.21.15.49:0/680516049 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.434+0000 7f3d90f9b700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d8c0b4860 0x7f3d8c144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.434+0000 7f3d8bfff700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d8c0b8660 0x7f3d8c145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.435+0000 7f3d9179c700 1 -- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d8c0b4860 msgr2=0x7f3d8c144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.435+0000 7f3d9179c700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d8c0b4860 0x7f3d8c144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.435+0000 7f3d9179c700 1 -- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d8c0b8660 msgr2=0x7f3d8c145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.435+0000 7f3d9179c700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d8c0b8660 0x7f3d8c145be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.435+0000 7f3d9179c700 1 -- 172.21.15.49:0/680516049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d84009b30 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.435+0000 7f3d9179c700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d8c0d12c0 0x7f3d8c1f1e80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0673b0 tx=0x7f3d84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.435+0000 7f3d897fa700 1 -- 172.21.15.49:0/680516049 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84004800 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.435+0000 7f3d897fa700 1 -- 172.21.15.49:0/680516049 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d84004960 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.435+0000 7f3d897fa700 1 -- 172.21.15.49:0/680516049 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8401c680 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.435+0000 7f3d931ff700 1 -- 172.21.15.49:0/680516049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8c145100 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.435+0000 7f3d931ff700 1 -- 172.21.15.49:0/680516049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8c146120 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.436+0000 7f3d931ff700 1 -- 172.21.15.49:0/680516049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8c145900 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.437+0000 7f3d897fa700 1 -- 172.21.15.49:0/680516049 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3d8401c7e0 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.439+0000 7f3d897fa700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d7006ace0 0x7f3d7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.440+0000 7f3d897fa700 1 -- 172.21.15.49:0/680516049 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3d8408afe0 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.440+0000 7f3d90f9b700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d7006ace0 0x7f3d7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.441+0000 7f3d90f9b700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d7006ace0 0x7f3d7006d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3d80007fd0 tx=0x7f3d80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:16.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.444+0000 7f3d897fa700 1 -- 172.21.15.49:0/680516049 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3d8c145900 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.647+0000 7f3d931ff700 1 -- 172.21.15.49:0/680516049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3d8c1462b0 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.648+0000 7f3d897fa700 1 -- 172.21.15.49:0/680516049 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3d840560e0 con 0x7f3d8c0d12c0 2023-07-18T03:40:16.643 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:40:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.650+0000 7f3d931ff700 1 -- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d7006ace0 msgr2=0x7f3d7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.650+0000 7f3d931ff700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d7006ace0 0x7f3d7006d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3d80007fd0 tx=0x7f3d80009040 comp rx=0 tx=0).stop 2023-07-18T03:40:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.650+0000 7f3d931ff700 1 -- 172.21.15.49:0/680516049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d8c0d12c0 msgr2=0x7f3d8c1f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.650+0000 7f3d931ff700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d8c0d12c0 0x7f3d8c1f1e80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0673b0 tx=0x7f3d84004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.650+0000 7f3d931ff700 1 -- 172.21.15.49:0/680516049 shutdown_connections 2023-07-18T03:40:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.650+0000 7f3d931ff700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3d7006ace0 0x7f3d7006d190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.650+0000 7f3d931ff700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d8c0d12c0 0x7f3d8c1f1e80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.650+0000 7f3d931ff700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d8c0b4860 0x7f3d8c144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.650+0000 7f3d931ff700 1 --2- 172.21.15.49:0/680516049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d8c0b8660 0x7f3d8c145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.650+0000 7f3d931ff700 1 -- 172.21.15.49:0/680516049 >> 172.21.15.49:0/680516049 conn(0x7f3d8c066860 msgr2=0x7f3d8c0d16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.650+0000 7f3d931ff700 1 -- 172.21.15.49:0/680516049 shutdown_connections 2023-07-18T03:40:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.650+0000 7f3d931ff700 1 -- 172.21.15.49:0/680516049 wait complete. 2023-07-18T03:40:16.664 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:40:16.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.877+0000 7f2f047ff700 1 Processor -- start 2023-07-18T03:40:16.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.877+0000 7f2f047ff700 1 -- start start 2023-07-18T03:40:16.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.878+0000 7f2f047ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2efc0b4860 0x7f2efc0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.878+0000 7f2f047ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2efc0d12c0 0x7f2efc0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.878+0000 7f2f047ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2efc0d02a0 con 0x7f2efc0b8660 2023-07-18T03:40:16.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.878+0000 7f2f047ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2efc0b5240 con 0x7f2efc0b4860 2023-07-18T03:40:16.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.878+0000 7f2f047ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc0cfce0 con 0x7f2efc0d12c0 2023-07-18T03:40:16.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.878+0000 7f2f0259b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2efc0b8660 0x7f2efc0cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49212/0 (socket says 172.21.15.49:49212) 2023-07-18T03:40:16.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.878+0000 7f2f0259b700 1 -- 172.21.15.49:0/1355462096 learned_addr learned my addr 172.21.15.49:0/1355462096 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f02d9c700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2efc0d12c0 0x7f2efc0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01d9a700 1 --2- 172.21.15.49:0/1355462096 >> v2:172.21.15.107:3300/0 conn(0x7f2efc0b4860 0x7f2efc0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653284406 0 0) 0x7f2efc0d02a0 con 0x7f2efc0b8660 2023-07-18T03:40:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ee80035b0 con 0x7f2efc0b8660 2023-07-18T03:40:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3303123588 0 0) 0x7f2ee80035b0 con 0x7f2efc0b8660 2023-07-18T03:40:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2efc0d12c0 msgr2=0x7f2efc0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2efc0d12c0 0x7f2efc0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 >> v2:172.21.15.107:3300/0 conn(0x7f2efc0b4860 msgr2=0x7f2efc0b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 --2- 172.21.15.49:0/1355462096 >> v2:172.21.15.107:3300/0 conn(0x7f2efc0b4860 0x7f2efc0b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc0cffc0 con 0x7f2efc0b8660 2023-07-18T03:40:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f2ef8001670 con 0x7f2efc0b8660 2023-07-18T03:40:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 >> v1:172.21.15.49:6789/0 conn(0x7f2efc0b8660 legacy=0x7f2efc0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee8003a00 0x7f2ee8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee8004430 0x7f2ee80048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee8004f10 0x7f2ee800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc0cfce0 con 0x7f2ee8004f10 2023-07-18T03:40:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.879+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee8005720 con 0x7f2ee8003a00 2023-07-18T03:40:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.880+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee800bfd0 con 0x7f2ee8004430 2023-07-18T03:40:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.880+0000 7f2f02d9c700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2efc0d12c0 0x7f2efc0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.880+0000 7f2f02d9c700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee8004f10 0x7f2ee800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.880+0000 7f2f01d9a700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee8004430 0x7f2ee80048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.880+0000 7f2f0259b700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee8003a00 0x7f2ee8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.880+0000 7f2f02d9c700 1 -- 172.21.15.49:0/1355462096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee8004430 msgr2=0x7f2ee80048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:16.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.880+0000 7f2f02d9c700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee8004430 0x7f2ee80048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.880+0000 7f2f02d9c700 1 -- 172.21.15.49:0/1355462096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee8003a00 msgr2=0x7f2ee8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:16.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.880+0000 7f2f02d9c700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee8003a00 0x7f2ee8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.880+0000 7f2f02d9c700 1 -- 172.21.15.49:0/1355462096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee80035b0 con 0x7f2ee8004f10 2023-07-18T03:40:16.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.880+0000 7f2f02d9c700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee8004f10 0x7f2ee800ba10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400e1f0 tx=0x7f2ee8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd35962f1d746e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:16.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.881+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef400ec90 con 0x7f2ee8004f10 2023-07-18T03:40:16.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.881+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef400f040 con 0x7f2ee8004f10 2023-07-18T03:40:16.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.881+0000 7f2f01599700 1 -- 172.21.15.49:0/1355462096 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4003a20 con 0x7f2ee8004f10 2023-07-18T03:40:16.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.881+0000 7f2f047ff700 1 -- 172.21.15.49:0/1355462096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee8004f10 msgr2=0x7f2ee800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:16.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.881+0000 7f2f047ff700 1 --2- 172.21.15.49:0/1355462096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ee8004f10 0x7f2ee800ba10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400e1f0 tx=0x7f2ee8005880 comp rx=0 tx=0).stop 2023-07-18T03:40:16.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.881+0000 7f2f02d9c700 1 -- 172.21.15.49:0/1355462096 reap_dead start 2023-07-18T03:40:16.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.881+0000 7f2f047ff700 1 -- 172.21.15.49:0/1355462096 shutdown_connections 2023-07-18T03:40:16.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.881+0000 7f2f047ff700 1 -- 172.21.15.49:0/1355462096 >> 172.21.15.49:0/1355462096 conn(0x7f2efc066860 msgr2=0x7f2efc06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:16.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.881+0000 7f2f047ff700 1 -- 172.21.15.49:0/1355462096 shutdown_connections 2023-07-18T03:40:16.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.881+0000 7f2f047ff700 1 -- 172.21.15.49:0/1355462096 wait complete. 2023-07-18T03:40:16.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.883+0000 7f2f047ff700 1 Processor -- start 2023-07-18T03:40:16.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.883+0000 7f2f047ff700 1 -- start start 2023-07-18T03:40:16.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.884+0000 7f2f047ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee8004f10 0x7f2efc15be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.884+0000 7f2f047ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2efc0d16c0 0x7f2efc0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.884+0000 7f2f047ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc1480f0 0x7f2efc148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.884+0000 7f2f047ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc149aa0 con 0x7f2efc1480f0 2023-07-18T03:40:16.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.884+0000 7f2f047ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc0d2070 con 0x7f2efc0d16c0 2023-07-18T03:40:16.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.884+0000 7f2f047ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc148e20 con 0x7f2ee8004f10 2023-07-18T03:40:16.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.885+0000 7f2f02d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc1480f0 0x7f2efc148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.885+0000 7f2f02d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc1480f0 0x7f2efc148560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49168/0 (socket says 172.21.15.49:49168) 2023-07-18T03:40:16.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.885+0000 7f2f02d9c700 1 -- 172.21.15.49:0/2181048265 learned_addr learned my addr 172.21.15.49:0/2181048265 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:16.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.885+0000 7f2f0259b700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee8004f10 0x7f2efc15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.885+0000 7f2f01d9a700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2efc0d16c0 0x7f2efc0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.885+0000 7f2f02d9c700 1 -- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee8004f10 msgr2=0x7f2efc15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:16.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.885+0000 7f2f02d9c700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee8004f10 0x7f2efc15be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.885+0000 7f2f02d9c700 1 -- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2efc0d16c0 msgr2=0x7f2efc0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:16.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.885+0000 7f2f02d9c700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2efc0d16c0 0x7f2efc0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:16.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.885+0000 7f2f02d9c700 1 -- 172.21.15.49:0/2181048265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc149100 con 0x7f2efc1480f0 2023-07-18T03:40:16.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.886+0000 7f2f02d9c700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc1480f0 0x7f2efc148560 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4015000 tx=0x7f2ef4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:16.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.886+0000 7f2ef2ffd700 1 -- 172.21.15.49:0/2181048265 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4004230 con 0x7f2efc1480f0 2023-07-18T03:40:16.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.886+0000 7f2ef2ffd700 1 -- 172.21.15.49:0/2181048265 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef4023070 con 0x7f2efc1480f0 2023-07-18T03:40:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.886+0000 7f2ef2ffd700 1 -- 172.21.15.49:0/2181048265 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4004610 con 0x7f2efc1480f0 2023-07-18T03:40:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.886+0000 7f2f047ff700 1 -- 172.21.15.49:0/2181048265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2efc1493e0 con 0x7f2efc1480f0 2023-07-18T03:40:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.886+0000 7f2f047ff700 1 -- 172.21.15.49:0/2181048265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2efc149ec0 con 0x7f2efc1480f0 2023-07-18T03:40:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.887+0000 7f2f047ff700 1 -- 172.21.15.49:0/2181048265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2efc149be0 con 0x7f2efc1480f0 2023-07-18T03:40:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.888+0000 7f2ef2ffd700 1 -- 172.21.15.49:0/2181048265 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2ef4025030 con 0x7f2efc1480f0 2023-07-18T03:40:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.889+0000 7f2ef2ffd700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2ee006ace0 0x7f2ee006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.889+0000 7f2ef2ffd700 1 -- 172.21.15.49:0/2181048265 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2ef408ae70 con 0x7f2efc1480f0 2023-07-18T03:40:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.890+0000 7f2f0259b700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2ee006ace0 0x7f2ee006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.890+0000 7f2f0259b700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2ee006ace0 0x7f2ee006d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8000c00 tx=0x7f2ef8007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.893+0000 7f2ef2ffd700 1 -- 172.21.15.49:0/2181048265 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2efc149be0 con 0x7f2efc1480f0 2023-07-18T03:40:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:16.952+0000 7f2ef2ffd700 1 -- 172.21.15.49:0/2181048265 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f2ef4009cc0 con 0x7f2efc1480f0 2023-07-18T03:40:17.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.103+0000 7f2f047ff700 1 -- 172.21.15.49:0/2181048265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2efc149be0 con 0x7f2efc1480f0 2023-07-18T03:40:17.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.104+0000 7f2ef2ffd700 1 -- 172.21.15.49:0/2181048265 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2efc149be0 con 0x7f2efc1480f0 2023-07-18T03:40:17.099 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:40:17.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.106+0000 7f2f047ff700 1 -- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2ee006ace0 msgr2=0x7f2ee006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:17.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.106+0000 7f2f047ff700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2ee006ace0 0x7f2ee006d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8000c00 tx=0x7f2ef8007980 comp rx=0 tx=0).stop 2023-07-18T03:40:17.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.106+0000 7f2f047ff700 1 -- 172.21.15.49:0/2181048265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc1480f0 msgr2=0x7f2efc148560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:17.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.106+0000 7f2f047ff700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc1480f0 0x7f2efc148560 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4015000 tx=0x7f2ef4009750 comp rx=0 tx=0).stop 2023-07-18T03:40:17.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.106+0000 7f2f047ff700 1 -- 172.21.15.49:0/2181048265 shutdown_connections 2023-07-18T03:40:17.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.106+0000 7f2f047ff700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2ee006ace0 0x7f2ee006d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.106+0000 7f2f047ff700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2efc1480f0 0x7f2efc148560 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.106+0000 7f2f047ff700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee8004f10 0x7f2efc15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.106+0000 7f2f047ff700 1 --2- 172.21.15.49:0/2181048265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2efc0d16c0 0x7f2efc0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.106+0000 7f2f047ff700 1 -- 172.21.15.49:0/2181048265 >> 172.21.15.49:0/2181048265 conn(0x7f2efc066860 msgr2=0x7f2efc0d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:17.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.106+0000 7f2f047ff700 1 -- 172.21.15.49:0/2181048265 shutdown_connections 2023-07-18T03:40:17.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.106+0000 7f2f047ff700 1 -- 172.21.15.49:0/2181048265 wait complete. 2023-07-18T03:40:17.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:40:17.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.333+0000 7f09377ff700 1 Processor -- start 2023-07-18T03:40:17.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.333+0000 7f09377ff700 1 -- start start 2023-07-18T03:40:17.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.334+0000 7f09377ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f09300b8660 0x7f09300cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:17.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.334+0000 7f09377ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f09300cd970 0x7f09300b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:17.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.334+0000 7f09377ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09300b4da0 0x7f09301601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:17.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.334+0000 7f09377ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09300d02a0 con 0x7f09300cd970 2023-07-18T03:40:17.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.334+0000 7f09377ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09300b51b0 con 0x7f09300b8660 2023-07-18T03:40:17.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.334+0000 7f09377ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09300cfce0 con 0x7f09300b4da0 2023-07-18T03:40:17.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.334+0000 7f0934d9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f09300cd970 0x7f09300b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:17.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.334+0000 7f0934d9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f09300cd970 0x7f09300b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49178/0 (socket says 172.21.15.49:49178) 2023-07-18T03:40:17.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.334+0000 7f0934d9a700 1 -- 172.21.15.49:0/640338062 learned_addr learned my addr 172.21.15.49:0/640338062 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:17.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.335+0000 7f093559b700 1 --2- 172.21.15.49:0/640338062 >> v2:172.21.15.107:3300/0 conn(0x7f09300b8660 0x7f09300cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:17.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.335+0000 7f0935d9c700 1 --2- 172.21.15.49:0/640338062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09300b4da0 0x7f09301601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:17.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.335+0000 7f0934d9a700 1 -- 172.21.15.49:0/640338062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09300b4da0 msgr2=0x7f09301601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:17.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.335+0000 7f0934d9a700 1 --2- 172.21.15.49:0/640338062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09300b4da0 0x7f09301601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.335+0000 7f0934d9a700 1 -- 172.21.15.49:0/640338062 >> v2:172.21.15.107:3300/0 conn(0x7f09300b8660 msgr2=0x7f09300cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:17.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.335+0000 7f0934d9a700 1 --2- 172.21.15.49:0/640338062 >> v2:172.21.15.107:3300/0 conn(0x7f09300b8660 0x7f09300cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.335+0000 7f0934d9a700 1 -- 172.21.15.49:0/640338062 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09300cffc0 con 0x7f09300cd970 2023-07-18T03:40:17.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.335+0000 7f0934d9a700 1 --2- 172.21.15.49:0/640338062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09300cd970 0x7f09300b4860 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f09300c9e60 tx=0x7f0920009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a914c6a181a8c9cf server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:17.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.335+0000 7f0927fff700 1 -- 172.21.15.49:0/640338062 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092000b870 con 0x7f09300cd970 2023-07-18T03:40:17.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.335+0000 7f0927fff700 1 -- 172.21.15.49:0/640338062 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092000b9d0 con 0x7f09300cd970 2023-07-18T03:40:17.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.335+0000 7f0927fff700 1 -- 172.21.15.49:0/640338062 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092000bd60 con 0x7f09300cd970 2023-07-18T03:40:17.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.336+0000 7f09377ff700 1 -- 172.21.15.49:0/640338062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09300cd970 msgr2=0x7f09300b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:17.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.336+0000 7f09377ff700 1 --2- 172.21.15.49:0/640338062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09300cd970 0x7f09300b4860 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f09300c9e60 tx=0x7f0920009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.336+0000 7f09377ff700 1 -- 172.21.15.49:0/640338062 shutdown_connections 2023-07-18T03:40:17.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.336+0000 7f09377ff700 1 --2- 172.21.15.49:0/640338062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09300b4da0 0x7f09301601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.336+0000 7f09377ff700 1 --2- 172.21.15.49:0/640338062 >> v2:172.21.15.107:3300/0 conn(0x7f09300b8660 0x7f09300cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.336+0000 7f09377ff700 1 --2- 172.21.15.49:0/640338062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09300cd970 0x7f09300b4860 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.336+0000 7f09377ff700 1 -- 172.21.15.49:0/640338062 >> 172.21.15.49:0/640338062 conn(0x7f0930066860 msgr2=0x7f093006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:17.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.336+0000 7f09377ff700 1 -- 172.21.15.49:0/640338062 shutdown_connections 2023-07-18T03:40:17.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.336+0000 7f09377ff700 1 -- 172.21.15.49:0/640338062 wait complete. 2023-07-18T03:40:17.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.338+0000 7f09377ff700 1 Processor -- start 2023-07-18T03:40:17.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.338+0000 7f09377ff700 1 -- start start 2023-07-18T03:40:17.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.339+0000 7f09377ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09300b4da0 0x7f09301f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:17.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.339+0000 7f09377ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09300b8660 0x7f09301f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:17.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.339+0000 7f09377ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09300cd970 0x7f09300c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:17.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.339+0000 7f09377ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09300cb940 con 0x7f09300b8660 2023-07-18T03:40:17.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.339+0000 7f09377ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09300c8d20 con 0x7f09300b4da0 2023-07-18T03:40:17.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.339+0000 7f09377ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09300cafa0 con 0x7f09300cd970 2023-07-18T03:40:17.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.339+0000 7f0934d9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09300b8660 0x7f09301f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:17.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.339+0000 7f0934d9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09300b8660 0x7f09301f2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49188/0 (socket says 172.21.15.49:49188) 2023-07-18T03:40:17.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.339+0000 7f0934d9a700 1 -- 172.21.15.49:0/1228858168 learned_addr learned my addr 172.21.15.49:0/1228858168 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:17.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.339+0000 7f093559b700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09300b4da0 0x7f09301f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:17.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.339+0000 7f0935d9c700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09300cd970 0x7f09300c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:17.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.340+0000 7f0934d9a700 1 -- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09300cd970 msgr2=0x7f09300c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:17.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.340+0000 7f0934d9a700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09300cd970 0x7f09300c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.340+0000 7f0934d9a700 1 -- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09300b4da0 msgr2=0x7f09301f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:17.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.340+0000 7f0934d9a700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09300b4da0 0x7f09301f54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.340+0000 7f0934d9a700 1 -- 172.21.15.49:0/1228858168 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0920009550 con 0x7f09300b8660 2023-07-18T03:40:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.340+0000 7f0934d9a700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09300b8660 0x7f09301f2540 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0920000c00 tx=0x7f0920004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.340+0000 7f0925ffb700 1 -- 172.21.15.49:0/1228858168 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920004310 con 0x7f09300b8660 2023-07-18T03:40:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.340+0000 7f0925ffb700 1 -- 172.21.15.49:0/1228858168 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092001e070 con 0x7f09300b8660 2023-07-18T03:40:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.340+0000 7f09377ff700 1 -- 172.21.15.49:0/1228858168 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09300cb280 con 0x7f09300b8660 2023-07-18T03:40:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.340+0000 7f0925ffb700 1 -- 172.21.15.49:0/1228858168 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09200036a0 con 0x7f09300b8660 2023-07-18T03:40:17.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.341+0000 7f09377ff700 1 -- 172.21.15.49:0/1228858168 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09301f2a80 con 0x7f09300b8660 2023-07-18T03:40:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.342+0000 7f09377ff700 1 -- 172.21.15.49:0/1228858168 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09301f2c70 con 0x7f09300b8660 2023-07-18T03:40:17.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.342+0000 7f0925ffb700 1 -- 172.21.15.49:0/1228858168 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f0920020030 con 0x7f09300b8660 2023-07-18T03:40:17.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.343+0000 7f0925ffb700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f091406ad70 0x7f091406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:17.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.343+0000 7f0925ffb700 1 -- 172.21.15.49:0/1228858168 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f092008a490 con 0x7f09300b8660 2023-07-18T03:40:17.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.344+0000 7f093559b700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f091406ad70 0x7f091406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:17.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.345+0000 7f093559b700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f091406ad70 0x7f091406d220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f092c006b20 tx=0x7f092c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:17.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.348+0000 7f0925ffb700 1 -- 172.21.15.49:0/1228858168 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0920055450 con 0x7f09300b8660 2023-07-18T03:40:17.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.556+0000 7f09377ff700 1 -- 172.21.15.49:0/1228858168 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09300ce6b0 con 0x7f09300b8660 2023-07-18T03:40:17.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.556+0000 7f0925ffb700 1 -- 172.21.15.49:0/1228858168 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0920058a70 con 0x7f09300b8660 2023-07-18T03:40:17.552 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:40:17.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.559+0000 7f09377ff700 1 -- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f091406ad70 msgr2=0x7f091406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:17.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.559+0000 7f09377ff700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f091406ad70 0x7f091406d220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f092c006b20 tx=0x7f092c008040 comp rx=0 tx=0).stop 2023-07-18T03:40:17.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.559+0000 7f09377ff700 1 -- 172.21.15.49:0/1228858168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09300b8660 msgr2=0x7f09301f2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:17.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.559+0000 7f09377ff700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09300b8660 0x7f09301f2540 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0920000c00 tx=0x7f0920004750 comp rx=0 tx=0).stop 2023-07-18T03:40:17.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.559+0000 7f09377ff700 1 -- 172.21.15.49:0/1228858168 shutdown_connections 2023-07-18T03:40:17.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.559+0000 7f09377ff700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09300cd970 0x7f09300c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.559+0000 7f09377ff700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09300b4da0 0x7f09301f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.559+0000 7f09377ff700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f091406ad70 0x7f091406d220 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.559+0000 7f09377ff700 1 --2- 172.21.15.49:0/1228858168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09300b8660 0x7f09301f2540 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.559+0000 7f09377ff700 1 -- 172.21.15.49:0/1228858168 >> 172.21.15.49:0/1228858168 conn(0x7f0930066860 msgr2=0x7f09300d1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:17.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.559+0000 7f09377ff700 1 -- 172.21.15.49:0/1228858168 shutdown_connections 2023-07-18T03:40:17.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.559+0000 7f09377ff700 1 -- 172.21.15.49:0/1228858168 wait complete. 2023-07-18T03:40:17.572 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:40:17.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.785+0000 7f970a5ff700 1 Processor -- start 2023-07-18T03:40:17.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.785+0000 7f970a5ff700 1 -- start start 2023-07-18T03:40:17.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.786+0000 7f970a5ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f97040b4860 0x7f97040b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:17.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.786+0000 7f970a5ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97040d02a0 con 0x7f97040d12c0 2023-07-18T03:40:17.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.786+0000 7f970a5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97040cfce0 con 0x7f97040b8660 2023-07-18T03:40:17.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.786+0000 7f970a5ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97040b5240 con 0x7f97040b4860 2023-07-18T03:40:17.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.786+0000 7f97037fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f97040b4860 0x7f97040b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:17.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.786+0000 7f9708b9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f97040d12c0 0x7f97040d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49218/0 (socket says 172.21.15.49:49218) 2023-07-18T03:40:17.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.786+0000 7f9708b9c700 1 -- 172.21.15.49:0/3102025490 learned_addr learned my addr 172.21.15.49:0/3102025490 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:17.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.787+0000 7f97037fe700 1 -- 172.21.15.49:0/3102025490 >> v1:172.21.15.107:6789/0 conn(0x7f97040b8660 legacy=0x7f97040cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:17.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.787+0000 7f97037fe700 1 -- 172.21.15.49:0/3102025490 >> v1:172.21.15.49:6789/0 conn(0x7f97040d12c0 legacy=0x7f97040d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:17.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.787+0000 7f97037fe700 1 -- 172.21.15.49:0/3102025490 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97040cffc0 con 0x7f97040b4860 2023-07-18T03:40:17.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.787+0000 7f97037fe700 1 --2- 172.21.15.49:0/3102025490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97040b4860 0x7f97040b4d00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009840 tx=0x7f96f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a4e5dd194b422e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:17.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.787+0000 7f9702ffd700 1 -- 172.21.15.49:0/3102025490 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8011830 con 0x7f97040b4860 2023-07-18T03:40:17.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.787+0000 7f9702ffd700 1 -- 172.21.15.49:0/3102025490 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8011990 con 0x7f97040b4860 2023-07-18T03:40:17.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.787+0000 7f9702ffd700 1 -- 172.21.15.49:0/3102025490 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8011d20 con 0x7f97040b4860 2023-07-18T03:40:17.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.788+0000 7f970a5ff700 1 -- 172.21.15.49:0/3102025490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97040b4860 msgr2=0x7f97040b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:17.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.788+0000 7f970a5ff700 1 --2- 172.21.15.49:0/3102025490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97040b4860 0x7f97040b4d00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009840 tx=0x7f96f8009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:17.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.788+0000 7f970a5ff700 1 -- 172.21.15.49:0/3102025490 shutdown_connections 2023-07-18T03:40:17.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.788+0000 7f970a5ff700 1 --2- 172.21.15.49:0/3102025490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97040b4860 0x7f97040b4d00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.788+0000 7f970a5ff700 1 -- 172.21.15.49:0/3102025490 >> 172.21.15.49:0/3102025490 conn(0x7f9704066860 msgr2=0x7f970406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:17.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.788+0000 7f970a5ff700 1 -- 172.21.15.49:0/3102025490 shutdown_connections 2023-07-18T03:40:17.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.788+0000 7f970a5ff700 1 -- 172.21.15.49:0/3102025490 wait complete. 2023-07-18T03:40:17.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.790+0000 7f970a5ff700 1 Processor -- start 2023-07-18T03:40:17.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.790+0000 7f970a5ff700 1 -- start start 2023-07-18T03:40:17.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f970a5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97040b4860 0x7f97040ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:17.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f970a5ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97040b8660 0x7f970415f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:17.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f970a5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97040d12c0 0x7f970415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:17.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f970a5ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97040cb980 con 0x7f97040b8660 2023-07-18T03:40:17.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f970a5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97040cbc60 con 0x7f97040d12c0 2023-07-18T03:40:17.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f970a5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97040cbf40 con 0x7f97040b4860 2023-07-18T03:40:17.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f97037fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97040b8660 0x7f970415f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:17.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f97037fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97040b8660 0x7f970415f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49206/0 (socket says 172.21.15.49:49206) 2023-07-18T03:40:17.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f97037fe700 1 -- 172.21.15.49:0/2343181368 learned_addr learned my addr 172.21.15.49:0/2343181368 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:17.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f9708b9c700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97040d12c0 0x7f970415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:17.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f9703fff700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97040b4860 0x7f97040ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:17.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f97037fe700 1 -- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97040b4860 msgr2=0x7f97040ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:17.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f97037fe700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97040b4860 0x7f97040ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f97037fe700 1 -- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97040d12c0 msgr2=0x7f970415b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:17.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f97037fe700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97040d12c0 0x7f970415b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:17.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.791+0000 7f97037fe700 1 -- 172.21.15.49:0/2343181368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f80094a0 con 0x7f97040b8660 2023-07-18T03:40:17.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.792+0000 7f9708b9c700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97040d12c0 0x7f970415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:17.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.792+0000 7f9703fff700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97040b4860 0x7f97040ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:17.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.792+0000 7f97037fe700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97040b8660 0x7f970415f000 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f96f80090b0 tx=0x7f96f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:17.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.792+0000 7f9700ff9700 1 -- 172.21.15.49:0/2343181368 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80038a0 con 0x7f97040b8660 2023-07-18T03:40:17.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.792+0000 7f9700ff9700 1 -- 172.21.15.49:0/2343181368 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f801f070 con 0x7f97040b8660 2023-07-18T03:40:17.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.792+0000 7f970a5ff700 1 -- 172.21.15.49:0/2343181368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97040cc220 con 0x7f97040b8660 2023-07-18T03:40:17.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.792+0000 7f9700ff9700 1 -- 172.21.15.49:0/2343181368 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8003c60 con 0x7f97040b8660 2023-07-18T03:40:17.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.792+0000 7f970a5ff700 1 -- 172.21.15.49:0/2343181368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97040cae00 con 0x7f97040b8660 2023-07-18T03:40:17.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.793+0000 7f970a5ff700 1 -- 172.21.15.49:0/2343181368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97040cc740 con 0x7f97040b8660 2023-07-18T03:40:17.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.794+0000 7f9700ff9700 1 -- 172.21.15.49:0/2343181368 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f96f801a430 con 0x7f97040b8660 2023-07-18T03:40:17.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.795+0000 7f9700ff9700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f96f006ad70 0x7f96f006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:17.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.795+0000 7f9700ff9700 1 -- 172.21.15.49:0/2343181368 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f96f8021070 con 0x7f97040b8660 2023-07-18T03:40:17.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.796+0000 7f9703fff700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f96f006ad70 0x7f96f006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:17.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.796+0000 7f9703fff700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f96f006ad70 0x7f96f006d220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97041f5ba0 tx=0x7f96f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:17.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:17.800+0000 7f9700ff9700 1 -- 172.21.15.49:0/2343181368 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f97040cc740 con 0x7f97040b8660 2023-07-18T03:40:18.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.007+0000 7f970a5ff700 1 -- 172.21.15.49:0/2343181368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97040cc740 con 0x7f97040b8660 2023-07-18T03:40:18.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.008+0000 7f9700ff9700 1 -- 172.21.15.49:0/2343181368 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97040cc740 con 0x7f97040b8660 2023-07-18T03:40:18.003 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:40:18.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.010+0000 7f970a5ff700 1 -- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f96f006ad70 msgr2=0x7f96f006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.010+0000 7f970a5ff700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f96f006ad70 0x7f96f006d220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97041f5ba0 tx=0x7f96f4009040 comp rx=0 tx=0).stop 2023-07-18T03:40:18.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.010+0000 7f970a5ff700 1 -- 172.21.15.49:0/2343181368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97040b8660 msgr2=0x7f970415f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.010+0000 7f970a5ff700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97040b8660 0x7f970415f000 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f96f80090b0 tx=0x7f96f8004750 comp rx=0 tx=0).stop 2023-07-18T03:40:18.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.010+0000 7f970a5ff700 1 -- 172.21.15.49:0/2343181368 shutdown_connections 2023-07-18T03:40:18.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.010+0000 7f970a5ff700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97040d12c0 0x7f970415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.010+0000 7f970a5ff700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97040b4860 0x7f97040ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.010+0000 7f970a5ff700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f96f006ad70 0x7f96f006d220 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.011+0000 7f970a5ff700 1 --2- 172.21.15.49:0/2343181368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97040b8660 0x7f970415f000 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.011+0000 7f970a5ff700 1 -- 172.21.15.49:0/2343181368 >> 172.21.15.49:0/2343181368 conn(0x7f9704066860 msgr2=0x7f97040c8920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:18.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.011+0000 7f970a5ff700 1 -- 172.21.15.49:0/2343181368 shutdown_connections 2023-07-18T03:40:18.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.011+0000 7f970a5ff700 1 -- 172.21.15.49:0/2343181368 wait complete. 2023-07-18T03:40:18.025 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-18T03:40:18.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.235+0000 7fd28bdff700 1 Processor -- start 2023-07-18T03:40:18.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.236+0000 7fd28bdff700 1 -- start start 2023-07-18T03:40:18.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.237+0000 7fd28bdff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd28415d9d0 0x7fd28406b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:18.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.237+0000 7fd28bdff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28406c4e0 con 0x7fd28406c9f0 2023-07-18T03:40:18.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.237+0000 7fd28bdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2840b2020 con 0x7fd284071170 2023-07-18T03:40:18.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.237+0000 7fd28bdff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd28415dda0 con 0x7fd28415d9d0 2023-07-18T03:40:18.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.237+0000 7fd289b9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd28415d9d0 0x7fd28406b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:18.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.237+0000 7fd28939a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd28406c9f0 0x7fd2840b1910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49228/0 (socket says 172.21.15.49:49228) 2023-07-18T03:40:18.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.237+0000 7fd28939a700 1 -- 172.21.15.49:0/3349159499 learned_addr learned my addr 172.21.15.49:0/3349159499 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:18.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.237+0000 7fd289b9b700 1 -- 172.21.15.49:0/3349159499 >> v1:172.21.15.107:6789/0 conn(0x7fd284071170 legacy=0x7fd2840715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.238+0000 7fd289b9b700 1 -- 172.21.15.49:0/3349159499 >> v1:172.21.15.49:6789/0 conn(0x7fd28406c9f0 legacy=0x7fd2840b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.238+0000 7fd289b9b700 1 -- 172.21.15.49:0/3349159499 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28406bf20 con 0x7fd28415d9d0 2023-07-18T03:40:18.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.238+0000 7fd288b99700 1 -- 172.21.15.49:0/3349159499 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299763672 0 0) 0x7fd28406c4e0 con 0x7fd28406c9f0 2023-07-18T03:40:18.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.238+0000 7fd289b9b700 1 --2- 172.21.15.49:0/3349159499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28415d9d0 0x7fd28406b5b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd280009840 tx=0x7fd280009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7625cf9d8059790 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:18.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.238+0000 7fd288b99700 1 -- 172.21.15.49:0/3349159499 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280011830 con 0x7fd28415d9d0 2023-07-18T03:40:18.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.238+0000 7fd288b99700 1 -- 172.21.15.49:0/3349159499 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd280011990 con 0x7fd28415d9d0 2023-07-18T03:40:18.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.238+0000 7fd288b99700 1 -- 172.21.15.49:0/3349159499 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280011d20 con 0x7fd28415d9d0 2023-07-18T03:40:18.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.239+0000 7fd28bdff700 1 -- 172.21.15.49:0/3349159499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28415d9d0 msgr2=0x7fd28406b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.239+0000 7fd28bdff700 1 --2- 172.21.15.49:0/3349159499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28415d9d0 0x7fd28406b5b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd280009840 tx=0x7fd280009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:18.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.239+0000 7fd28bdff700 1 -- 172.21.15.49:0/3349159499 shutdown_connections 2023-07-18T03:40:18.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.239+0000 7fd28bdff700 1 --2- 172.21.15.49:0/3349159499 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd28415d9d0 0x7fd28406b5b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.239+0000 7fd28bdff700 1 -- 172.21.15.49:0/3349159499 >> 172.21.15.49:0/3349159499 conn(0x7fd2840b05d0 msgr2=0x7fd2840b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:18.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.239+0000 7fd28bdff700 1 -- 172.21.15.49:0/3349159499 shutdown_connections 2023-07-18T03:40:18.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.239+0000 7fd28bdff700 1 -- 172.21.15.49:0/3349159499 wait complete. 2023-07-18T03:40:18.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.240+0000 7fd28bdff700 1 Processor -- start 2023-07-18T03:40:18.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.241+0000 7fd28bdff700 1 -- start start 2023-07-18T03:40:18.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28bdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd28406c9f0 0x7fd2841f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:18.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28bdff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd284071170 0x7fd284151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:18.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28bdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd28415d9d0 0x7fd2840cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:18.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28bdff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840c3b30 con 0x7fd284071170 2023-07-18T03:40:18.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28bdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2841f65f0 con 0x7fd28406c9f0 2023-07-18T03:40:18.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28bdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2841f6030 con 0x7fd28415d9d0 2023-07-18T03:40:18.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28939a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd284071170 0x7fd284151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:18.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28939a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd284071170 0x7fd284151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49238/0 (socket says 172.21.15.49:49238) 2023-07-18T03:40:18.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28939a700 1 -- 172.21.15.49:0/648604016 learned_addr learned my addr 172.21.15.49:0/648604016 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:18.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd289b9b700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd28406c9f0 0x7fd2841f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:18.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28a39c700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd28415d9d0 0x7fd2840cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:18.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28939a700 1 -- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd28415d9d0 msgr2=0x7fd2840cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28939a700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd28415d9d0 0x7fd2840cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28939a700 1 -- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd28406c9f0 msgr2=0x7fd2841f57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28939a700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd28406c9f0 0x7fd2841f57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28939a700 1 -- 172.21.15.49:0/648604016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2800094a0 con 0x7fd284071170 2023-07-18T03:40:18.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.242+0000 7fd28a39c700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd28415d9d0 0x7fd2840cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.243+0000 7fd28939a700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd284071170 0x7fd284151f90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd274001eb0 tx=0x7fd27400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.243+0000 7fd27a7fc700 1 -- 172.21.15.49:0/648604016 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274004c80 con 0x7fd284071170 2023-07-18T03:40:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.243+0000 7fd27a7fc700 1 -- 172.21.15.49:0/648604016 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd274004de0 con 0x7fd284071170 2023-07-18T03:40:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.243+0000 7fd28bdff700 1 -- 172.21.15.49:0/648604016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2841f6310 con 0x7fd284071170 2023-07-18T03:40:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.243+0000 7fd27a7fc700 1 -- 172.21.15.49:0/648604016 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274007310 con 0x7fd284071170 2023-07-18T03:40:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.243+0000 7fd28bdff700 1 -- 172.21.15.49:0/648604016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2841f5ce0 con 0x7fd284071170 2023-07-18T03:40:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.244+0000 7fd28bdff700 1 -- 172.21.15.49:0/648604016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd284074350 con 0x7fd284071170 2023-07-18T03:40:18.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.245+0000 7fd27a7fc700 1 -- 172.21.15.49:0/648604016 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fd274007470 con 0x7fd284071170 2023-07-18T03:40:18.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.247+0000 7fd27a7fc700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd26806ad70 0x7fd26806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:18.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.247+0000 7fd27a7fc700 1 -- 172.21.15.49:0/648604016 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd2740891a0 con 0x7fd284071170 2023-07-18T03:40:18.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.247+0000 7fd289b9b700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd26806ad70 0x7fd26806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:18.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.248+0000 7fd289b9b700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd26806ad70 0x7fd26806d220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd2800090b0 tx=0x7fd28001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:18.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.251+0000 7fd27a7fc700 1 -- 172.21.15.49:0/648604016 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd2740541f0 con 0x7fd284071170 2023-07-18T03:40:18.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.478+0000 7fd28bdff700 1 -- 172.21.15.49:0/648604016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd2840b4890 con 0x7fd284071170 2023-07-18T03:40:18.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.478+0000 7fd27a7fc700 1 -- 172.21.15.49:0/648604016 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd274057810 con 0x7fd284071170 2023-07-18T03:40:18.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.481+0000 7fd28bdff700 1 -- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd26806ad70 msgr2=0x7fd26806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.481+0000 7fd28bdff700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd26806ad70 0x7fd26806d220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd2800090b0 tx=0x7fd28001c040 comp rx=0 tx=0).stop 2023-07-18T03:40:18.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.481+0000 7fd28bdff700 1 -- 172.21.15.49:0/648604016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd284071170 msgr2=0x7fd284151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.481+0000 7fd28bdff700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd284071170 0x7fd284151f90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd274001eb0 tx=0x7fd27400c800 comp rx=0 tx=0).stop 2023-07-18T03:40:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.481+0000 7fd28bdff700 1 -- 172.21.15.49:0/648604016 shutdown_connections 2023-07-18T03:40:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.481+0000 7fd28bdff700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd28415d9d0 0x7fd2840cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.481+0000 7fd28bdff700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd28406c9f0 0x7fd2841f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.481+0000 7fd28bdff700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd26806ad70 0x7fd26806d220 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.481+0000 7fd28bdff700 1 --2- 172.21.15.49:0/648604016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd284071170 0x7fd284151f90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.481+0000 7fd28bdff700 1 -- 172.21.15.49:0/648604016 >> 172.21.15.49:0/648604016 conn(0x7fd2840b05d0 msgr2=0x7fd2840cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:18.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.481+0000 7fd28bdff700 1 -- 172.21.15.49:0/648604016 shutdown_connections 2023-07-18T03:40:18.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.481+0000 7fd28bdff700 1 -- 172.21.15.49:0/648604016 wait complete. 2023-07-18T03:40:18.478 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-07-18T03:40:18.494 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-18T03:40:18.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:18.495 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-18T03:40:18.589 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:40:18.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:18.591 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-18T03:40:18.637 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:40:18.638 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:18.685 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:40:18.686 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:18.770 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:18.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.980+0000 7f333a5ff700 1 Processor -- start 2023-07-18T03:40:18.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.981+0000 7f333a5ff700 1 -- start start 2023-07-18T03:40:18.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f333a5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f33340d1220 0x7f33340b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:18.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f333a5ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f33340b1e00 0x7f33340afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:18.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f333a5ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33340d2ce0 con 0x7f333415c950 2023-07-18T03:40:18.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f333a5ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33340b3350 con 0x7f33340b1e00 2023-07-18T03:40:18.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f333a5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33340b3630 con 0x7f33340d1220 2023-07-18T03:40:18.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f3338b9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f33340b1e00 0x7f33340afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:18.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f3333fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f333415c950 0x7f33340d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36066/0 (socket says 172.21.15.49:36066) 2023-07-18T03:40:18.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f3338b9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f33340b1e00 0x7f33340afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46588/0 (socket says 172.21.15.49:46588) 2023-07-18T03:40:18.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f3333fff700 1 -- 172.21.15.49:0/1289025727 learned_addr learned my addr 172.21.15.49:0/1289025727 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:18.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f33337fe700 1 --2- 172.21.15.49:0/1289025727 >> v2:172.21.15.107:3300/0 conn(0x7f33340d1220 0x7f33340b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:18.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f3338b9c700 1 -- 172.21.15.49:0/1289025727 >> v2:172.21.15.107:3300/0 conn(0x7f33340d1220 msgr2=0x7f33340b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:18.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f3338b9c700 1 --2- 172.21.15.49:0/1289025727 >> v2:172.21.15.107:3300/0 conn(0x7f33340d1220 0x7f33340b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f3338b9c700 1 -- 172.21.15.49:0/1289025727 >> v1:172.21.15.49:6789/0 conn(0x7f333415c950 legacy=0x7f33340d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.983+0000 7f3338b9c700 1 -- 172.21.15.49:0/1289025727 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33340b3910 con 0x7f33340b1e00 2023-07-18T03:40:18.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.982+0000 7f3332ffd700 1 -- 172.21.15.49:0/1289025727 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 538777240 0 0) 0x7f33340d2ce0 con 0x7f333415c950 2023-07-18T03:40:18.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.983+0000 7f3338b9c700 1 --2- 172.21.15.49:0/1289025727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33340b1e00 0x7f33340afac0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3334072140 tx=0x7f332c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c75bb42f8eec34 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:18.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.983+0000 7f3332ffd700 1 -- 172.21.15.49:0/1289025727 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c00e070 con 0x7f33340b1e00 2023-07-18T03:40:18.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.983+0000 7f3332ffd700 1 -- 172.21.15.49:0/1289025727 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c00bb20 con 0x7f33340b1e00 2023-07-18T03:40:18.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.983+0000 7f3332ffd700 1 -- 172.21.15.49:0/1289025727 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c012430 con 0x7f33340b1e00 2023-07-18T03:40:18.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.983+0000 7f333a5ff700 1 -- 172.21.15.49:0/1289025727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33340b1e00 msgr2=0x7f33340afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.983+0000 7f333a5ff700 1 --2- 172.21.15.49:0/1289025727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33340b1e00 0x7f33340afac0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3334072140 tx=0x7f332c00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:18.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.984+0000 7f333a5ff700 1 -- 172.21.15.49:0/1289025727 shutdown_connections 2023-07-18T03:40:18.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.984+0000 7f333a5ff700 1 --2- 172.21.15.49:0/1289025727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33340b1e00 0x7f33340afac0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.984+0000 7f333a5ff700 1 --2- 172.21.15.49:0/1289025727 >> v2:172.21.15.107:3300/0 conn(0x7f33340d1220 0x7f33340b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.984+0000 7f333a5ff700 1 -- 172.21.15.49:0/1289025727 >> 172.21.15.49:0/1289025727 conn(0x7f333409c1d0 msgr2=0x7f33340d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:18.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.984+0000 7f333a5ff700 1 -- 172.21.15.49:0/1289025727 shutdown_connections 2023-07-18T03:40:18.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.984+0000 7f333a5ff700 1 -- 172.21.15.49:0/1289025727 wait complete. 2023-07-18T03:40:18.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.986+0000 7f333a5ff700 1 Processor -- start 2023-07-18T03:40:18.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.986+0000 7f333a5ff700 1 -- start start 2023-07-18T03:40:18.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f333a5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33340b1e00 0x7f3334152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:18.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f333a5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33340d1220 0x7f33340b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:18.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f333a5ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f333415c950 0x7f3334152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:18.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f333a5ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340b7e40 con 0x7f333415c950 2023-07-18T03:40:18.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f333a5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3334152650 con 0x7f33340d1220 2023-07-18T03:40:18.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f333a5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33340ce360 con 0x7f33340b1e00 2023-07-18T03:40:18.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f3338b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f333415c950 0x7f3334152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:18.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f3338b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f333415c950 0x7f3334152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46596/0 (socket says 172.21.15.49:46596) 2023-07-18T03:40:18.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f3338b9c700 1 -- 172.21.15.49:0/1290699830 learned_addr learned my addr 172.21.15.49:0/1290699830 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:18.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f3333fff700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33340b1e00 0x7f3334152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:18.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f3338b9c700 1 -- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33340b1e00 msgr2=0x7f3334152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f33337fe700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33340d1220 0x7f33340b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:18.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f3338b9c700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33340b1e00 0x7f3334152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f3338b9c700 1 -- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33340d1220 msgr2=0x7f33340b82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:18.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f3338b9c700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33340d1220 0x7f33340b82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:18.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f3338b9c700 1 -- 172.21.15.49:0/1290699830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c009a70 con 0x7f333415c950 2023-07-18T03:40:18.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f3333fff700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33340b1e00 0x7f3334152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:18.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.988+0000 7f33337fe700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33340d1220 0x7f33340b82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:18.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.989+0000 7f3338b9c700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f333415c950 0x7f3334152110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f33340673b0 tx=0x7f332c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:18.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.989+0000 7f3330ff9700 1 -- 172.21.15.49:0/1290699830 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c012430 con 0x7f333415c950 2023-07-18T03:40:18.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.989+0000 7f3330ff9700 1 -- 172.21.15.49:0/1290699830 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c020070 con 0x7f333415c950 2023-07-18T03:40:18.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.989+0000 7f3330ff9700 1 -- 172.21.15.49:0/1290699830 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c004280 con 0x7f333415c950 2023-07-18T03:40:18.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.989+0000 7f333a5ff700 1 -- 172.21.15.49:0/1290699830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33340ce640 con 0x7f333415c950 2023-07-18T03:40:18.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.989+0000 7f333a5ff700 1 -- 172.21.15.49:0/1290699830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33340cd550 con 0x7f333415c950 2023-07-18T03:40:18.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.990+0000 7f333a5ff700 1 -- 172.21.15.49:0/1290699830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33340ce830 con 0x7f333415c950 2023-07-18T03:40:18.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.991+0000 7f3330ff9700 1 -- 172.21.15.49:0/1290699830 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f332c0043e0 con 0x7f333415c950 2023-07-18T03:40:18.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.992+0000 7f3330ff9700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f332006ad70 0x7f332006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:18.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.992+0000 7f3330ff9700 1 -- 172.21.15.49:0/1290699830 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f332c022070 con 0x7f333415c950 2023-07-18T03:40:18.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.992+0000 7f3333fff700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f332006ad70 0x7f332006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:18.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.993+0000 7f3333fff700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f332006ad70 0x7f332006d220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f33341f57c0 tx=0x7f3324009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:18.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:18.996+0000 7f3330ff9700 1 -- 172.21.15.49:0/1290699830 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f332c056320 con 0x7f333415c950 2023-07-18T03:40:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.209+0000 7f333a5ff700 1 -- 172.21.15.49:0/1290699830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f333404e280 con 0x7f333415c950 2023-07-18T03:40:19.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.210+0000 7f3330ff9700 1 -- 172.21.15.49:0/1290699830 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18257 (secure 0 0 0) 0x7f332c059940 con 0x7f333415c950 2023-07-18T03:40:19.206 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:19.206 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:58.343119+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:39:54.412334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-18T03:39:55.841218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:19.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.213+0000 7f333a5ff700 1 -- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f332006ad70 msgr2=0x7f332006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:19.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.213+0000 7f333a5ff700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f332006ad70 0x7f332006d220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f33341f57c0 tx=0x7f3324009040 comp rx=0 tx=0).stop 2023-07-18T03:40:19.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.213+0000 7f333a5ff700 1 -- 172.21.15.49:0/1290699830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f333415c950 msgr2=0x7f3334152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:19.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.213+0000 7f333a5ff700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f333415c950 0x7f3334152110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f33340673b0 tx=0x7f332c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.213+0000 7f333a5ff700 1 -- 172.21.15.49:0/1290699830 shutdown_connections 2023-07-18T03:40:19.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.213+0000 7f333a5ff700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f332006ad70 0x7f332006d220 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.213+0000 7f333a5ff700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f333415c950 0x7f3334152110 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.213+0000 7f333a5ff700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33340b1e00 0x7f3334152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.213+0000 7f333a5ff700 1 --2- 172.21.15.49:0/1290699830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33340d1220 0x7f33340b82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.213+0000 7f333a5ff700 1 -- 172.21.15.49:0/1290699830 >> 172.21.15.49:0/1290699830 conn(0x7f333409c1d0 msgr2=0x7f333406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:19.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.213+0000 7f333a5ff700 1 -- 172.21.15.49:0/1290699830 shutdown_connections 2023-07-18T03:40:19.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.213+0000 7f333a5ff700 1 -- 172.21.15.49:0/1290699830 wait complete. 2023-07-18T03:40:19.226 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:39:54.412334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:39:55.841218+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:40:19.227 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:40:19.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.439+0000 7f3f7e9ff700 1 Processor -- start 2023-07-18T03:40:19.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.439+0000 7f3f7e9ff700 1 -- start start 2023-07-18T03:40:19.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.440+0000 7f3f7e9ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3f7815d9d0 0x7f3f7806b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:19.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.440+0000 7f3f7e9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7806c9f0 0x7f3f780b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:19.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.440+0000 7f3f7e9ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f7806c4e0 con 0x7f3f780b1e50 2023-07-18T03:40:19.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.440+0000 7f3f7e9ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f7815dda0 con 0x7f3f7815d9d0 2023-07-18T03:40:19.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.440+0000 7f3f7e9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7806bf20 con 0x7f3f7806c9f0 2023-07-18T03:40:19.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.440+0000 7f3f77fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3f7815d9d0 0x7f3f7806b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:19.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.440+0000 7f3f7cf9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3f780b1e50 0x7f3f78071170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36068/0 (socket says 172.21.15.49:36068) 2023-07-18T03:40:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.440+0000 7f3f7cf9c700 1 -- 172.21.15.49:0/2373068604 learned_addr learned my addr 172.21.15.49:0/2373068604 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.440+0000 7f3f77fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3f7815d9d0 0x7f3f7806b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46602/0 (socket says 172.21.15.49:46602) 2023-07-18T03:40:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.441+0000 7f3f777fe700 1 --2- 172.21.15.49:0/2373068604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7806c9f0 0x7f3f780b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.441+0000 7f3f77fff700 1 -- 172.21.15.49:0/2373068604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7806c9f0 msgr2=0x7f3f780b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.441+0000 7f3f77fff700 1 --2- 172.21.15.49:0/2373068604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7806c9f0 0x7f3f780b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.441+0000 7f3f77fff700 1 -- 172.21.15.49:0/2373068604 >> v1:172.21.15.49:6789/0 conn(0x7f3f780b1e50 legacy=0x7f3f78071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.441+0000 7f3f77fff700 1 -- 172.21.15.49:0/2373068604 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7806c200 con 0x7f3f7815d9d0 2023-07-18T03:40:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.441+0000 7f3f76ffd700 1 -- 172.21.15.49:0/2373068604 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901292249 0 0) 0x7f3f7806c4e0 con 0x7f3f780b1e50 2023-07-18T03:40:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.441+0000 7f3f77fff700 1 --2- 172.21.15.49:0/2373068604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f7815d9d0 0x7f3f7806b5b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009840 tx=0x7f3f68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb782d312fe0324c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.441+0000 7f3f76ffd700 1 -- 172.21.15.49:0/2373068604 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68011830 con 0x7f3f7815d9d0 2023-07-18T03:40:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.441+0000 7f3f76ffd700 1 -- 172.21.15.49:0/2373068604 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f68011990 con 0x7f3f7815d9d0 2023-07-18T03:40:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.441+0000 7f3f76ffd700 1 -- 172.21.15.49:0/2373068604 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68011d20 con 0x7f3f7815d9d0 2023-07-18T03:40:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.442+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/2373068604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f7815d9d0 msgr2=0x7f3f7806b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.442+0000 7f3f7e9ff700 1 --2- 172.21.15.49:0/2373068604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f7815d9d0 0x7f3f7806b5b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009840 tx=0x7f3f68009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.442+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/2373068604 shutdown_connections 2023-07-18T03:40:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.442+0000 7f3f7e9ff700 1 --2- 172.21.15.49:0/2373068604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f7815d9d0 0x7f3f7806b5b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.442+0000 7f3f7e9ff700 1 --2- 172.21.15.49:0/2373068604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7806c9f0 0x7f3f780b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.442+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/2373068604 >> 172.21.15.49:0/2373068604 conn(0x7f3f780b05d0 msgr2=0x7f3f780b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:19.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.442+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/2373068604 shutdown_connections 2023-07-18T03:40:19.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.442+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/2373068604 wait complete. 2023-07-18T03:40:19.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.444+0000 7f3f7e9ff700 1 Processor -- start 2023-07-18T03:40:19.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.444+0000 7f3f7e9ff700 1 -- start start 2023-07-18T03:40:19.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.445+0000 7f3f7e9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7806c9f0 0x7f3f781e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:19.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.445+0000 7f3f7e9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f780b1e50 0x7f3f780c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:19.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.445+0000 7f3f7e9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f7815d9d0 0x7f3f780cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:19.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.445+0000 7f3f7e9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f780c3640 con 0x7f3f7815d9d0 2023-07-18T03:40:19.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.445+0000 7f3f7e9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f781f56a0 con 0x7f3f7806c9f0 2023-07-18T03:40:19.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.445+0000 7f3f7e9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f781f5980 con 0x7f3f780b1e50 2023-07-18T03:40:19.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.445+0000 7f3f7cf9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f7815d9d0 0x7f3f780cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:19.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.445+0000 7f3f7cf9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f7815d9d0 0x7f3f780cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46608/0 (socket says 172.21.15.49:46608) 2023-07-18T03:40:19.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.445+0000 7f3f7cf9c700 1 -- 172.21.15.49:0/1824334067 learned_addr learned my addr 172.21.15.49:0/1824334067 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:19.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.445+0000 7f3f777fe700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f780b1e50 0x7f3f780c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:19.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.445+0000 7f3f77fff700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7806c9f0 0x7f3f781e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:19.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.446+0000 7f3f7cf9c700 1 -- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f780b1e50 msgr2=0x7f3f780c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:19.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.446+0000 7f3f7cf9c700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f780b1e50 0x7f3f780c4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.446+0000 7f3f7cf9c700 1 -- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7806c9f0 msgr2=0x7f3f781e2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:19.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.446+0000 7f3f7cf9c700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7806c9f0 0x7f3f781e2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.446+0000 7f3f7cf9c700 1 -- 172.21.15.49:0/1824334067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f680094a0 con 0x7f3f7815d9d0 2023-07-18T03:40:19.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.446+0000 7f3f777fe700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f780b1e50 0x7f3f780c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:19.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.446+0000 7f3f7cf9c700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f7815d9d0 0x7f3f780cd8d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000cbe0 tx=0x7f3f7000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:19.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.446+0000 7f3f74ff9700 1 -- 172.21.15.49:0/1824334067 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7000ab50 con 0x7f3f7815d9d0 2023-07-18T03:40:19.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.446+0000 7f3f74ff9700 1 -- 172.21.15.49:0/1824334067 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7000acd0 con 0x7f3f7815d9d0 2023-07-18T03:40:19.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.446+0000 7f3f74ff9700 1 -- 172.21.15.49:0/1824334067 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70007310 con 0x7f3f7815d9d0 2023-07-18T03:40:19.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.446+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/1824334067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f781f5c60 con 0x7f3f7815d9d0 2023-07-18T03:40:19.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.446+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/1824334067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f780c53f0 con 0x7f3f7815d9d0 2023-07-18T03:40:19.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.448+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/1824334067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f780c3840 con 0x7f3f7815d9d0 2023-07-18T03:40:19.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.448+0000 7f3f74ff9700 1 -- 172.21.15.49:0/1824334067 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f3f70007470 con 0x7f3f7815d9d0 2023-07-18T03:40:19.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.449+0000 7f3f74ff9700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3f6406ad70 0x7f3f6406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:19.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.449+0000 7f3f74ff9700 1 -- 172.21.15.49:0/1824334067 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f70088f60 con 0x7f3f7815d9d0 2023-07-18T03:40:19.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.450+0000 7f3f77fff700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3f6406ad70 0x7f3f6406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:19.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.450+0000 7f3f77fff700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3f6406ad70 0x7f3f6406d220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009fd0 tx=0x7f3f6801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:19.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.454+0000 7f3f74ff9700 1 -- 172.21.15.49:0/1824334067 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3f700bb330 con 0x7f3f7815d9d0 2023-07-18T03:40:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.663+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/1824334067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3f78074350 con 0x7f3f7815d9d0 2023-07-18T03:40:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.663+0000 7f3f74ff9700 1 -- 172.21.15.49:0/1824334067 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3f700180a0 con 0x7f3f7815d9d0 2023-07-18T03:40:19.659 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:40:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.666+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3f6406ad70 msgr2=0x7f3f6406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.666+0000 7f3f7e9ff700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3f6406ad70 0x7f3f6406d220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009fd0 tx=0x7f3f6801c040 comp rx=0 tx=0).stop 2023-07-18T03:40:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.666+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/1824334067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f7815d9d0 msgr2=0x7f3f780cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.666+0000 7f3f7e9ff700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f7815d9d0 0x7f3f780cd8d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000cbe0 tx=0x7f3f7000cef0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.666+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/1824334067 shutdown_connections 2023-07-18T03:40:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.666+0000 7f3f7e9ff700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3f6406ad70 0x7f3f6406d220 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.666+0000 7f3f7e9ff700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f7815d9d0 0x7f3f780cd8d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.666+0000 7f3f7e9ff700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7806c9f0 0x7f3f781e2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.666+0000 7f3f7e9ff700 1 --2- 172.21.15.49:0/1824334067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f780b1e50 0x7f3f780c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.666+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/1824334067 >> 172.21.15.49:0/1824334067 conn(0x7f3f780b05d0 msgr2=0x7f3f78071170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.666+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/1824334067 shutdown_connections 2023-07-18T03:40:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.666+0000 7f3f7e9ff700 1 -- 172.21.15.49:0/1824334067 wait complete. 2023-07-18T03:40:19.681 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:40:19.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.894+0000 7f02ba1ff700 1 Processor -- start 2023-07-18T03:40:19.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.894+0000 7f02ba1ff700 1 -- start start 2023-07-18T03:40:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.895+0000 7f02ba1ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f02b40b8a60 0x7f02b40b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:19.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.895+0000 7f02ba1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f02b40d1220 0x7f02b40b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:19.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.895+0000 7f02ba1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b40b8e30 con 0x7f02b40b4530 2023-07-18T03:40:19.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.895+0000 7f02ba1ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b40afaa0 con 0x7f02b40b8a60 2023-07-18T03:40:19.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.895+0000 7f02ba1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b40afd80 con 0x7f02b40d1220 2023-07-18T03:40:19.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.895+0000 7f02b37fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f02b40b8a60 0x7f02b40b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:19.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.895+0000 7f02b37fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f02b40b8a60 0x7f02b40b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46612/0 (socket says 172.21.15.49:46612) 2023-07-18T03:40:19.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.895+0000 7f02b37fe700 1 -- 172.21.15.49:0/3103479848 learned_addr learned my addr 172.21.15.49:0/3103479848 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:19.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.896+0000 7f02b3fff700 1 --2- 172.21.15.49:0/3103479848 >> v2:172.21.15.107:3300/0 conn(0x7f02b40d1220 0x7f02b40b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:19.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.896+0000 7f02b37fe700 1 -- 172.21.15.49:0/3103479848 >> v2:172.21.15.107:3300/0 conn(0x7f02b40d1220 msgr2=0x7f02b40b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:19.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.896+0000 7f02b37fe700 1 --2- 172.21.15.49:0/3103479848 >> v2:172.21.15.107:3300/0 conn(0x7f02b40d1220 0x7f02b40b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.896+0000 7f02b37fe700 1 -- 172.21.15.49:0/3103479848 >> v1:172.21.15.107:6789/0 conn(0x7f02b40b4530 legacy=0x7f02b40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:19.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.896+0000 7f02b37fe700 1 -- 172.21.15.49:0/3103479848 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b40b0060 con 0x7f02b40b8a60 2023-07-18T03:40:19.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.896+0000 7f02b3fff700 1 --2- 172.21.15.49:0/3103479848 >> v2:172.21.15.107:3300/0 conn(0x7f02b40d1220 0x7f02b40b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:19.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.896+0000 7f02b37fe700 1 --2- 172.21.15.49:0/3103479848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02b40b8a60 0x7f02b40b3ff0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f02b40674e0 tx=0x7f02a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3fc0c9fe17d851 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:19.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.896+0000 7f02b27fc700 1 -- 172.21.15.49:0/3103479848 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a400b870 con 0x7f02b40b8a60 2023-07-18T03:40:19.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.897+0000 7f02b27fc700 1 -- 172.21.15.49:0/3103479848 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a400b9d0 con 0x7f02b40b8a60 2023-07-18T03:40:19.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.897+0000 7f02b27fc700 1 -- 172.21.15.49:0/3103479848 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a400bd60 con 0x7f02b40b8a60 2023-07-18T03:40:19.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.897+0000 7f02ba1ff700 1 -- 172.21.15.49:0/3103479848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02b40b8a60 msgr2=0x7f02b40b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:19.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.897+0000 7f02ba1ff700 1 --2- 172.21.15.49:0/3103479848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02b40b8a60 0x7f02b40b3ff0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f02b40674e0 tx=0x7f02a4009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.897+0000 7f02ba1ff700 1 -- 172.21.15.49:0/3103479848 shutdown_connections 2023-07-18T03:40:19.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.897+0000 7f02ba1ff700 1 --2- 172.21.15.49:0/3103479848 >> v2:172.21.15.107:3300/0 conn(0x7f02b40d1220 0x7f02b40b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.897+0000 7f02ba1ff700 1 --2- 172.21.15.49:0/3103479848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02b40b8a60 0x7f02b40b3ff0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.897+0000 7f02ba1ff700 1 -- 172.21.15.49:0/3103479848 >> 172.21.15.49:0/3103479848 conn(0x7f02b409c1b0 msgr2=0x7f02b40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:19.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.897+0000 7f02ba1ff700 1 -- 172.21.15.49:0/3103479848 shutdown_connections 2023-07-18T03:40:19.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.897+0000 7f02ba1ff700 1 -- 172.21.15.49:0/3103479848 wait complete. 2023-07-18T03:40:19.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.899+0000 7f02ba1ff700 1 Processor -- start 2023-07-18T03:40:19.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.899+0000 7f02ba1ff700 1 -- start start 2023-07-18T03:40:19.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.900+0000 7f02ba1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02b40b4530 0x7f02b41f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:19.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.900+0000 7f02ba1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02b40b8a60 0x7f02b41f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:19.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.900+0000 7f02ba1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02b40d1220 0x7f02b41f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:19.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.900+0000 7f02ba1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b41f3420 con 0x7f02b40b8a60 2023-07-18T03:40:19.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.900+0000 7f02ba1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b41f6990 con 0x7f02b40d1220 2023-07-18T03:40:19.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.900+0000 7f02ba1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b41f60f0 con 0x7f02b40b4530 2023-07-18T03:40:19.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.901+0000 7f02b2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02b40b8a60 0x7f02b41f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:19.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.901+0000 7f02b2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02b40b8a60 0x7f02b41f2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46614/0 (socket says 172.21.15.49:46614) 2023-07-18T03:40:19.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.901+0000 7f02b2ffd700 1 -- 172.21.15.49:0/1970916530 learned_addr learned my addr 172.21.15.49:0/1970916530 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:19.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.901+0000 7f02b37fe700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02b40b4530 0x7f02b41f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:19.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.901+0000 7f02b3fff700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02b40d1220 0x7f02b41f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:19.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.901+0000 7f02b2ffd700 1 -- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02b40b4530 msgr2=0x7f02b41f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:19.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.901+0000 7f02b2ffd700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02b40b4530 0x7f02b41f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.901+0000 7f02b2ffd700 1 -- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02b40d1220 msgr2=0x7f02b41f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.901+0000 7f02b2ffd700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02b40d1220 0x7f02b41f1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.901+0000 7f02b2ffd700 1 -- 172.21.15.49:0/1970916530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a4009550 con 0x7f02b40b8a60 2023-07-18T03:40:19.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.901+0000 7f02b2ffd700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02b40b8a60 0x7f02b41f2ee0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f02a800c4e0 tx=0x7f02a800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:19.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.902+0000 7f029bfff700 1 -- 172.21.15.49:0/1970916530 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a80052d0 con 0x7f02b40b8a60 2023-07-18T03:40:19.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.902+0000 7f029bfff700 1 -- 172.21.15.49:0/1970916530 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a8005430 con 0x7f02b40b8a60 2023-07-18T03:40:19.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.902+0000 7f029bfff700 1 -- 172.21.15.49:0/1970916530 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a80038c0 con 0x7f02b40b8a60 2023-07-18T03:40:19.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.902+0000 7f02ba1ff700 1 -- 172.21.15.49:0/1970916530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b41f63d0 con 0x7f02b40b8a60 2023-07-18T03:40:19.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.902+0000 7f02ba1ff700 1 -- 172.21.15.49:0/1970916530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b40b5f30 con 0x7f02b40b8a60 2023-07-18T03:40:19.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.903+0000 7f02ba1ff700 1 -- 172.21.15.49:0/1970916530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b40d0960 con 0x7f02b40b8a60 2023-07-18T03:40:19.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.906+0000 7f029bfff700 1 -- 172.21.15.49:0/1970916530 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f02a8003a20 con 0x7f02b40b8a60 2023-07-18T03:40:19.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.907+0000 7f029bfff700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f02a006ad70 0x7f02a006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:19.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.907+0000 7f029bfff700 1 -- 172.21.15.49:0/1970916530 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f02a8088d20 con 0x7f02b40b8a60 2023-07-18T03:40:19.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.907+0000 7f02b37fe700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f02a006ad70 0x7f02a006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:19.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.908+0000 7f02b37fe700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f02a006ad70 0x7f02a006d220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02a4000c00 tx=0x7f02a4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:19.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:19.910+0000 7f029bfff700 1 -- 172.21.15.49:0/1970916530 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f02a8053d60 con 0x7f02b40b8a60 2023-07-18T03:40:20.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.117+0000 7f02ba1ff700 1 -- 172.21.15.49:0/1970916530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02b40b5d90 con 0x7f02b40b8a60 2023-07-18T03:40:20.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.117+0000 7f029bfff700 1 -- 172.21.15.49:0/1970916530 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f02b40b5d90 con 0x7f02b40b8a60 2023-07-18T03:40:20.113 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:40:20.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.119+0000 7f02ba1ff700 1 -- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f02a006ad70 msgr2=0x7f02a006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:20.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.119+0000 7f02ba1ff700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f02a006ad70 0x7f02a006d220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02a4000c00 tx=0x7f02a4008030 comp rx=0 tx=0).stop 2023-07-18T03:40:20.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.119+0000 7f02ba1ff700 1 -- 172.21.15.49:0/1970916530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02b40b8a60 msgr2=0x7f02b41f2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:20.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.119+0000 7f02ba1ff700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02b40b8a60 0x7f02b41f2ee0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f02a800c4e0 tx=0x7f02a800c7f0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.120+0000 7f02ba1ff700 1 -- 172.21.15.49:0/1970916530 shutdown_connections 2023-07-18T03:40:20.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.120+0000 7f02ba1ff700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02b40d1220 0x7f02b41f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.120+0000 7f02ba1ff700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02b40b4530 0x7f02b41f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.120+0000 7f02ba1ff700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f02a006ad70 0x7f02a006d220 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.120+0000 7f02ba1ff700 1 --2- 172.21.15.49:0/1970916530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02b40b8a60 0x7f02b41f2ee0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.120+0000 7f02ba1ff700 1 -- 172.21.15.49:0/1970916530 >> 172.21.15.49:0/1970916530 conn(0x7f02b409c1b0 msgr2=0x7f02b406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:20.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.120+0000 7f02ba1ff700 1 -- 172.21.15.49:0/1970916530 shutdown_connections 2023-07-18T03:40:20.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.120+0000 7f02ba1ff700 1 -- 172.21.15.49:0/1970916530 wait complete. 2023-07-18T03:40:20.134 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:40:20.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.349+0000 7f1c4bbff700 1 Processor -- start 2023-07-18T03:40:20.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.349+0000 7f1c4bbff700 1 -- start start 2023-07-18T03:40:20.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.350+0000 7f1c4bbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c440d1e40 0x7f1c440cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:20.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.350+0000 7f1c4bbff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1c440c71b0 0x7f1c440b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:20.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.350+0000 7f1c4bbff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c440cd5a0 con 0x7f1c4415c860 2023-07-18T03:40:20.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.350+0000 7f1c4bbff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c440d22b0 con 0x7f1c440c71b0 2023-07-18T03:40:20.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.350+0000 7f1c4bbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c44146740 con 0x7f1c440d1e40 2023-07-18T03:40:20.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.351+0000 7f1c4999b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1c4415c860 0x7f1c440c6aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36078/0 (socket says 172.21.15.49:36078) 2023-07-18T03:40:20.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.351+0000 7f1c4999b700 1 -- 172.21.15.49:0/2139413403 learned_addr learned my addr 172.21.15.49:0/2139413403 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:20.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.351+0000 7f1c4919a700 1 --2- 172.21.15.49:0/2139413403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c440d1e40 0x7f1c440cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:20.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.351+0000 7f1c4a19c700 1 --2- 172.21.15.49:0/2139413403 >> v2:172.21.15.107:3300/0 conn(0x7f1c440c71b0 0x7f1c440b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:20.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.351+0000 7f1c48999700 1 -- 172.21.15.49:0/2139413403 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908984483 0 0) 0x7f1c440cd5a0 con 0x7f1c4415c860 2023-07-18T03:40:20.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.351+0000 7f1c48999700 1 -- 172.21.15.49:0/2139413403 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c300035b0 con 0x7f1c4415c860 2023-07-18T03:40:20.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.352+0000 7f1c4a19c700 1 -- 172.21.15.49:0/2139413403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c440d1e40 msgr2=0x7f1c440cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:20.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.352+0000 7f1c4a19c700 1 --2- 172.21.15.49:0/2139413403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c440d1e40 0x7f1c440cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.352+0000 7f1c4a19c700 1 -- 172.21.15.49:0/2139413403 >> v1:172.21.15.49:6789/0 conn(0x7f1c4415c860 legacy=0x7f1c440c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.352+0000 7f1c4a19c700 1 -- 172.21.15.49:0/2139413403 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c440cd7f0 con 0x7f1c440c71b0 2023-07-18T03:40:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.352+0000 7f1c4a19c700 1 --2- 172.21.15.49:0/2139413403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c440c71b0 0x7f1c440b49d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009d90 tx=0x7f1c3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcbea89c1f675ff server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.352+0000 7f1c48999700 1 -- 172.21.15.49:0/2139413403 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c00e070 con 0x7f1c440c71b0 2023-07-18T03:40:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.352+0000 7f1c48999700 1 -- 172.21.15.49:0/2139413403 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c00bb20 con 0x7f1c440c71b0 2023-07-18T03:40:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.352+0000 7f1c48999700 1 -- 172.21.15.49:0/2139413403 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c00be90 con 0x7f1c440c71b0 2023-07-18T03:40:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.353+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2139413403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c440c71b0 msgr2=0x7f1c440b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.353+0000 7f1c4bbff700 1 --2- 172.21.15.49:0/2139413403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c440c71b0 0x7f1c440b49d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009d90 tx=0x7f1c3c00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.353+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2139413403 shutdown_connections 2023-07-18T03:40:20.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.353+0000 7f1c4bbff700 1 --2- 172.21.15.49:0/2139413403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c440c71b0 0x7f1c440b49d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.353+0000 7f1c4bbff700 1 --2- 172.21.15.49:0/2139413403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c440d1e40 0x7f1c440cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.353+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2139413403 >> 172.21.15.49:0/2139413403 conn(0x7f1c44065bd0 msgr2=0x7f1c44065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:20.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.353+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2139413403 shutdown_connections 2023-07-18T03:40:20.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.353+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2139413403 wait complete. 2023-07-18T03:40:20.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.355+0000 7f1c4bbff700 1 Processor -- start 2023-07-18T03:40:20.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.355+0000 7f1c4bbff700 1 -- start start 2023-07-18T03:40:20.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.356+0000 7f1c4bbff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c440c71b0 0x7f1c441f5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:20.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.356+0000 7f1c4bbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c440d1e40 0x7f1c440b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:20.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.356+0000 7f1c4bbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c4415c860 0x7f1c440cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:20.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.356+0000 7f1c4bbff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c440cb9b0 con 0x7f1c440c71b0 2023-07-18T03:40:20.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.356+0000 7f1c4bbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c440cada0 con 0x7f1c440d1e40 2023-07-18T03:40:20.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.356+0000 7f1c4bbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c440cb080 con 0x7f1c4415c860 2023-07-18T03:40:20.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.356+0000 7f1c4999b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c440c71b0 0x7f1c441f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:20.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.356+0000 7f1c4999b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c440c71b0 0x7f1c441f5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46626/0 (socket says 172.21.15.49:46626) 2023-07-18T03:40:20.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.356+0000 7f1c4999b700 1 -- 172.21.15.49:0/2581864713 learned_addr learned my addr 172.21.15.49:0/2581864713 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:20.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.356+0000 7f1c4a19c700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c4415c860 0x7f1c440cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:20.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.356+0000 7f1c4919a700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c440d1e40 0x7f1c440b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:20.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.357+0000 7f1c4999b700 1 -- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c4415c860 msgr2=0x7f1c440cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:20.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.357+0000 7f1c4999b700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c4415c860 0x7f1c440cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.357+0000 7f1c4999b700 1 -- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c440d1e40 msgr2=0x7f1c440b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:20.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.357+0000 7f1c4999b700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c440d1e40 0x7f1c440b82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.357+0000 7f1c4999b700 1 -- 172.21.15.49:0/2581864713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c009a70 con 0x7f1c440c71b0 2023-07-18T03:40:20.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.357+0000 7f1c4999b700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c440c71b0 0x7f1c441f5210 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000c420 tx=0x7f1c4000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:20.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.357+0000 7f1c3a7fc700 1 -- 172.21.15.49:0/2581864713 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c40005d70 con 0x7f1c440c71b0 2023-07-18T03:40:20.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.357+0000 7f1c3a7fc700 1 -- 172.21.15.49:0/2581864713 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4000ce30 con 0x7f1c440c71b0 2023-07-18T03:40:20.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.357+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2581864713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c440cb360 con 0x7f1c440c71b0 2023-07-18T03:40:20.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.357+0000 7f1c3a7fc700 1 -- 172.21.15.49:0/2581864713 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c400072d0 con 0x7f1c440c71b0 2023-07-18T03:40:20.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.358+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2581864713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c440caa30 con 0x7f1c440c71b0 2023-07-18T03:40:20.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.358+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2581864713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c44074350 con 0x7f1c440c71b0 2023-07-18T03:40:20.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.359+0000 7f1c3a7fc700 1 -- 172.21.15.49:0/2581864713 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f1c40007430 con 0x7f1c440c71b0 2023-07-18T03:40:20.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.360+0000 7f1c3a7fc700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1c2806ad70 0x7f1c2806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:20.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.360+0000 7f1c3a7fc700 1 -- 172.21.15.49:0/2581864713 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1c400887b0 con 0x7f1c440c71b0 2023-07-18T03:40:20.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.361+0000 7f1c4919a700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1c2806ad70 0x7f1c2806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:20.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.362+0000 7f1c4919a700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1c2806ad70 0x7f1c2806d220 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c4406ba10 tx=0x7f1c34009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:20.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.364+0000 7f1c3a7fc700 1 -- 172.21.15.49:0/2581864713 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1c40053820 con 0x7f1c440c71b0 2023-07-18T03:40:20.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.574+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2581864713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c440c6f90 con 0x7f1c440c71b0 2023-07-18T03:40:20.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.575+0000 7f1c3a7fc700 1 -- 172.21.15.49:0/2581864713 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c40056e40 con 0x7f1c440c71b0 2023-07-18T03:40:20.571 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:40:20.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.577+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1c2806ad70 msgr2=0x7f1c2806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:20.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.577+0000 7f1c4bbff700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1c2806ad70 0x7f1c2806d220 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c4406ba10 tx=0x7f1c34009210 comp rx=0 tx=0).stop 2023-07-18T03:40:20.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.577+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2581864713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c440c71b0 msgr2=0x7f1c441f5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:20.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.577+0000 7f1c4bbff700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c440c71b0 0x7f1c441f5210 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000c420 tx=0x7f1c4000c730 comp rx=0 tx=0).stop 2023-07-18T03:40:20.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.577+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2581864713 shutdown_connections 2023-07-18T03:40:20.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.577+0000 7f1c4bbff700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1c4415c860 0x7f1c440cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.577+0000 7f1c4bbff700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1c2806ad70 0x7f1c2806d220 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.577+0000 7f1c4bbff700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c440c71b0 0x7f1c441f5210 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.577+0000 7f1c4bbff700 1 --2- 172.21.15.49:0/2581864713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c440d1e40 0x7f1c440b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.577+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2581864713 >> 172.21.15.49:0/2581864713 conn(0x7f1c44065bd0 msgr2=0x7f1c440c91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:20.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.577+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2581864713 shutdown_connections 2023-07-18T03:40:20.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.577+0000 7f1c4bbff700 1 -- 172.21.15.49:0/2581864713 wait complete. 2023-07-18T03:40:20.591 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:40:20.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.805+0000 7f1e417ff700 1 Processor -- start 2023-07-18T03:40:20.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.805+0000 7f1e417ff700 1 -- start start 2023-07-18T03:40:20.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.806+0000 7f1e417ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1e3c0b8a60 0x7f1e3c0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:20.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.806+0000 7f1e417ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e3c0b4530 0x7f1e3c0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:20.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.806+0000 7f1e417ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3c0d1490 con 0x7f1e3c0d1050 2023-07-18T03:40:20.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.806+0000 7f1e417ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c0b8e30 con 0x7f1e3c0b4530 2023-07-18T03:40:20.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.806+0000 7f1e417ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c0b1e30 con 0x7f1e3c0b8a60 2023-07-18T03:40:20.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.806+0000 7f1e33fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e3c0b4530 0x7f1e3c0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:20.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.806+0000 7f1e33fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e3c0b4530 0x7f1e3c0d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46632/0 (socket says 172.21.15.49:46632) 2023-07-18T03:40:20.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.806+0000 7f1e3b7fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1e3c0d1050 0x7f1e3c0b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36086/0 (socket says 172.21.15.49:36086) 2023-07-18T03:40:20.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.806+0000 7f1e33fff700 1 -- 172.21.15.49:0/1440791277 learned_addr learned my addr 172.21.15.49:0/1440791277 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:20.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.807+0000 7f1e3affd700 1 --2- 172.21.15.49:0/1440791277 >> v2:172.21.15.107:3300/0 conn(0x7f1e3c0b8a60 0x7f1e3c0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:20.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.807+0000 7f1e33fff700 1 -- 172.21.15.49:0/1440791277 >> v2:172.21.15.107:3300/0 conn(0x7f1e3c0b8a60 msgr2=0x7f1e3c0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:20.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.807+0000 7f1e33fff700 1 --2- 172.21.15.49:0/1440791277 >> v2:172.21.15.107:3300/0 conn(0x7f1e3c0b8a60 0x7f1e3c0b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.807+0000 7f1e33fff700 1 -- 172.21.15.49:0/1440791277 >> v1:172.21.15.49:6789/0 conn(0x7f1e3c0d1050 legacy=0x7f1e3c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:20.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.807+0000 7f1e33fff700 1 -- 172.21.15.49:0/1440791277 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c0afaa0 con 0x7f1e3c0b4530 2023-07-18T03:40:20.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.807+0000 7f1e33fff700 1 --2- 172.21.15.49:0/1440791277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c0b4530 0x7f1e3c0d0b10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0b5b00 tx=0x7f1e24009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff25cc5c18a92d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:20.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.807+0000 7f1e3a7fc700 1 -- 172.21.15.49:0/1440791277 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400b870 con 0x7f1e3c0b4530 2023-07-18T03:40:20.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.807+0000 7f1e3a7fc700 1 -- 172.21.15.49:0/1440791277 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2400b9d0 con 0x7f1e3c0b4530 2023-07-18T03:40:20.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.807+0000 7f1e3a7fc700 1 -- 172.21.15.49:0/1440791277 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400bd60 con 0x7f1e3c0b4530 2023-07-18T03:40:20.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.808+0000 7f1e417ff700 1 -- 172.21.15.49:0/1440791277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c0b4530 msgr2=0x7f1e3c0d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:20.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.808+0000 7f1e417ff700 1 --2- 172.21.15.49:0/1440791277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c0b4530 0x7f1e3c0d0b10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0b5b00 tx=0x7f1e24009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.808+0000 7f1e417ff700 1 -- 172.21.15.49:0/1440791277 shutdown_connections 2023-07-18T03:40:20.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.808+0000 7f1e417ff700 1 --2- 172.21.15.49:0/1440791277 >> v2:172.21.15.107:3300/0 conn(0x7f1e3c0b8a60 0x7f1e3c0b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.808+0000 7f1e417ff700 1 --2- 172.21.15.49:0/1440791277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c0b4530 0x7f1e3c0d0b10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.808+0000 7f1e417ff700 1 -- 172.21.15.49:0/1440791277 >> 172.21.15.49:0/1440791277 conn(0x7f1e3c09c1b0 msgr2=0x7f1e3c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:20.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.808+0000 7f1e417ff700 1 -- 172.21.15.49:0/1440791277 shutdown_connections 2023-07-18T03:40:20.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.808+0000 7f1e417ff700 1 -- 172.21.15.49:0/1440791277 wait complete. 2023-07-18T03:40:20.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.810+0000 7f1e417ff700 1 Processor -- start 2023-07-18T03:40:20.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.810+0000 7f1e417ff700 1 -- start start 2023-07-18T03:40:20.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.811+0000 7f1e417ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0b4530 0x7f1e3c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:20.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.811+0000 7f1e417ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e3c0b8a60 0x7f1e3c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:20.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.811+0000 7f1e417ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c0d1050 0x7f1e3c0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:20.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.811+0000 7f1e417ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c15f590 con 0x7f1e3c0d1050 2023-07-18T03:40:20.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.811+0000 7f1e417ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c15fac0 con 0x7f1e3c0b4530 2023-07-18T03:40:20.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.811+0000 7f1e417ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c15f7e0 con 0x7f1e3c0b8a60 2023-07-18T03:40:20.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.811+0000 7f1e3b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c0d1050 0x7f1e3c0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:20.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.811+0000 7f1e3b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c0d1050 0x7f1e3c0cfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46642/0 (socket says 172.21.15.49:46642) 2023-07-18T03:40:20.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.811+0000 7f1e3b7fe700 1 -- 172.21.15.49:0/586113716 learned_addr learned my addr 172.21.15.49:0/586113716 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:20.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.812+0000 7f1e33fff700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e3c0b8a60 0x7f1e3c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:20.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.812+0000 7f1e3affd700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0b4530 0x7f1e3c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:20.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.812+0000 7f1e3b7fe700 1 -- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e3c0b8a60 msgr2=0x7f1e3c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:20.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.812+0000 7f1e3b7fe700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e3c0b8a60 0x7f1e3c0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.812+0000 7f1e3b7fe700 1 -- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0b4530 msgr2=0x7f1e3c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:20.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.812+0000 7f1e3b7fe700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0b4530 0x7f1e3c0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:20.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.812+0000 7f1e3b7fe700 1 -- 172.21.15.49:0/586113716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e24009550 con 0x7f1e3c0d1050 2023-07-18T03:40:20.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.812+0000 7f1e3b7fe700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c0d1050 0x7f1e3c0cfc60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400cbf0 tx=0x7f1e3400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:20.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.812+0000 7f1e337fe700 1 -- 172.21.15.49:0/586113716 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3400aa60 con 0x7f1e3c0d1050 2023-07-18T03:40:20.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.812+0000 7f1e337fe700 1 -- 172.21.15.49:0/586113716 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3400abc0 con 0x7f1e3c0d1050 2023-07-18T03:40:20.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.812+0000 7f1e417ff700 1 -- 172.21.15.49:0/586113716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3c1f4000 con 0x7f1e3c0d1050 2023-07-18T03:40:20.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.812+0000 7f1e337fe700 1 -- 172.21.15.49:0/586113716 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34007070 con 0x7f1e3c0d1050 2023-07-18T03:40:20.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.813+0000 7f1e417ff700 1 -- 172.21.15.49:0/586113716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3c0ca550 con 0x7f1e3c0d1050 2023-07-18T03:40:20.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.813+0000 7f1e417ff700 1 -- 172.21.15.49:0/586113716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3c074350 con 0x7f1e3c0d1050 2023-07-18T03:40:20.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.814+0000 7f1e337fe700 1 -- 172.21.15.49:0/586113716 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f1e340071d0 con 0x7f1e3c0d1050 2023-07-18T03:40:20.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.815+0000 7f1e337fe700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e1c06ad70 0x7f1e1c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:20.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.815+0000 7f1e337fe700 1 -- 172.21.15.49:0/586113716 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1e34088ff0 con 0x7f1e3c0d1050 2023-07-18T03:40:20.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.816+0000 7f1e3affd700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e1c06ad70 0x7f1e1c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:20.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.816+0000 7f1e3affd700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e1c06ad70 0x7f1e1c06d220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c15eb90 tx=0x7f1e2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:20.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:20.820+0000 7f1e337fe700 1 -- 172.21.15.49:0/586113716 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1e340bb330 con 0x7f1e3c0d1050 2023-07-18T03:40:21.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.026+0000 7f1e417ff700 1 -- 172.21.15.49:0/586113716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e3c0b4310 con 0x7f1e3c0d1050 2023-07-18T03:40:21.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.026+0000 7f1e337fe700 1 -- 172.21.15.49:0/586113716 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e34017020 con 0x7f1e3c0d1050 2023-07-18T03:40:21.022 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:40:21.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.029+0000 7f1e417ff700 1 -- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e1c06ad70 msgr2=0x7f1e1c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:21.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.029+0000 7f1e417ff700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e1c06ad70 0x7f1e1c06d220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c15eb90 tx=0x7f1e2c008040 comp rx=0 tx=0).stop 2023-07-18T03:40:21.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.029+0000 7f1e417ff700 1 -- 172.21.15.49:0/586113716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c0d1050 msgr2=0x7f1e3c0cfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:21.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.029+0000 7f1e417ff700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c0d1050 0x7f1e3c0cfc60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400cbf0 tx=0x7f1e3400cf00 comp rx=0 tx=0).stop 2023-07-18T03:40:21.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.029+0000 7f1e417ff700 1 -- 172.21.15.49:0/586113716 shutdown_connections 2023-07-18T03:40:21.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.029+0000 7f1e417ff700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e1c06ad70 0x7f1e1c06d220 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.029+0000 7f1e417ff700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c0d1050 0x7f1e3c0cfc60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.029+0000 7f1e417ff700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0b4530 0x7f1e3c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.029+0000 7f1e417ff700 1 --2- 172.21.15.49:0/586113716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e3c0b8a60 0x7f1e3c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.029+0000 7f1e417ff700 1 -- 172.21.15.49:0/586113716 >> 172.21.15.49:0/586113716 conn(0x7f1e3c09c1b0 msgr2=0x7f1e3c0b7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:21.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.029+0000 7f1e417ff700 1 -- 172.21.15.49:0/586113716 shutdown_connections 2023-07-18T03:40:21.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.029+0000 7f1e417ff700 1 -- 172.21.15.49:0/586113716 wait complete. 2023-07-18T03:40:21.043 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-18T03:40:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.255+0000 7f0567dff700 1 Processor -- start 2023-07-18T03:40:21.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.255+0000 7f0567dff700 1 -- start start 2023-07-18T03:40:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.256+0000 7f0567dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f056015d9d0 0x7f056006b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:21.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.256+0000 7f0567dff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0560071170 0x7f05600715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:21.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.256+0000 7f0567dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0560071b30 con 0x7f056006c9f0 2023-07-18T03:40:21.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.256+0000 7f0567dff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056006c4e0 con 0x7f0560071170 2023-07-18T03:40:21.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.256+0000 7f0567dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05600b2020 con 0x7f056015d9d0 2023-07-18T03:40:21.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.256+0000 7f056639c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0560071170 0x7f05600715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:21.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.256+0000 7f056539a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f056006c9f0 0x7f05600b1910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36094/0 (socket says 172.21.15.49:36094) 2023-07-18T03:40:21.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.256+0000 7f056539a700 1 -- 172.21.15.49:0/351080426 learned_addr learned my addr 172.21.15.49:0/351080426 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:21.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.256+0000 7f0565b9b700 1 --2- 172.21.15.49:0/351080426 >> v2:172.21.15.107:3300/0 conn(0x7f056015d9d0 0x7f056006b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:21.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.257+0000 7f056639c700 1 -- 172.21.15.49:0/351080426 >> v2:172.21.15.107:3300/0 conn(0x7f056015d9d0 msgr2=0x7f056006b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:21.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.257+0000 7f056639c700 1 --2- 172.21.15.49:0/351080426 >> v2:172.21.15.107:3300/0 conn(0x7f056015d9d0 0x7f056006b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.257+0000 7f056639c700 1 -- 172.21.15.49:0/351080426 >> v1:172.21.15.49:6789/0 conn(0x7f056006c9f0 legacy=0x7f05600b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.257+0000 7f0564b99700 1 -- 172.21.15.49:0/351080426 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1545840649 0 0) 0x7f0560071b30 con 0x7f056006c9f0 2023-07-18T03:40:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.257+0000 7f056639c700 1 -- 172.21.15.49:0/351080426 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056015dda0 con 0x7f0560071170 2023-07-18T03:40:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.257+0000 7f0565b9b700 1 --2- 172.21.15.49:0/351080426 >> v2:172.21.15.107:3300/0 conn(0x7f056015d9d0 0x7f056006b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:21.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.257+0000 7f056639c700 1 --2- 172.21.15.49:0/351080426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0560071170 0x7f05600715f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f056006dcc0 tx=0x7f055800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fdc76e506bf535 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.257+0000 7f0564b99700 1 -- 172.21.15.49:0/351080426 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055800e070 con 0x7f0560071170 2023-07-18T03:40:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.257+0000 7f0564b99700 1 -- 172.21.15.49:0/351080426 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055800bbd0 con 0x7f0560071170 2023-07-18T03:40:21.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.257+0000 7f0564b99700 1 -- 172.21.15.49:0/351080426 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558012430 con 0x7f0560071170 2023-07-18T03:40:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.258+0000 7f0567dff700 1 -- 172.21.15.49:0/351080426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0560071170 msgr2=0x7f05600715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.258+0000 7f0567dff700 1 --2- 172.21.15.49:0/351080426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0560071170 0x7f05600715f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f056006dcc0 tx=0x7f055800b330 comp rx=0 tx=0).stop 2023-07-18T03:40:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.258+0000 7f0567dff700 1 -- 172.21.15.49:0/351080426 shutdown_connections 2023-07-18T03:40:21.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.258+0000 7f0567dff700 1 --2- 172.21.15.49:0/351080426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0560071170 0x7f05600715f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.258+0000 7f0567dff700 1 --2- 172.21.15.49:0/351080426 >> v2:172.21.15.107:3300/0 conn(0x7f056015d9d0 0x7f056006b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.258+0000 7f0567dff700 1 -- 172.21.15.49:0/351080426 >> 172.21.15.49:0/351080426 conn(0x7f05600b05d0 msgr2=0x7f05600b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.258+0000 7f0567dff700 1 -- 172.21.15.49:0/351080426 shutdown_connections 2023-07-18T03:40:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.258+0000 7f0567dff700 1 -- 172.21.15.49:0/351080426 wait complete. 2023-07-18T03:40:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.260+0000 7f0567dff700 1 Processor -- start 2023-07-18T03:40:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.260+0000 7f0567dff700 1 -- start start 2023-07-18T03:40:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.261+0000 7f0567dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f056006c9f0 0x7f05601521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:21.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.261+0000 7f0567dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0560071170 0x7f05600ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.261+0000 7f0567dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f056015d9d0 0x7f05600acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.261+0000 7f0567dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05601f5950 con 0x7f0560071170 2023-07-18T03:40:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.261+0000 7f0567dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05601f5bc0 con 0x7f056015d9d0 2023-07-18T03:40:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.261+0000 7f0567dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05601f5ea0 con 0x7f056006c9f0 2023-07-18T03:40:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.261+0000 7f056539a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0560071170 0x7f05600ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.261+0000 7f056539a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0560071170 0x7f05600ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46664/0 (socket says 172.21.15.49:46664) 2023-07-18T03:40:21.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.261+0000 7f056539a700 1 -- 172.21.15.49:0/1392630696 learned_addr learned my addr 172.21.15.49:0/1392630696 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.261+0000 7f056639c700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f056015d9d0 0x7f05600acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.261+0000 7f0565b9b700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f056006c9f0 0x7f05601521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.262+0000 7f056539a700 1 -- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f056006c9f0 msgr2=0x7f05601521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.262+0000 7f056539a700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f056006c9f0 0x7f05601521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.262+0000 7f056539a700 1 -- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f056015d9d0 msgr2=0x7f05600acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.262+0000 7f056539a700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f056015d9d0 0x7f05600acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.262+0000 7f056539a700 1 -- 172.21.15.49:0/1392630696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0558009a70 con 0x7f0560071170 2023-07-18T03:40:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.262+0000 7f056539a700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0560071170 0x7f05600ac860 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f055000c540 tx=0x7f055000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.262+0000 7f05567fc700 1 -- 172.21.15.49:0/1392630696 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550004c80 con 0x7f0560071170 2023-07-18T03:40:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.262+0000 7f05567fc700 1 -- 172.21.15.49:0/1392630696 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0550004de0 con 0x7f0560071170 2023-07-18T03:40:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.262+0000 7f05567fc700 1 -- 172.21.15.49:0/1392630696 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550007310 con 0x7f0560071170 2023-07-18T03:40:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.262+0000 7f0567dff700 1 -- 172.21.15.49:0/1392630696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05601f6180 con 0x7f0560071170 2023-07-18T03:40:21.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.262+0000 7f0567dff700 1 -- 172.21.15.49:0/1392630696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05601526f0 con 0x7f0560071170 2023-07-18T03:40:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.263+0000 7f0567dff700 1 -- 172.21.15.49:0/1392630696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05601f66a0 con 0x7f0560071170 2023-07-18T03:40:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.264+0000 7f05567fc700 1 -- 172.21.15.49:0/1392630696 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f0550007470 con 0x7f0560071170 2023-07-18T03:40:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.265+0000 7f05567fc700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f054406ad70 0x7f054406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:21.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.265+0000 7f05567fc700 1 -- 172.21.15.49:0/1392630696 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f05500891b0 con 0x7f0560071170 2023-07-18T03:40:21.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.266+0000 7f0565b9b700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f054406ad70 0x7f054406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:21.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.266+0000 7f0565b9b700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f054406ad70 0x7f054406d220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f055c00c940 tx=0x7f055c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.270+0000 7f05567fc700 1 -- 172.21.15.49:0/1392630696 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f05601f66a0 con 0x7f0560071170 2023-07-18T03:40:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.501+0000 7f0567dff700 1 -- 172.21.15.49:0/1392630696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f05601f66a0 con 0x7f0560071170 2023-07-18T03:40:21.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.501+0000 7f05567fc700 1 -- 172.21.15.49:0/1392630696 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f05601f66a0 con 0x7f0560071170 2023-07-18T03:40:21.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.503+0000 7f0567dff700 1 -- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f054406ad70 msgr2=0x7f054406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:21.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.503+0000 7f0567dff700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f054406ad70 0x7f054406d220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f055c00c940 tx=0x7f055c008040 comp rx=0 tx=0).stop 2023-07-18T03:40:21.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.504+0000 7f0567dff700 1 -- 172.21.15.49:0/1392630696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0560071170 msgr2=0x7f05600ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:21.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.504+0000 7f0567dff700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0560071170 0x7f05600ac860 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f055000c540 tx=0x7f055000c850 comp rx=0 tx=0).stop 2023-07-18T03:40:21.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.504+0000 7f0567dff700 1 -- 172.21.15.49:0/1392630696 shutdown_connections 2023-07-18T03:40:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.504+0000 7f0567dff700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f056015d9d0 0x7f05600acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.504+0000 7f0567dff700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f056006c9f0 0x7f05601521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.504+0000 7f0567dff700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f054406ad70 0x7f054406d220 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.504+0000 7f0567dff700 1 --2- 172.21.15.49:0/1392630696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0560071170 0x7f05600ac860 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.504+0000 7f0567dff700 1 -- 172.21.15.49:0/1392630696 >> 172.21.15.49:0/1392630696 conn(0x7f05600b05d0 msgr2=0x7f05600cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.504+0000 7f0567dff700 1 -- 172.21.15.49:0/1392630696 shutdown_connections 2023-07-18T03:40:21.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.504+0000 7f0567dff700 1 -- 172.21.15.49:0/1392630696 wait complete. 2023-07-18T03:40:21.501 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-07-18T03:40:21.517 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-07-18T03:40:21.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:21.518 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-07-18T03:40:21.622 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:40:21.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:21.623 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-18T03:40:21.668 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:40:21.669 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:21.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345544k,nr_inodes=4086386,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-18T03:40:21.713 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:40:21.714 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:21.757 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345444k,nr_inodes=4086361,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-18T03:40:21.757 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:40:21.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.967+0000 7f90139ff700 1 Processor -- start 2023-07-18T03:40:21.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.967+0000 7f90139ff700 1 -- start start 2023-07-18T03:40:21.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.968+0000 7f90139ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f900c0d1220 0x7f900c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:21.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.969+0000 7f90139ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f900c0b1e00 0x7f900c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:21.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.969+0000 7f90139ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900c0d2ce0 con 0x7f900c15c950 2023-07-18T03:40:21.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.969+0000 7f90139ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c0b3350 con 0x7f900c0b1e00 2023-07-18T03:40:21.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.969+0000 7f90139ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0b3630 con 0x7f900c0d1220 2023-07-18T03:40:21.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.969+0000 7f9011f9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f900c0b1e00 0x7f900c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:21.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.969+0000 7f901179b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f900c15c950 0x7f900c0d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:38744/0 (socket says 172.21.15.49:38744) 2023-07-18T03:40:21.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.969+0000 7f901179b700 1 -- 172.21.15.49:0/3294103899 learned_addr learned my addr 172.21.15.49:0/3294103899 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:21.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.969+0000 7f9010f9a700 1 --2- 172.21.15.49:0/3294103899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f900c0d1220 0x7f900c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.969+0000 7f9010f9a700 1 -- 172.21.15.49:0/3294103899 >> v2:172.21.15.49:3300/0 conn(0x7f900c0b1e00 msgr2=0x7f900c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.969+0000 7f9010f9a700 1 --2- 172.21.15.49:0/3294103899 >> v2:172.21.15.49:3300/0 conn(0x7f900c0b1e00 0x7f900c0afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.969+0000 7f9010f9a700 1 -- 172.21.15.49:0/3294103899 >> v1:172.21.15.107:6789/0 conn(0x7f900c15c950 legacy=0x7f900c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.969+0000 7f9010f9a700 1 -- 172.21.15.49:0/3294103899 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0b3910 con 0x7f900c0d1220 2023-07-18T03:40:21.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.970+0000 7f9010f9a700 1 --2- 172.21.15.49:0/3294103899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f900c0d1220 0x7f900c0b43f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009840 tx=0x7f8ffc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d82ba644d33913 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:21.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.972+0000 7f9003fff700 1 -- 172.21.15.49:0/3294103899 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc011830 con 0x7f900c0d1220 2023-07-18T03:40:21.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.972+0000 7f9003fff700 1 -- 172.21.15.49:0/3294103899 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc011990 con 0x7f900c0d1220 2023-07-18T03:40:21.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.972+0000 7f9003fff700 1 -- 172.21.15.49:0/3294103899 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc011d20 con 0x7f900c0d1220 2023-07-18T03:40:21.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.973+0000 7f90139ff700 1 -- 172.21.15.49:0/3294103899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f900c0d1220 msgr2=0x7f900c0b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:21.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.973+0000 7f90139ff700 1 --2- 172.21.15.49:0/3294103899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f900c0d1220 0x7f900c0b43f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009840 tx=0x7f8ffc009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:21.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.973+0000 7f90139ff700 1 -- 172.21.15.49:0/3294103899 shutdown_connections 2023-07-18T03:40:21.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.973+0000 7f90139ff700 1 --2- 172.21.15.49:0/3294103899 >> v2:172.21.15.49:3300/0 conn(0x7f900c0b1e00 0x7f900c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.973+0000 7f90139ff700 1 --2- 172.21.15.49:0/3294103899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f900c0d1220 0x7f900c0b43f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.973+0000 7f90139ff700 1 -- 172.21.15.49:0/3294103899 >> 172.21.15.49:0/3294103899 conn(0x7f900c09c1d0 msgr2=0x7f900c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.973+0000 7f90139ff700 1 -- 172.21.15.49:0/3294103899 shutdown_connections 2023-07-18T03:40:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.973+0000 7f90139ff700 1 -- 172.21.15.49:0/3294103899 wait complete. 2023-07-18T03:40:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.975+0000 7f90139ff700 1 Processor -- start 2023-07-18T03:40:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.975+0000 7f90139ff700 1 -- start start 2023-07-18T03:40:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f90139ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0b1e00 0x7f900c156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f90139ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f900c0d1220 0x7f900c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f90139ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f900c15c950 0x7f900c0cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f90139ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c158d60 con 0x7f900c0b1e00 2023-07-18T03:40:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f90139ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c159000 con 0x7f900c15c950 2023-07-18T03:40:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f90139ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1592e0 con 0x7f900c0d1220 2023-07-18T03:40:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f901179b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0b1e00 0x7f900c156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f901179b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0b1e00 0x7f900c156fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46678/0 (socket says 172.21.15.49:46678) 2023-07-18T03:40:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f901179b700 1 -- 172.21.15.49:0/2988775342 learned_addr learned my addr 172.21.15.49:0/2988775342 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f9011f9c700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f900c15c950 0x7f900c0cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f9010f9a700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f900c0d1220 0x7f900c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f901179b700 1 -- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f900c0d1220 msgr2=0x7f900c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f901179b700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f900c0d1220 0x7f900c0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f901179b700 1 -- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f900c15c950 msgr2=0x7f900c0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:21.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f901179b700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f900c15c950 0x7f900c0cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:21.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.976+0000 7f901179b700 1 -- 172.21.15.49:0/2988775342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc0094a0 con 0x7f900c0b1e00 2023-07-18T03:40:21.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.977+0000 7f901179b700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0b1e00 0x7f900c156fb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f900800c500 tx=0x7f900800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.977+0000 7f9001ffb700 1 -- 172.21.15.49:0/2988775342 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008005dc0 con 0x7f900c0b1e00 2023-07-18T03:40:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.977+0000 7f9001ffb700 1 -- 172.21.15.49:0/2988775342 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90080070d0 con 0x7f900c0b1e00 2023-07-18T03:40:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.977+0000 7f9001ffb700 1 -- 172.21.15.49:0/2988775342 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008007460 con 0x7f900c0b1e00 2023-07-18T03:40:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.977+0000 7f90139ff700 1 -- 172.21.15.49:0/2988775342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c1595c0 con 0x7f900c0b1e00 2023-07-18T03:40:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.977+0000 7f90139ff700 1 -- 172.21.15.49:0/2988775342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c1574f0 con 0x7f900c0b1e00 2023-07-18T03:40:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.978+0000 7f90139ff700 1 -- 172.21.15.49:0/2988775342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c04e280 con 0x7f900c0b1e00 2023-07-18T03:40:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.979+0000 7f9001ffb700 1 -- 172.21.15.49:0/2988775342 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f90080075c0 con 0x7f900c0b1e00 2023-07-18T03:40:21.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.980+0000 7f9001ffb700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8ff006ad70 0x7f8ff006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:21.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.980+0000 7f9001ffb700 1 -- 172.21.15.49:0/2988775342 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9008088890 con 0x7f900c0b1e00 2023-07-18T03:40:21.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.981+0000 7f9010f9a700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8ff006ad70 0x7f8ff006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:21.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.981+0000 7f9010f9a700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8ff006ad70 0x7f8ff006d220 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f900c0d1610 tx=0x7f8ffc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:21.985+0000 7f9001ffb700 1 -- 172.21.15.49:0/2988775342 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f90080538e0 con 0x7f900c0b1e00 2023-07-18T03:40:22.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.240+0000 7f90139ff700 1 -- 172.21.15.49:0/2988775342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f900c08c5a0 con 0x7f900c0b1e00 2023-07-18T03:40:22.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.241+0000 7f9001ffb700 1 -- 172.21.15.49:0/2988775342 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f9008056f00 con 0x7f900c0b1e00 2023-07-18T03:40:22.235 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:22.235 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1095518881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1095518881},{"type":"v1","addr":"172.21.15.49:6834","nonce":1095518881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/843626338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":843626338},{"type":"v1","addr":"172.21.15.107:6835","nonce":843626338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3238680863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3238680863},{"type":"v1","addr":"172.21.15.107:6837","nonce":3238680863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:39:56.844870+0000","modified":"2023-07-18T03:40:00.356383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5172":{"gid":5172,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6835/3161575771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":3161575771},{"type":"v1","addr":"172.21.15.49:6835","nonce":3161575771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-18T03:40:22.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.243+0000 7f90139ff700 1 -- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8ff006ad70 msgr2=0x7f8ff006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.243+0000 7f90139ff700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8ff006ad70 0x7f8ff006d220 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f900c0d1610 tx=0x7f8ffc008030 comp rx=0 tx=0).stop 2023-07-18T03:40:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.243+0000 7f90139ff700 1 -- 172.21.15.49:0/2988775342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0b1e00 msgr2=0x7f900c156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.243+0000 7f90139ff700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0b1e00 0x7f900c156fb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f900800c500 tx=0x7f900800c810 comp rx=0 tx=0).stop 2023-07-18T03:40:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.243+0000 7f90139ff700 1 -- 172.21.15.49:0/2988775342 shutdown_connections 2023-07-18T03:40:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.243+0000 7f90139ff700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f900c15c950 0x7f900c0cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.243+0000 7f90139ff700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8ff006ad70 0x7f8ff006d220 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.243+0000 7f90139ff700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f900c0b1e00 0x7f900c156fb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.243+0000 7f90139ff700 1 --2- 172.21.15.49:0/2988775342 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f900c0d1220 0x7f900c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.243+0000 7f90139ff700 1 -- 172.21.15.49:0/2988775342 >> 172.21.15.49:0/2988775342 conn(0x7f900c09c1d0 msgr2=0x7f900c0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:22.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.243+0000 7f90139ff700 1 -- 172.21.15.49:0/2988775342 shutdown_connections 2023-07-18T03:40:22.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.243+0000 7f90139ff700 1 -- 172.21.15.49:0/2988775342 wait complete. 2023-07-18T03:40:22.239 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 19 2023-07-18T03:40:22.256 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:22.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.467+0000 7f6b9f1ff700 1 Processor -- start 2023-07-18T03:40:22.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.467+0000 7f6b9f1ff700 1 -- start start 2023-07-18T03:40:22.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.468+0000 7f6b9f1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6b9815afd0 0x7f6b980cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:22.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.468+0000 7f6b9f1ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b9815b430 con 0x7f6b980b3e90 2023-07-18T03:40:22.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.468+0000 7f6b9f1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b980ce110 con 0x7f6b9815e890 2023-07-18T03:40:22.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.468+0000 7f6b9f1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b980cf380 con 0x7f6b9815afd0 2023-07-18T03:40:22.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.469+0000 7f6b97fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6b980b3e90 0x7f6b9815a8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36108/0 (socket says 172.21.15.49:36108) 2023-07-18T03:40:22.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.469+0000 7f6b97fff700 1 -- 172.21.15.49:0/2451860174 learned_addr learned my addr 172.21.15.49:0/2451860174 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:22.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.469+0000 7f6b9d79c700 1 --2- 172.21.15.49:0/2451860174 >> v2:172.21.15.107:3300/0 conn(0x7f6b9815afd0 0x7f6b980cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:22.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.469+0000 7f6b977fe700 1 -- 172.21.15.49:0/2451860174 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205277901 0 0) 0x7f6b9815b430 con 0x7f6b980b3e90 2023-07-18T03:40:22.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.469+0000 7f6b977fe700 1 -- 172.21.15.49:0/2451860174 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b880035b0 con 0x7f6b980b3e90 2023-07-18T03:40:22.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.469+0000 7f6b9d79c700 1 -- 172.21.15.49:0/2451860174 >> v1:172.21.15.107:6789/0 conn(0x7f6b9815e890 legacy=0x7f6b980b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.469+0000 7f6b9d79c700 1 -- 172.21.15.49:0/2451860174 >> v1:172.21.15.49:6789/0 conn(0x7f6b980b3e90 legacy=0x7f6b9815a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.469+0000 7f6b9d79c700 1 -- 172.21.15.49:0/2451860174 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9815b660 con 0x7f6b9815afd0 2023-07-18T03:40:22.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.470+0000 7f6b9d79c700 1 --2- 172.21.15.49:0/2451860174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b9815afd0 0x7f6b980cee40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009d90 tx=0x7f6b9000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3cd899c06d7920 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:22.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.470+0000 7f6b977fe700 1 -- 172.21.15.49:0/2451860174 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9000e070 con 0x7f6b9815afd0 2023-07-18T03:40:22.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.470+0000 7f6b977fe700 1 -- 172.21.15.49:0/2451860174 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9000bb20 con 0x7f6b9815afd0 2023-07-18T03:40:22.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.470+0000 7f6b977fe700 1 -- 172.21.15.49:0/2451860174 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90012430 con 0x7f6b9815afd0 2023-07-18T03:40:22.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.470+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/2451860174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b9815afd0 msgr2=0x7f6b980cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.470+0000 7f6b9f1ff700 1 --2- 172.21.15.49:0/2451860174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b9815afd0 0x7f6b980cee40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009d90 tx=0x7f6b9000b330 comp rx=0 tx=0).stop 2023-07-18T03:40:22.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.470+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/2451860174 shutdown_connections 2023-07-18T03:40:22.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.470+0000 7f6b9f1ff700 1 --2- 172.21.15.49:0/2451860174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b9815afd0 0x7f6b980cee40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.470+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/2451860174 >> 172.21.15.49:0/2451860174 conn(0x7f6b98069f30 msgr2=0x7f6b9806a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:22.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.470+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/2451860174 shutdown_connections 2023-07-18T03:40:22.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.471+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/2451860174 wait complete. 2023-07-18T03:40:22.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.472+0000 7f6b9f1ff700 1 Processor -- start 2023-07-18T03:40:22.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.472+0000 7f6b9f1ff700 1 -- start start 2023-07-18T03:40:22.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.473+0000 7f6b9f1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b980b3e90 0x7f6b980b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:22.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.473+0000 7f6b9f1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b9815afd0 0x7f6b981e56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:22.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.473+0000 7f6b9f1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b9815e890 0x7f6b981e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:22.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.473+0000 7f6b9f1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b981e4640 con 0x7f6b980b3e90 2023-07-18T03:40:22.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.473+0000 7f6b9f1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b981e4920 con 0x7f6b9815e890 2023-07-18T03:40:22.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.473+0000 7f6b9f1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b981e4c00 con 0x7f6b9815afd0 2023-07-18T03:40:22.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.473+0000 7f6b9cf9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b980b3e90 0x7f6b980b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:22.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.474+0000 7f6b9cf9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b980b3e90 0x7f6b980b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46694/0 (socket says 172.21.15.49:46694) 2023-07-18T03:40:22.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.474+0000 7f6b9cf9b700 1 -- 172.21.15.49:0/946571476 learned_addr learned my addr 172.21.15.49:0/946571476 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:22.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.474+0000 7f6b97fff700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b9815afd0 0x7f6b981e56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:22.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.474+0000 7f6b9d79c700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b9815e890 0x7f6b981e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:22.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.474+0000 7f6b9cf9b700 1 -- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b9815afd0 msgr2=0x7f6b981e56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.474+0000 7f6b9cf9b700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b9815afd0 0x7f6b981e56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.474+0000 7f6b9cf9b700 1 -- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b9815e890 msgr2=0x7f6b981e3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.474+0000 7f6b9cf9b700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b9815e890 0x7f6b981e3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.474+0000 7f6b9cf9b700 1 -- 172.21.15.49:0/946571476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b90009a70 con 0x7f6b980b3e90 2023-07-18T03:40:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.474+0000 7f6b9cf9b700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b980b3e90 0x7f6b980b0760 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00c450 tx=0x7f6b8c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.474+0000 7f6b957fa700 1 -- 172.21.15.49:0/946571476 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0052d0 con 0x7f6b980b3e90 2023-07-18T03:40:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.475+0000 7f6b957fa700 1 -- 172.21.15.49:0/946571476 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c005430 con 0x7f6b980b3e90 2023-07-18T03:40:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.475+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/946571476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b981e4ee0 con 0x7f6b980b3e90 2023-07-18T03:40:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.475+0000 7f6b957fa700 1 -- 172.21.15.49:0/946571476 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0038c0 con 0x7f6b980b3e90 2023-07-18T03:40:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.475+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/946571476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b98153bb0 con 0x7f6b980b3e90 2023-07-18T03:40:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.476+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/946571476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b981e5400 con 0x7f6b980b3e90 2023-07-18T03:40:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.477+0000 7f6b957fa700 1 -- 172.21.15.49:0/946571476 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f6b8c003a20 con 0x7f6b980b3e90 2023-07-18T03:40:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.478+0000 7f6b957fa700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6b7c06ad70 0x7f6b7c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.478+0000 7f6b957fa700 1 -- 172.21.15.49:0/946571476 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6b8c0888b0 con 0x7f6b980b3e90 2023-07-18T03:40:22.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.478+0000 7f6b97fff700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6b7c06ad70 0x7f6b7c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:22.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.479+0000 7f6b97fff700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6b7c06ad70 0x7f6b7c06d220 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6b980cacd0 tx=0x7f6b84007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:22.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.482+0000 7f6b957fa700 1 -- 172.21.15.49:0/946571476 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6b8c053870 con 0x7f6b980b3e90 2023-07-18T03:40:22.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.691+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/946571476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b9804e280 con 0x7f6b980b3e90 2023-07-18T03:40:22.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.693+0000 7f6b957fa700 1 -- 172.21.15.49:0/946571476 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18257 (secure 0 0 0) 0x7f6b8c00ce60 con 0x7f6b980b3e90 2023-07-18T03:40:22.687 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:22.688 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:39:58.343119+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:39:54.412334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-18T03:39:55.841218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:22.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.695+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6b7c06ad70 msgr2=0x7f6b7c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.695+0000 7f6b9f1ff700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6b7c06ad70 0x7f6b7c06d220 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6b980cacd0 tx=0x7f6b84007dc0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.695+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/946571476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b980b3e90 msgr2=0x7f6b980b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.695+0000 7f6b9f1ff700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b980b3e90 0x7f6b980b0760 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00c450 tx=0x7f6b8c00c760 comp rx=0 tx=0).stop 2023-07-18T03:40:22.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.695+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/946571476 shutdown_connections 2023-07-18T03:40:22.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.695+0000 7f6b9f1ff700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b9815e890 0x7f6b981e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.695+0000 7f6b9f1ff700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6b7c06ad70 0x7f6b7c06d220 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.695+0000 7f6b9f1ff700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b980b3e90 0x7f6b980b0760 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.695+0000 7f6b9f1ff700 1 --2- 172.21.15.49:0/946571476 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b9815afd0 0x7f6b981e56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.695+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/946571476 >> 172.21.15.49:0/946571476 conn(0x7f6b98069f30 msgr2=0x7f6b98159300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:22.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.695+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/946571476 shutdown_connections 2023-07-18T03:40:22.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.695+0000 7f6b9f1ff700 1 -- 172.21.15.49:0/946571476 wait complete. 2023-07-18T03:40:22.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:39:54.412334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:39:55.841218+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:40:22.709 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:40:22.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.922+0000 7f7c095ff700 1 Processor -- start 2023-07-18T03:40:22.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.922+0000 7f7c095ff700 1 -- start start 2023-07-18T03:40:22.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.924+0000 7f7c095ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c040b8660 0x7f7c040cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:22.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.924+0000 7f7c095ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c040cd970 0x7f7c040b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:22.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.924+0000 7f7c095ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c040d02a0 con 0x7f7c040b4da0 2023-07-18T03:40:22.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.924+0000 7f7c095ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c040b51c0 con 0x7f7c040b8660 2023-07-18T03:40:22.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.924+0000 7f7c095ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040cfce0 con 0x7f7c040cd970 2023-07-18T03:40:22.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.924+0000 7f7c02ffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c040b8660 0x7f7c040cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:22.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.924+0000 7f7c02ffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c040b8660 0x7f7c040cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46702/0 (socket says 172.21.15.49:46702) 2023-07-18T03:40:22.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.924+0000 7f7c02ffd700 1 -- 172.21.15.49:0/586922378 learned_addr learned my addr 172.21.15.49:0/586922378 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:22.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.924+0000 7f7c037fe700 1 --1- 172.21.15.49:0/586922378 >> v1:172.21.15.49:6789/0 conn(0x7f7c040b4da0 0x7f7c041601f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36112/0 (socket says 172.21.15.49:36112) 2023-07-18T03:40:22.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.924+0000 7f7bfbfff700 1 --2- 172.21.15.49:0/586922378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c040cd970 0x7f7c040b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:22.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.924+0000 7f7c02ffd700 1 -- 172.21.15.49:0/586922378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c040cd970 msgr2=0x7f7c040b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:22.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.924+0000 7f7c02ffd700 1 --2- 172.21.15.49:0/586922378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c040cd970 0x7f7c040b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.925+0000 7f7c02ffd700 1 -- 172.21.15.49:0/586922378 >> v1:172.21.15.49:6789/0 conn(0x7f7c040b4da0 legacy=0x7f7c041601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.925+0000 7f7c027fc700 1 -- 172.21.15.49:0/586922378 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737216861 0 0) 0x7f7c040d02a0 con 0x7f7c040b4da0 2023-07-18T03:40:22.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.925+0000 7f7c02ffd700 1 -- 172.21.15.49:0/586922378 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c040cffc0 con 0x7f7c040b8660 2023-07-18T03:40:22.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.925+0000 7f7c02ffd700 1 --2- 172.21.15.49:0/586922378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c040b8660 0x7f7c040cd430 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40098f0 tx=0x7f7bf4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ed3e995d9f1a9d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:22.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.925+0000 7f7c027fc700 1 -- 172.21.15.49:0/586922378 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400b8a0 con 0x7f7c040b8660 2023-07-18T03:40:22.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.925+0000 7f7c027fc700 1 -- 172.21.15.49:0/586922378 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf400ba00 con 0x7f7c040b8660 2023-07-18T03:40:22.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.925+0000 7f7c027fc700 1 -- 172.21.15.49:0/586922378 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400bd90 con 0x7f7c040b8660 2023-07-18T03:40:22.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.926+0000 7f7c095ff700 1 -- 172.21.15.49:0/586922378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c040b8660 msgr2=0x7f7c040cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.926+0000 7f7c095ff700 1 --2- 172.21.15.49:0/586922378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c040b8660 0x7f7c040cd430 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40098f0 tx=0x7f7bf4009c00 comp rx=0 tx=0).stop 2023-07-18T03:40:22.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.926+0000 7f7c095ff700 1 -- 172.21.15.49:0/586922378 shutdown_connections 2023-07-18T03:40:22.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.926+0000 7f7c095ff700 1 --2- 172.21.15.49:0/586922378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c040b8660 0x7f7c040cd430 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.926+0000 7f7c095ff700 1 --2- 172.21.15.49:0/586922378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c040cd970 0x7f7c040b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.926+0000 7f7c095ff700 1 -- 172.21.15.49:0/586922378 >> 172.21.15.49:0/586922378 conn(0x7f7c04066860 msgr2=0x7f7c0406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:22.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.926+0000 7f7c095ff700 1 -- 172.21.15.49:0/586922378 shutdown_connections 2023-07-18T03:40:22.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.926+0000 7f7c095ff700 1 -- 172.21.15.49:0/586922378 wait complete. 2023-07-18T03:40:22.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.927+0000 7f7c095ff700 1 Processor -- start 2023-07-18T03:40:22.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.928+0000 7f7c095ff700 1 -- start start 2023-07-18T03:40:22.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.928+0000 7f7c095ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c040b4da0 0x7f7c041e6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:22.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.928+0000 7f7c095ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c040b8660 0x7f7c040c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:22.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.928+0000 7f7c095ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c040cd970 0x7f7c040c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:22.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c095ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040cb3c0 con 0x7f7c040b4da0 2023-07-18T03:40:22.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c095ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040ca7b0 con 0x7f7c040cd970 2023-07-18T03:40:22.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c095ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040caa90 con 0x7f7c040b8660 2023-07-18T03:40:22.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c02ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c040b4da0 0x7f7c041e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:22.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c02ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c040b4da0 0x7f7c041e6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46706/0 (socket says 172.21.15.49:46706) 2023-07-18T03:40:22.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c02ffd700 1 -- 172.21.15.49:0/876639408 learned_addr learned my addr 172.21.15.49:0/876639408 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:22.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7bfbfff700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c040b8660 0x7f7c040c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:22.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c037fe700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c040cd970 0x7f7c040c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:22.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c02ffd700 1 -- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c040b8660 msgr2=0x7f7c040c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c02ffd700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c040b8660 0x7f7c040c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c02ffd700 1 -- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c040cd970 msgr2=0x7f7c040c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:22.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c02ffd700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c040cd970 0x7f7c040c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:22.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c02ffd700 1 -- 172.21.15.49:0/876639408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf4009550 con 0x7f7c040b4da0 2023-07-18T03:40:22.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.929+0000 7f7c037fe700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c040cd970 0x7f7c040c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:22.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.930+0000 7f7c02ffd700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c040b4da0 0x7f7c041e6220 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009c40 tx=0x7f7bf4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:22.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.930+0000 7f7bfb7fe700 1 -- 172.21.15.49:0/876639408 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4004380 con 0x7f7c040b4da0 2023-07-18T03:40:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.930+0000 7f7bfb7fe700 1 -- 172.21.15.49:0/876639408 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf401f070 con 0x7f7c040b4da0 2023-07-18T03:40:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.930+0000 7f7bfb7fe700 1 -- 172.21.15.49:0/876639408 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40036a0 con 0x7f7c040b4da0 2023-07-18T03:40:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.930+0000 7f7c095ff700 1 -- 172.21.15.49:0/876639408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c040cad70 con 0x7f7c040b4da0 2023-07-18T03:40:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.930+0000 7f7c095ff700 1 -- 172.21.15.49:0/876639408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c041e6760 con 0x7f7c040b4da0 2023-07-18T03:40:22.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.931+0000 7f7c095ff700 1 -- 172.21.15.49:0/876639408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c04074350 con 0x7f7c040b4da0 2023-07-18T03:40:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.932+0000 7f7bfb7fe700 1 -- 172.21.15.49:0/876639408 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f7bf4021030 con 0x7f7c040b4da0 2023-07-18T03:40:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.933+0000 7f7bfb7fe700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7be406ad70 0x7f7be406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.933+0000 7f7bfb7fe700 1 -- 172.21.15.49:0/876639408 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7bf408a9d0 con 0x7f7c040b4da0 2023-07-18T03:40:22.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.933+0000 7f7bfbfff700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7be406ad70 0x7f7be406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.934+0000 7f7bfbfff700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7be406ad70 0x7f7be406d220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7c040d3ad0 tx=0x7f7bec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:22.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:22.937+0000 7f7bfb7fe700 1 -- 172.21.15.49:0/876639408 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7bf4055a20 con 0x7f7c040b4da0 2023-07-18T03:40:23.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.147+0000 7f7c095ff700 1 -- 172.21.15.49:0/876639408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7c040b2150 con 0x7f7c040b4da0 2023-07-18T03:40:23.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.147+0000 7f7bfb7fe700 1 -- 172.21.15.49:0/876639408 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7bf4015020 con 0x7f7c040b4da0 2023-07-18T03:40:23.142 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:40:23.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.150+0000 7f7c095ff700 1 -- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7be406ad70 msgr2=0x7f7be406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.150+0000 7f7c095ff700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7be406ad70 0x7f7be406d220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7c040d3ad0 tx=0x7f7bec008040 comp rx=0 tx=0).stop 2023-07-18T03:40:23.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.150+0000 7f7c095ff700 1 -- 172.21.15.49:0/876639408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c040b4da0 msgr2=0x7f7c041e6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.150+0000 7f7c095ff700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c040b4da0 0x7f7c041e6220 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009c40 tx=0x7f7bf4004750 comp rx=0 tx=0).stop 2023-07-18T03:40:23.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.150+0000 7f7c095ff700 1 -- 172.21.15.49:0/876639408 shutdown_connections 2023-07-18T03:40:23.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.150+0000 7f7c095ff700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c040cd970 0x7f7c040c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.150+0000 7f7c095ff700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7be406ad70 0x7f7be406d220 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.150+0000 7f7c095ff700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c040b4da0 0x7f7c041e6220 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.150+0000 7f7c095ff700 1 --2- 172.21.15.49:0/876639408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c040b8660 0x7f7c040c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.150+0000 7f7c095ff700 1 -- 172.21.15.49:0/876639408 >> 172.21.15.49:0/876639408 conn(0x7f7c04066860 msgr2=0x7f7c040cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:23.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.150+0000 7f7c095ff700 1 -- 172.21.15.49:0/876639408 shutdown_connections 2023-07-18T03:40:23.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.150+0000 7f7c095ff700 1 -- 172.21.15.49:0/876639408 wait complete. 2023-07-18T03:40:23.163 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:40:23.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.379+0000 7fd45efff700 1 Processor -- start 2023-07-18T03:40:23.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.379+0000 7fd45efff700 1 -- start start 2023-07-18T03:40:23.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.380+0000 7fd45efff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd4580b12f0 0x7fd4580ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.380+0000 7fd45efff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4580adea0 0x7fd4580cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.380+0000 7fd45efff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45815ce70 con 0x7fd4580c92d0 2023-07-18T03:40:23.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.380+0000 7fd45efff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45815da60 con 0x7fd4580b12f0 2023-07-18T03:40:23.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.380+0000 7fd45efff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45815d0c0 con 0x7fd4580adea0 2023-07-18T03:40:23.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.380+0000 7fd457fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd4580b12f0 0x7fd4580ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.381+0000 7fd457fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd4580b12f0 0x7fd4580ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46720/0 (socket says 172.21.15.49:46720) 2023-07-18T03:40:23.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.381+0000 7fd457fff700 1 -- 172.21.15.49:0/2645029186 learned_addr learned my addr 172.21.15.49:0/2645029186 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:23.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.381+0000 7fd45cd9b700 1 --1- 172.21.15.49:0/2645029186 >> v1:172.21.15.49:6789/0 conn(0x7fd4580c92d0 0x7fd4580c96a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36126/0 (socket says 172.21.15.49:36126) 2023-07-18T03:40:23.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.381+0000 7fd457fff700 1 -- 172.21.15.49:0/2645029186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4580adea0 msgr2=0x7fd4580cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:23.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.381+0000 7fd457fff700 1 --2- 172.21.15.49:0/2645029186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4580adea0 0x7fd4580cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.381+0000 7fd457fff700 1 -- 172.21.15.49:0/2645029186 >> v1:172.21.15.49:6789/0 conn(0x7fd4580c92d0 legacy=0x7fd4580c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.381+0000 7fd457fff700 1 -- 172.21.15.49:0/2645029186 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45815d3a0 con 0x7fd4580b12f0 2023-07-18T03:40:23.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.381+0000 7fd457fff700 1 --2- 172.21.15.49:0/2645029186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4580b12f0 0x7fd4580ad8f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd4440098f0 tx=0x7fd444009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232bcc7214481654 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:23.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.381+0000 7fd4577fe700 1 -- 172.21.15.49:0/2645029186 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44400b8a0 con 0x7fd4580b12f0 2023-07-18T03:40:23.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.382+0000 7fd4577fe700 1 -- 172.21.15.49:0/2645029186 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44400ba00 con 0x7fd4580b12f0 2023-07-18T03:40:23.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.382+0000 7fd4577fe700 1 -- 172.21.15.49:0/2645029186 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44400bd90 con 0x7fd4580b12f0 2023-07-18T03:40:23.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.382+0000 7fd45efff700 1 -- 172.21.15.49:0/2645029186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4580b12f0 msgr2=0x7fd4580ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.382+0000 7fd45efff700 1 --2- 172.21.15.49:0/2645029186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4580b12f0 0x7fd4580ad8f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd4440098f0 tx=0x7fd444009c00 comp rx=0 tx=0).stop 2023-07-18T03:40:23.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.382+0000 7fd45efff700 1 -- 172.21.15.49:0/2645029186 shutdown_connections 2023-07-18T03:40:23.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.382+0000 7fd45efff700 1 --2- 172.21.15.49:0/2645029186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4580adea0 0x7fd4580cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.382+0000 7fd45efff700 1 --2- 172.21.15.49:0/2645029186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4580b12f0 0x7fd4580ad8f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.382+0000 7fd45efff700 1 -- 172.21.15.49:0/2645029186 >> 172.21.15.49:0/2645029186 conn(0x7fd4580b05d0 msgr2=0x7fd4580b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:23.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.382+0000 7fd45efff700 1 -- 172.21.15.49:0/2645029186 shutdown_connections 2023-07-18T03:40:23.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.382+0000 7fd45efff700 1 -- 172.21.15.49:0/2645029186 wait complete. 2023-07-18T03:40:23.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.384+0000 7fd45efff700 1 Processor -- start 2023-07-18T03:40:23.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.384+0000 7fd45efff700 1 -- start start 2023-07-18T03:40:23.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.385+0000 7fd45efff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4580adea0 0x7fd458154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.385+0000 7fd45efff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4580b12f0 0x7fd45815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.385+0000 7fd45efff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4580c92d0 0x7fd458157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.385+0000 7fd45efff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4581551d0 con 0x7fd4580b12f0 2023-07-18T03:40:23.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.385+0000 7fd45efff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd458159bc0 con 0x7fd4580c92d0 2023-07-18T03:40:23.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.385+0000 7fd45efff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd458159ea0 con 0x7fd4580adea0 2023-07-18T03:40:23.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.385+0000 7fd457fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4580b12f0 0x7fd45815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.385+0000 7fd457fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4580b12f0 0x7fd45815ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46722/0 (socket says 172.21.15.49:46722) 2023-07-18T03:40:23.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.386+0000 7fd457fff700 1 -- 172.21.15.49:0/2224751197 learned_addr learned my addr 172.21.15.49:0/2224751197 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:23.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.386+0000 7fd45cd9b700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4580adea0 0x7fd458154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.386+0000 7fd45d59c700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4580c92d0 0x7fd458157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.386+0000 7fd457fff700 1 -- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4580adea0 msgr2=0x7fd458154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.386+0000 7fd457fff700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4580adea0 0x7fd458154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.386+0000 7fd457fff700 1 -- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4580c92d0 msgr2=0x7fd458157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.386+0000 7fd457fff700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4580c92d0 0x7fd458157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.386+0000 7fd457fff700 1 -- 172.21.15.49:0/2224751197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd444009550 con 0x7fd4580b12f0 2023-07-18T03:40:23.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.386+0000 7fd457fff700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4580b12f0 0x7fd45815ac60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd444000c00 tx=0x7fd444004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:23.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.386+0000 7fd4557fa700 1 -- 172.21.15.49:0/2224751197 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444004510 con 0x7fd4580b12f0 2023-07-18T03:40:23.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.387+0000 7fd4557fa700 1 -- 172.21.15.49:0/2224751197 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4440036a0 con 0x7fd4580b12f0 2023-07-18T03:40:23.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.387+0000 7fd45efff700 1 -- 172.21.15.49:0/2224751197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45815a180 con 0x7fd4580b12f0 2023-07-18T03:40:23.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.387+0000 7fd4557fa700 1 -- 172.21.15.49:0/2224751197 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444003a10 con 0x7fd4580b12f0 2023-07-18T03:40:23.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.387+0000 7fd45efff700 1 -- 172.21.15.49:0/2224751197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45815a740 con 0x7fd4580b12f0 2023-07-18T03:40:23.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.388+0000 7fd45efff700 1 -- 172.21.15.49:0/2224751197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd458074350 con 0x7fd4580b12f0 2023-07-18T03:40:23.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.389+0000 7fd4557fa700 1 -- 172.21.15.49:0/2224751197 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fd444003b70 con 0x7fd4580b12f0 2023-07-18T03:40:23.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.390+0000 7fd4557fa700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd43c06ad70 0x7fd43c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.390+0000 7fd4557fa700 1 -- 172.21.15.49:0/2224751197 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd444021030 con 0x7fd4580b12f0 2023-07-18T03:40:23.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.390+0000 7fd45cd9b700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd43c06ad70 0x7fd43c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.391+0000 7fd45cd9b700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd43c06ad70 0x7fd43c06d220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd44c000f60 tx=0x7fd44c007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:23.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.394+0000 7fd4557fa700 1 -- 172.21.15.49:0/2224751197 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd444055ca0 con 0x7fd4580b12f0 2023-07-18T03:40:23.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.598+0000 7fd45efff700 1 -- 172.21.15.49:0/2224751197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd45806acb0 con 0x7fd4580b12f0 2023-07-18T03:40:23.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.599+0000 7fd4557fa700 1 -- 172.21.15.49:0/2224751197 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd444015020 con 0x7fd4580b12f0 2023-07-18T03:40:23.593 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:40:23.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.601+0000 7fd45efff700 1 -- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd43c06ad70 msgr2=0x7fd43c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.601+0000 7fd45efff700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd43c06ad70 0x7fd43c06d220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd44c000f60 tx=0x7fd44c007ed0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.601+0000 7fd45efff700 1 -- 172.21.15.49:0/2224751197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4580b12f0 msgr2=0x7fd45815ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.601+0000 7fd45efff700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4580b12f0 0x7fd45815ac60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd444000c00 tx=0x7fd444004750 comp rx=0 tx=0).stop 2023-07-18T03:40:23.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.601+0000 7fd45efff700 1 -- 172.21.15.49:0/2224751197 shutdown_connections 2023-07-18T03:40:23.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.601+0000 7fd45efff700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4580c92d0 0x7fd458157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.601+0000 7fd45efff700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4580adea0 0x7fd458154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.601+0000 7fd45efff700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd43c06ad70 0x7fd43c06d220 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.601+0000 7fd45efff700 1 --2- 172.21.15.49:0/2224751197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4580b12f0 0x7fd45815ac60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.601+0000 7fd45efff700 1 -- 172.21.15.49:0/2224751197 >> 172.21.15.49:0/2224751197 conn(0x7fd4580b05d0 msgr2=0x7fd45806c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:23.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.601+0000 7fd45efff700 1 -- 172.21.15.49:0/2224751197 shutdown_connections 2023-07-18T03:40:23.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.601+0000 7fd45efff700 1 -- 172.21.15.49:0/2224751197 wait complete. 2023-07-18T03:40:23.614 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:40:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.829+0000 7f609c7ff700 1 Processor -- start 2023-07-18T03:40:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.829+0000 7f609c7ff700 1 -- start start 2023-07-18T03:40:23.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.830+0000 7f609c7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60940b4860 0x7f60940b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.830+0000 7f609c7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f60940d12c0 0x7f60940d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.830+0000 7f609c7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60940d02a0 con 0x7f60940b8660 2023-07-18T03:40:23.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.830+0000 7f609c7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60940b5210 con 0x7f60940d12c0 2023-07-18T03:40:23.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.830+0000 7f609c7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60940cfce0 con 0x7f60940b4860 2023-07-18T03:40:23.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.831+0000 7f6099d9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60940b4860 0x7f60940b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.831+0000 7f609ad9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f60940d12c0 0x7f60940d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.831+0000 7f6099d9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60940b4860 0x7f60940b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:34530/0 (socket says 172.21.15.49:34530) 2023-07-18T03:40:23.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.831+0000 7f6099d9a700 1 -- 172.21.15.49:0/864958102 learned_addr learned my addr 172.21.15.49:0/864958102 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:23.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.831+0000 7f609ad9c700 1 -- 172.21.15.49:0/864958102 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60940b4860 msgr2=0x7f60940b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:23.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.831+0000 7f609ad9c700 1 --2- 172.21.15.49:0/864958102 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60940b4860 0x7f60940b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.831+0000 7f609ad9c700 1 -- 172.21.15.49:0/864958102 >> v1:172.21.15.107:6789/0 conn(0x7f60940b8660 legacy=0x7f60940cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.831+0000 7f609ad9c700 1 -- 172.21.15.49:0/864958102 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60940cffc0 con 0x7f60940d12c0 2023-07-18T03:40:23.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.831+0000 7f609ad9c700 1 --2- 172.21.15.49:0/864958102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60940d12c0 0x7f60940d1700 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f608c009d90 tx=0x7f608c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d2bdb489fe48026 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:23.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.832+0000 7f6099599700 1 -- 172.21.15.49:0/864958102 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c00e070 con 0x7f60940d12c0 2023-07-18T03:40:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.832+0000 7f6099599700 1 -- 172.21.15.49:0/864958102 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608c00bb20 con 0x7f60940d12c0 2023-07-18T03:40:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.832+0000 7f6099599700 1 -- 172.21.15.49:0/864958102 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c012430 con 0x7f60940d12c0 2023-07-18T03:40:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.832+0000 7f609c7ff700 1 -- 172.21.15.49:0/864958102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60940d12c0 msgr2=0x7f60940d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.832+0000 7f609c7ff700 1 --2- 172.21.15.49:0/864958102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60940d12c0 0x7f60940d1700 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f608c009d90 tx=0x7f608c00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:23.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.832+0000 7f609c7ff700 1 -- 172.21.15.49:0/864958102 shutdown_connections 2023-07-18T03:40:23.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.832+0000 7f609c7ff700 1 --2- 172.21.15.49:0/864958102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60940d12c0 0x7f60940d1700 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.832+0000 7f609c7ff700 1 --2- 172.21.15.49:0/864958102 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60940b4860 0x7f60940b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.832+0000 7f609c7ff700 1 -- 172.21.15.49:0/864958102 >> 172.21.15.49:0/864958102 conn(0x7f6094066860 msgr2=0x7f609406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:23.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.832+0000 7f609c7ff700 1 -- 172.21.15.49:0/864958102 shutdown_connections 2023-07-18T03:40:23.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.832+0000 7f609c7ff700 1 -- 172.21.15.49:0/864958102 wait complete. 2023-07-18T03:40:23.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.834+0000 7f609c7ff700 1 Processor -- start 2023-07-18T03:40:23.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.834+0000 7f609c7ff700 1 -- start start 2023-07-18T03:40:23.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.835+0000 7f609c7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60940b4860 0x7f6094144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.835+0000 7f609c7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60940b8660 0x7f6094145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.835+0000 7f609c7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60940d12c0 0x7f60941f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.835+0000 7f609c7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6094145750 con 0x7f60940d12c0 2023-07-18T03:40:23.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.835+0000 7f609c7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6094144b40 con 0x7f60940b4860 2023-07-18T03:40:23.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.835+0000 7f609c7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6094144e20 con 0x7f60940b8660 2023-07-18T03:40:23.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.836+0000 7f609ad9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60940d12c0 0x7f60941f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.836+0000 7f609ad9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60940d12c0 0x7f60941f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46738/0 (socket says 172.21.15.49:46738) 2023-07-18T03:40:23.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.836+0000 7f609ad9c700 1 -- 172.21.15.49:0/3893994489 learned_addr learned my addr 172.21.15.49:0/3893994489 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:23.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.836+0000 7f609a59b700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60940b4860 0x7f6094144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.836+0000 7f6099d9a700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60940b8660 0x7f6094145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.836+0000 7f609ad9c700 1 -- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60940b8660 msgr2=0x7f6094145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.836+0000 7f609ad9c700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60940b8660 0x7f6094145be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.836+0000 7f609ad9c700 1 -- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60940b4860 msgr2=0x7f6094144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.836+0000 7f609ad9c700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60940b4860 0x7f6094144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.836+0000 7f609ad9c700 1 -- 172.21.15.49:0/3893994489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c009a70 con 0x7f60940d12c0 2023-07-18T03:40:23.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.836+0000 7f609ad9c700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60940d12c0 0x7f60941f1e80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f608c014040 tx=0x7f608c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:23.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.837+0000 7f608affd700 1 -- 172.21.15.49:0/3893994489 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c012430 con 0x7f60940d12c0 2023-07-18T03:40:23.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.837+0000 7f608affd700 1 -- 172.21.15.49:0/3893994489 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608c020070 con 0x7f60940d12c0 2023-07-18T03:40:23.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.837+0000 7f608affd700 1 -- 172.21.15.49:0/3893994489 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c003ea0 con 0x7f60940d12c0 2023-07-18T03:40:23.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.837+0000 7f609c7ff700 1 -- 172.21.15.49:0/3893994489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6094145100 con 0x7f60940d12c0 2023-07-18T03:40:23.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.837+0000 7f609c7ff700 1 -- 172.21.15.49:0/3893994489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6094146120 con 0x7f60940d12c0 2023-07-18T03:40:23.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.838+0000 7f609c7ff700 1 -- 172.21.15.49:0/3893994489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6094145900 con 0x7f60940d12c0 2023-07-18T03:40:23.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.839+0000 7f608affd700 1 -- 172.21.15.49:0/3893994489 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f608c004040 con 0x7f60940d12c0 2023-07-18T03:40:23.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.840+0000 7f608affd700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f607806ad70 0x7f607806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.840+0000 7f608affd700 1 -- 172.21.15.49:0/3893994489 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f608c022070 con 0x7f60940d12c0 2023-07-18T03:40:23.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.840+0000 7f609a59b700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f607806ad70 0x7f607806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.841+0000 7f609a59b700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f607806ad70 0x7f607806d220 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6090007fd0 tx=0x7f6090009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:23.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:23.844+0000 7f608affd700 1 -- 172.21.15.49:0/3893994489 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6094145900 con 0x7f60940d12c0 2023-07-18T03:40:24.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.056+0000 7f609c7ff700 1 -- 172.21.15.49:0/3893994489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6094074350 con 0x7f60940d12c0 2023-07-18T03:40:24.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.057+0000 7f608affd700 1 -- 172.21.15.49:0/3893994489 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f608c02c020 con 0x7f60940d12c0 2023-07-18T03:40:24.051 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:40:24.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.059+0000 7f609c7ff700 1 -- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f607806ad70 msgr2=0x7f607806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.059+0000 7f609c7ff700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f607806ad70 0x7f607806d220 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6090007fd0 tx=0x7f6090009040 comp rx=0 tx=0).stop 2023-07-18T03:40:24.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.059+0000 7f609c7ff700 1 -- 172.21.15.49:0/3893994489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60940d12c0 msgr2=0x7f60941f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.059+0000 7f609c7ff700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60940d12c0 0x7f60941f1e80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f608c014040 tx=0x7f608c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.059+0000 7f609c7ff700 1 -- 172.21.15.49:0/3893994489 shutdown_connections 2023-07-18T03:40:24.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.059+0000 7f609c7ff700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f607806ad70 0x7f607806d220 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.059+0000 7f609c7ff700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60940d12c0 0x7f60941f1e80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.059+0000 7f609c7ff700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60940b4860 0x7f6094144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.059+0000 7f609c7ff700 1 --2- 172.21.15.49:0/3893994489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60940b8660 0x7f6094145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.059+0000 7f609c7ff700 1 -- 172.21.15.49:0/3893994489 >> 172.21.15.49:0/3893994489 conn(0x7f6094066860 msgr2=0x7f60940d16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:24.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.060+0000 7f609c7ff700 1 -- 172.21.15.49:0/3893994489 shutdown_connections 2023-07-18T03:40:24.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.060+0000 7f609c7ff700 1 -- 172.21.15.49:0/3893994489 wait complete. 2023-07-18T03:40:24.072 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:40:24.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.286+0000 7f7f579ff700 1 Processor -- start 2023-07-18T03:40:24.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.286+0000 7f7f579ff700 1 -- start start 2023-07-18T03:40:24.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.287+0000 7f7f579ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7f500b12f0 0x7f7f500ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.288+0000 7f7f579ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5015ce70 con 0x7f7f500c92d0 2023-07-18T03:40:24.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.288+0000 7f7f579ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5015d0c0 con 0x7f7f500adea0 2023-07-18T03:40:24.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.288+0000 7f7f579ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f5015da60 con 0x7f7f500b12f0 2023-07-18T03:40:24.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.288+0000 7f7f5579b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7f500c92d0 0x7f7f500c96a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36132/0 (socket says 172.21.15.49:36132) 2023-07-18T03:40:24.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.288+0000 7f7f5579b700 1 -- 172.21.15.49:0/221740964 learned_addr learned my addr 172.21.15.49:0/221740964 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.288+0000 7f7f54f9a700 1 --2- 172.21.15.49:0/221740964 >> v2:172.21.15.107:3300/0 conn(0x7f7f500b12f0 0x7f7f500ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.288+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596593937 0 0) 0x7f7f5015ce70 con 0x7f7f500c92d0 2023-07-18T03:40:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.288+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f3c0035b0 con 0x7f7f500c92d0 2023-07-18T03:40:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918248864 0 0) 0x7f7f5015d0c0 con 0x7f7f500adea0 2023-07-18T03:40:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f5015ce70 con 0x7f7f500adea0 2023-07-18T03:40:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1325505761 0 0) 0x7f7f3c0035b0 con 0x7f7f500c92d0 2023-07-18T03:40:24.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 >> v2:172.21.15.107:3300/0 conn(0x7f7f500b12f0 msgr2=0x7f7f500ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 --2- 172.21.15.49:0/221740964 >> v2:172.21.15.107:3300/0 conn(0x7f7f500b12f0 0x7f7f500ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 >> v1:172.21.15.107:6789/0 conn(0x7f7f500adea0 legacy=0x7f7f500cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5015d3a0 con 0x7f7f500c92d0 2023-07-18T03:40:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f7f4c001670 con 0x7f7f500c92d0 2023-07-18T03:40:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 >> v1:172.21.15.49:6789/0 conn(0x7f7f500c92d0 legacy=0x7f7f500c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 --2- 172.21.15.49:0/221740964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 0x7f7f3c003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 --2- 172.21.15.49:0/221740964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f3c004850 0x7f7f3c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 --2- 172.21.15.49:0/221740964 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f3c005250 0x7f7f3c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5015da60 con 0x7f7f3c003be0 2023-07-18T03:40:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c00c160 con 0x7f7f3c004850 2023-07-18T03:40:24.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c00e530 con 0x7f7f3c005250 2023-07-18T03:40:24.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f5579b700 1 --2- 172.21.15.49:0/221740964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 0x7f7f3c003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f54f9a700 1 --2- 172.21.15.49:0/221740964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f3c004850 0x7f7f3c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.289+0000 7f7f55f9c700 1 --2- 172.21.15.49:0/221740964 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f3c005250 0x7f7f3c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.290+0000 7f7f5579b700 1 -- 172.21.15.49:0/221740964 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f3c005250 msgr2=0x7f7f3c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:24.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.290+0000 7f7f5579b700 1 --2- 172.21.15.49:0/221740964 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f3c005250 0x7f7f3c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.290+0000 7f7f5579b700 1 -- 172.21.15.49:0/221740964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f3c004850 msgr2=0x7f7f3c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:24.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.290+0000 7f7f5579b700 1 --2- 172.21.15.49:0/221740964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f3c004850 0x7f7f3c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.290+0000 7f7f5579b700 1 -- 172.21.15.49:0/221740964 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c0035b0 con 0x7f7f3c003be0 2023-07-18T03:40:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.290+0000 7f7f54f9a700 1 --2- 172.21.15.49:0/221740964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f3c004850 0x7f7f3c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.290+0000 7f7f55f9c700 1 -- 172.21.15.49:0/221740964 reap_dead start 2023-07-18T03:40:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.290+0000 7f7f5579b700 1 --2- 172.21.15.49:0/221740964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 0x7f7f3c003fd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c001eb0 tx=0x7f7f3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c709da8993cd5be server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.290+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00cc70 con 0x7f7f3c003be0 2023-07-18T03:40:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.290+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c00cdf0 con 0x7f7f3c003be0 2023-07-18T03:40:24.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.290+0000 7f7f47fff700 1 -- 172.21.15.49:0/221740964 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c005ba0 con 0x7f7f3c003be0 2023-07-18T03:40:24.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.291+0000 7f7f579ff700 1 -- 172.21.15.49:0/221740964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 msgr2=0x7f7f3c003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.291+0000 7f7f579ff700 1 --2- 172.21.15.49:0/221740964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 0x7f7f3c003fd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c001eb0 tx=0x7f7f3c005880 comp rx=0 tx=0).stop 2023-07-18T03:40:24.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.291+0000 7f7f579ff700 1 -- 172.21.15.49:0/221740964 shutdown_connections 2023-07-18T03:40:24.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.291+0000 7f7f579ff700 1 --2- 172.21.15.49:0/221740964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 0x7f7f3c003fd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.291+0000 7f7f579ff700 1 -- 172.21.15.49:0/221740964 >> 172.21.15.49:0/221740964 conn(0x7f7f500b05d0 msgr2=0x7f7f500b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:24.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.291+0000 7f7f579ff700 1 -- 172.21.15.49:0/221740964 shutdown_connections 2023-07-18T03:40:24.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.291+0000 7f7f579ff700 1 -- 172.21.15.49:0/221740964 wait complete. 2023-07-18T03:40:24.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.293+0000 7f7f579ff700 1 Processor -- start 2023-07-18T03:40:24.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.293+0000 7f7f579ff700 1 -- start start 2023-07-18T03:40:24.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.294+0000 7f7f579ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 0x7f7f5006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.294+0000 7f7f579ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f50151f00 0x7f7f500c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.294+0000 7f7f579ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f500c8640 0x7f7f500cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.294+0000 7f7f579ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5006b5b0 con 0x7f7f3c003be0 2023-07-18T03:40:24.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.294+0000 7f7f579ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5006b850 con 0x7f7f50151f00 2023-07-18T03:40:24.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.294+0000 7f7f579ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5006bb30 con 0x7f7f500c8640 2023-07-18T03:40:24.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.294+0000 7f7f5579b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 0x7f7f5006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.294+0000 7f7f5579b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 0x7f7f5006c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46750/0 (socket says 172.21.15.49:46750) 2023-07-18T03:40:24.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.294+0000 7f7f5579b700 1 -- 172.21.15.49:0/3228452984 learned_addr learned my addr 172.21.15.49:0/3228452984 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:24.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.294+0000 7f7f55f9c700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f500c8640 0x7f7f500cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.294+0000 7f7f54f9a700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f50151f00 0x7f7f500c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.295+0000 7f7f5579b700 1 -- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f500c8640 msgr2=0x7f7f500cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.295+0000 7f7f5579b700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f500c8640 0x7f7f500cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.295+0000 7f7f5579b700 1 -- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f50151f00 msgr2=0x7f7f500c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.295+0000 7f7f5579b700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f50151f00 0x7f7f500c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.295+0000 7f7f5579b700 1 -- 172.21.15.49:0/3228452984 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5006be10 con 0x7f7f3c003be0 2023-07-18T03:40:24.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.295+0000 7f7f54f9a700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f50151f00 0x7f7f500c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:24.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.295+0000 7f7f5579b700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 0x7f7f5006c8f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00c130 tx=0x7f7f4c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:24.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.295+0000 7f7f45ffb700 1 -- 172.21.15.49:0/3228452984 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c004c80 con 0x7f7f3c003be0 2023-07-18T03:40:24.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.295+0000 7f7f45ffb700 1 -- 172.21.15.49:0/3228452984 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c004de0 con 0x7f7f3c003be0 2023-07-18T03:40:24.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.295+0000 7f7f579ff700 1 -- 172.21.15.49:0/3228452984 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5006c0f0 con 0x7f7f3c003be0 2023-07-18T03:40:24.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.295+0000 7f7f45ffb700 1 -- 172.21.15.49:0/3228452984 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0072f0 con 0x7f7f3c003be0 2023-07-18T03:40:24.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.295+0000 7f7f579ff700 1 -- 172.21.15.49:0/3228452984 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5006ce30 con 0x7f7f3c003be0 2023-07-18T03:40:24.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.296+0000 7f7f579ff700 1 -- 172.21.15.49:0/3228452984 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5006c740 con 0x7f7f3c003be0 2023-07-18T03:40:24.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.297+0000 7f7f45ffb700 1 -- 172.21.15.49:0/3228452984 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f7f4c007450 con 0x7f7f3c003be0 2023-07-18T03:40:24.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.298+0000 7f7f45ffb700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7f3406ad70 0x7f7f3406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.298+0000 7f7f45ffb700 1 -- 172.21.15.49:0/3228452984 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f7f4c089ec0 con 0x7f7f3c003be0 2023-07-18T03:40:24.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.299+0000 7f7f54f9a700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7f3406ad70 0x7f7f3406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.299+0000 7f7f54f9a700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7f3406ad70 0x7f7f3406d220 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7f500cb4c0 tx=0x7f7f40008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:24.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.302+0000 7f7f45ffb700 1 -- 172.21.15.49:0/3228452984 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7f4c054e70 con 0x7f7f3c003be0 2023-07-18T03:40:24.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.507+0000 7f7f579ff700 1 -- 172.21.15.49:0/3228452984 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f500c8a10 con 0x7f7f3c003be0 2023-07-18T03:40:24.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.508+0000 7f7f45ffb700 1 -- 172.21.15.49:0/3228452984 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f4c0170a0 con 0x7f7f3c003be0 2023-07-18T03:40:24.503 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:40:24.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.510+0000 7f7f579ff700 1 -- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7f3406ad70 msgr2=0x7f7f3406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.510+0000 7f7f579ff700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7f3406ad70 0x7f7f3406d220 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7f500cb4c0 tx=0x7f7f40008040 comp rx=0 tx=0).stop 2023-07-18T03:40:24.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.510+0000 7f7f579ff700 1 -- 172.21.15.49:0/3228452984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 msgr2=0x7f7f5006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.510+0000 7f7f579ff700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 0x7f7f5006c8f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00c130 tx=0x7f7f4c001930 comp rx=0 tx=0).stop 2023-07-18T03:40:24.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.510+0000 7f7f579ff700 1 -- 172.21.15.49:0/3228452984 shutdown_connections 2023-07-18T03:40:24.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.510+0000 7f7f579ff700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f500c8640 0x7f7f500cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.510+0000 7f7f579ff700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7f3406ad70 0x7f7f3406d220 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.510+0000 7f7f579ff700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c003be0 0x7f7f5006c8f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.510+0000 7f7f579ff700 1 --2- 172.21.15.49:0/3228452984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f50151f00 0x7f7f500c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.510+0000 7f7f579ff700 1 -- 172.21.15.49:0/3228452984 >> 172.21.15.49:0/3228452984 conn(0x7f7f500b05d0 msgr2=0x7f7f50070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:24.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.510+0000 7f7f579ff700 1 -- 172.21.15.49:0/3228452984 shutdown_connections 2023-07-18T03:40:24.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.510+0000 7f7f579ff700 1 -- 172.21.15.49:0/3228452984 wait complete. 2023-07-18T03:40:24.522 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:40:24.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.735+0000 7fe2123ff700 1 Processor -- start 2023-07-18T03:40:24.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.735+0000 7fe2123ff700 1 -- start start 2023-07-18T03:40:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.736+0000 7fe2123ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20c0b4510 0x7fe20c0b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.736+0000 7fe2123ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe20c0d0b10 0x7fe20c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.736+0000 7fe2123ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20c0b7f50 con 0x7fe20c0b5d90 2023-07-18T03:40:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.736+0000 7fe2123ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20c0b48e0 con 0x7fe20c0d0b10 2023-07-18T03:40:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.736+0000 7fe2123ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0d14d0 con 0x7fe20c0b4510 2023-07-18T03:40:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.737+0000 7fe21099c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe20c0d0b10 0x7fe20c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.737+0000 7fe21099c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe20c0d0b10 0x7fe20c0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46764/0 (socket says 172.21.15.49:46764) 2023-07-18T03:40:24.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.737+0000 7fe21099c700 1 -- 172.21.15.49:0/1363256764 learned_addr learned my addr 172.21.15.49:0/1363256764 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:24.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.737+0000 7fe20bfff700 1 --2- 172.21.15.49:0/1363256764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20c0b4510 0x7fe20c0b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.737+0000 7fe21099c700 1 -- 172.21.15.49:0/1363256764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20c0b4510 msgr2=0x7fe20c0b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:24.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.737+0000 7fe21099c700 1 --2- 172.21.15.49:0/1363256764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20c0b4510 0x7fe20c0b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.737+0000 7fe21099c700 1 -- 172.21.15.49:0/1363256764 >> v1:172.21.15.107:6789/0 conn(0x7fe20c0b5d90 legacy=0x7fe20c0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.737+0000 7fe21099c700 1 -- 172.21.15.49:0/1363256764 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c0cdca0 con 0x7fe20c0d0b10 2023-07-18T03:40:24.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.737+0000 7fe21099c700 1 --2- 172.21.15.49:0/1363256764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe20c0d0b10 0x7fe20c0d0f90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe204009e50 tx=0x7fe20400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119127cadb52b55a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:24.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.738+0000 7fe20affd700 1 -- 172.21.15.49:0/1363256764 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20400e070 con 0x7fe20c0d0b10 2023-07-18T03:40:24.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.738+0000 7fe20affd700 1 -- 172.21.15.49:0/1363256764 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20400b9c0 con 0x7fe20c0d0b10 2023-07-18T03:40:24.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.738+0000 7fe20affd700 1 -- 172.21.15.49:0/1363256764 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20400bd70 con 0x7fe20c0d0b10 2023-07-18T03:40:24.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.738+0000 7fe2123ff700 1 -- 172.21.15.49:0/1363256764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe20c0d0b10 msgr2=0x7fe20c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.738+0000 7fe2123ff700 1 --2- 172.21.15.49:0/1363256764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe20c0d0b10 0x7fe20c0d0f90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe204009e50 tx=0x7fe20400b330 comp rx=0 tx=0).stop 2023-07-18T03:40:24.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.738+0000 7fe2123ff700 1 -- 172.21.15.49:0/1363256764 shutdown_connections 2023-07-18T03:40:24.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.738+0000 7fe2123ff700 1 --2- 172.21.15.49:0/1363256764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe20c0d0b10 0x7fe20c0d0f90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.738+0000 7fe2123ff700 1 --2- 172.21.15.49:0/1363256764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20c0b4510 0x7fe20c0b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.738+0000 7fe2123ff700 1 -- 172.21.15.49:0/1363256764 >> 172.21.15.49:0/1363256764 conn(0x7fe20c09c1d0 msgr2=0x7fe20c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:24.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.738+0000 7fe2123ff700 1 -- 172.21.15.49:0/1363256764 shutdown_connections 2023-07-18T03:40:24.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.738+0000 7fe2123ff700 1 -- 172.21.15.49:0/1363256764 wait complete. 2023-07-18T03:40:24.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.740+0000 7fe2123ff700 1 Processor -- start 2023-07-18T03:40:24.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.740+0000 7fe2123ff700 1 -- start start 2023-07-18T03:40:24.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.741+0000 7fe2123ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20c0b4510 0x7fe20c0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.741+0000 7fe2123ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20c0b5d90 0x7fe20c0c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.741+0000 7fe2123ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe20c0d0b10 0x7fe20c06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.741+0000 7fe2123ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0c80d0 con 0x7fe20c0d0b10 2023-07-18T03:40:24.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.741+0000 7fe2123ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0cd060 con 0x7fe20c0b5d90 2023-07-18T03:40:24.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.741+0000 7fe2123ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0c74c0 con 0x7fe20c0b4510 2023-07-18T03:40:24.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.741+0000 7fe21099c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe20c0d0b10 0x7fe20c06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.742+0000 7fe21099c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe20c0d0b10 0x7fe20c06b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46776/0 (socket says 172.21.15.49:46776) 2023-07-18T03:40:24.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.742+0000 7fe21099c700 1 -- 172.21.15.49:0/4049225868 learned_addr learned my addr 172.21.15.49:0/4049225868 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:24.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.742+0000 7fe20bfff700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20c0b4510 0x7fe20c0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.742+0000 7fe20b7fe700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20c0b5d90 0x7fe20c0c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.742+0000 7fe21099c700 1 -- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20c0b4510 msgr2=0x7fe20c0ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.742+0000 7fe21099c700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20c0b4510 0x7fe20c0ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.742+0000 7fe21099c700 1 -- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20c0b5d90 msgr2=0x7fe20c0c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:24.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.742+0000 7fe21099c700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20c0b5d90 0x7fe20c0c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:24.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.742+0000 7fe21099c700 1 -- 172.21.15.49:0/4049225868 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204009b30 con 0x7fe20c0d0b10 2023-07-18T03:40:24.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.742+0000 7fe21099c700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe20c0d0b10 0x7fe20c06b7d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe20c067360 tx=0x7fe204004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:24.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.742+0000 7fe208ff9700 1 -- 172.21.15.49:0/4049225868 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040045f0 con 0x7fe20c0d0b10 2023-07-18T03:40:24.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.743+0000 7fe208ff9700 1 -- 172.21.15.49:0/4049225868 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe204004750 con 0x7fe20c0d0b10 2023-07-18T03:40:24.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.743+0000 7fe2123ff700 1 -- 172.21.15.49:0/4049225868 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20c0c77a0 con 0x7fe20c0d0b10 2023-07-18T03:40:24.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.743+0000 7fe208ff9700 1 -- 172.21.15.49:0/4049225868 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040092e0 con 0x7fe20c0d0b10 2023-07-18T03:40:24.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.743+0000 7fe2123ff700 1 -- 172.21.15.49:0/4049225868 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20c0c7d60 con 0x7fe20c0d0b10 2023-07-18T03:40:24.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.744+0000 7fe2123ff700 1 -- 172.21.15.49:0/4049225868 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20c0c8280 con 0x7fe20c0d0b10 2023-07-18T03:40:24.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.745+0000 7fe208ff9700 1 -- 172.21.15.49:0/4049225868 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fe20401c410 con 0x7fe20c0d0b10 2023-07-18T03:40:24.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.746+0000 7fe208ff9700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1f806ad70 0x7fe1f806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:24.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.746+0000 7fe208ff9700 1 -- 172.21.15.49:0/4049225868 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fe20408b4d0 con 0x7fe20c0d0b10 2023-07-18T03:40:24.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.746+0000 7fe20bfff700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1f806ad70 0x7fe1f806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:24.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.747+0000 7fe20bfff700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1f806ad70 0x7fe1f806d220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc006fd0 tx=0x7fe1fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:24.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:24.750+0000 7fe208ff9700 1 -- 172.21.15.49:0/4049225868 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe20c0c8280 con 0x7fe20c0d0b10 2023-07-18T03:40:24.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.003+0000 7fe2123ff700 1 -- 172.21.15.49:0/4049225868 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe20c04e280 con 0x7fe20c0d0b10 2023-07-18T03:40:24.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.004+0000 7fe208ff9700 1 -- 172.21.15.49:0/4049225868 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fe2040564a0 con 0x7fe20c0d0b10 2023-07-18T03:40:24.999 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:24.999 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1095518881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1095518881},{"type":"v1","addr":"172.21.15.49:6834","nonce":1095518881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/843626338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":843626338},{"type":"v1","addr":"172.21.15.107:6835","nonce":843626338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3238680863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3238680863},{"type":"v1","addr":"172.21.15.107:6837","nonce":3238680863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:39:56.844870+0000","modified":"2023-07-18T03:40:00.356383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5172":{"gid":5172,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6835/3161575771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":3161575771},{"type":"v1","addr":"172.21.15.49:6835","nonce":3161575771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-18T03:40:25.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.007+0000 7fe2123ff700 1 -- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1f806ad70 msgr2=0x7fe1f806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.007+0000 7fe2123ff700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1f806ad70 0x7fe1f806d220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc006fd0 tx=0x7fe1fc008040 comp rx=0 tx=0).stop 2023-07-18T03:40:25.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.007+0000 7fe2123ff700 1 -- 172.21.15.49:0/4049225868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe20c0d0b10 msgr2=0x7fe20c06b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.007+0000 7fe2123ff700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe20c0d0b10 0x7fe20c06b7d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe20c067360 tx=0x7fe204004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.007+0000 7fe2123ff700 1 -- 172.21.15.49:0/4049225868 shutdown_connections 2023-07-18T03:40:25.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.007+0000 7fe2123ff700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe1f806ad70 0x7fe1f806d220 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.007+0000 7fe2123ff700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe20c0d0b10 0x7fe20c06b7d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.007+0000 7fe2123ff700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20c0b4510 0x7fe20c0ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.007+0000 7fe2123ff700 1 --2- 172.21.15.49:0/4049225868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20c0b5d90 0x7fe20c0c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.007+0000 7fe2123ff700 1 -- 172.21.15.49:0/4049225868 >> 172.21.15.49:0/4049225868 conn(0x7fe20c09c1d0 msgr2=0x7fe20c0ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:25.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.007+0000 7fe2123ff700 1 -- 172.21.15.49:0/4049225868 shutdown_connections 2023-07-18T03:40:25.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.007+0000 7fe2123ff700 1 -- 172.21.15.49:0/4049225868 wait complete. 2023-07-18T03:40:25.003 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 19 2023-07-18T03:40:25.019 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.231+0000 7f202a3ff700 1 Processor -- start 2023-07-18T03:40:25.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.231+0000 7f202a3ff700 1 -- start start 2023-07-18T03:40:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.233+0000 7f202a3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240d1220 0x7f20240b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.233+0000 7f202a3ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f20240b1e00 0x7f20240afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.233+0000 7f202a3ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20240d2ce0 con 0x7f202415c950 2023-07-18T03:40:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.233+0000 7f202a3ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20240b3350 con 0x7f20240b1e00 2023-07-18T03:40:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.233+0000 7f202a3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20240b3630 con 0x7f20240d1220 2023-07-18T03:40:25.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.233+0000 7f202899c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f20240b1e00 0x7f20240afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:25.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.233+0000 7f2023fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f202415c950 0x7f20240d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36144/0 (socket says 172.21.15.49:36144) 2023-07-18T03:40:25.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.233+0000 7f2023fff700 1 -- 172.21.15.49:0/2620190000 learned_addr learned my addr 172.21.15.49:0/2620190000 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:25.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.233+0000 7f201bfff700 1 --2- 172.21.15.49:0/2620190000 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240d1220 0x7f20240b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:25.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.233+0000 7f202899c700 1 -- 172.21.15.49:0/2620190000 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240d1220 msgr2=0x7f20240b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:25.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.233+0000 7f202899c700 1 --2- 172.21.15.49:0/2620190000 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240d1220 0x7f20240b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.233+0000 7f202899c700 1 -- 172.21.15.49:0/2620190000 >> v1:172.21.15.49:6789/0 conn(0x7f202415c950 legacy=0x7f20240d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.234+0000 7f202899c700 1 -- 172.21.15.49:0/2620190000 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20240b3910 con 0x7f20240b1e00 2023-07-18T03:40:25.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.234+0000 7f20237fe700 1 -- 172.21.15.49:0/2620190000 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875065441 0 0) 0x7f20240d2ce0 con 0x7f202415c950 2023-07-18T03:40:25.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.234+0000 7f202899c700 1 --2- 172.21.15.49:0/2620190000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20240b1e00 0x7f20240afac0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f201c009d90 tx=0x7f201c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24ba83180f4200 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:25.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.234+0000 7f20237fe700 1 -- 172.21.15.49:0/2620190000 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00e070 con 0x7f20240b1e00 2023-07-18T03:40:25.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.234+0000 7f20237fe700 1 -- 172.21.15.49:0/2620190000 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c00bbd0 con 0x7f20240b1e00 2023-07-18T03:40:25.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.234+0000 7f20237fe700 1 -- 172.21.15.49:0/2620190000 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c012430 con 0x7f20240b1e00 2023-07-18T03:40:25.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.235+0000 7f202a3ff700 1 -- 172.21.15.49:0/2620190000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20240b1e00 msgr2=0x7f20240afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.235+0000 7f202a3ff700 1 --2- 172.21.15.49:0/2620190000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20240b1e00 0x7f20240afac0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f201c009d90 tx=0x7f201c00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:25.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.235+0000 7f202a3ff700 1 -- 172.21.15.49:0/2620190000 shutdown_connections 2023-07-18T03:40:25.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.235+0000 7f202a3ff700 1 --2- 172.21.15.49:0/2620190000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20240b1e00 0x7f20240afac0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.235+0000 7f202a3ff700 1 --2- 172.21.15.49:0/2620190000 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240d1220 0x7f20240b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.235+0000 7f202a3ff700 1 -- 172.21.15.49:0/2620190000 >> 172.21.15.49:0/2620190000 conn(0x7f202409c1d0 msgr2=0x7f20240d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:25.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.235+0000 7f202a3ff700 1 -- 172.21.15.49:0/2620190000 shutdown_connections 2023-07-18T03:40:25.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.235+0000 7f202a3ff700 1 -- 172.21.15.49:0/2620190000 wait complete. 2023-07-18T03:40:25.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.237+0000 7f202a3ff700 1 Processor -- start 2023-07-18T03:40:25.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.237+0000 7f202a3ff700 1 -- start start 2023-07-18T03:40:25.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f202a3ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20240b1e00 0x7f202414f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:25.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f202a3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240d1220 0x7f20240c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:25.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f202a3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f202415c950 0x7f20240c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:25.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f202a3ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20240cf000 con 0x7f20240b1e00 2023-07-18T03:40:25.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f202a3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20240cea90 con 0x7f202415c950 2023-07-18T03:40:25.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f202a3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20240b6b60 con 0x7f20240d1220 2023-07-18T03:40:25.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f2023fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20240b1e00 0x7f202414f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:25.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f2023fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20240b1e00 0x7f202414f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46794/0 (socket says 172.21.15.49:46794) 2023-07-18T03:40:25.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f2023fff700 1 -- 172.21.15.49:0/3401033435 learned_addr learned my addr 172.21.15.49:0/3401033435 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:25.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f201bfff700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240d1220 0x7f20240c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:25.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f202899c700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f202415c950 0x7f20240c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:25.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f2023fff700 1 -- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240d1220 msgr2=0x7f20240c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f2023fff700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240d1220 0x7f20240c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f2023fff700 1 -- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f202415c950 msgr2=0x7f20240c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f2023fff700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f202415c950 0x7f20240c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.238+0000 7f2023fff700 1 -- 172.21.15.49:0/3401033435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c009a70 con 0x7f20240b1e00 2023-07-18T03:40:25.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.239+0000 7f2023fff700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20240b1e00 0x7f202414f650 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f201400c4a0 tx=0x7f201400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:25.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.239+0000 7f20217fa700 1 -- 172.21.15.49:0/3401033435 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20140074e0 con 0x7f20240b1e00 2023-07-18T03:40:25.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.239+0000 7f20217fa700 1 -- 172.21.15.49:0/3401033435 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2014007640 con 0x7f20240b1e00 2023-07-18T03:40:25.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.239+0000 7f20217fa700 1 -- 172.21.15.49:0/3401033435 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014005a70 con 0x7f20240b1e00 2023-07-18T03:40:25.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.239+0000 7f202a3ff700 1 -- 172.21.15.49:0/3401033435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202414fb90 con 0x7f20240b1e00 2023-07-18T03:40:25.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.239+0000 7f202a3ff700 1 -- 172.21.15.49:0/3401033435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20240ce4d0 con 0x7f20240b1e00 2023-07-18T03:40:25.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.240+0000 7f202a3ff700 1 -- 172.21.15.49:0/3401033435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202404e280 con 0x7f20240b1e00 2023-07-18T03:40:25.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.241+0000 7f20217fa700 1 -- 172.21.15.49:0/3401033435 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f2014005bd0 con 0x7f20240b1e00 2023-07-18T03:40:25.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.242+0000 7f20217fa700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f200406ad70 0x7f200406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:25.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.242+0000 7f20217fa700 1 -- 172.21.15.49:0/3401033435 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f2014088a10 con 0x7f20240b1e00 2023-07-18T03:40:25.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.243+0000 7f201bfff700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f200406ad70 0x7f200406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:25.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.243+0000 7f201bfff700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f200406ad70 0x7f200406d220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f20240b2980 tx=0x7f200c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:25.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.247+0000 7f20217fa700 1 -- 172.21.15.49:0/3401033435 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2014053a30 con 0x7f20240b1e00 2023-07-18T03:40:25.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.457+0000 7f202a3ff700 1 -- 172.21.15.49:0/3401033435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f202408c5a0 con 0x7f20240b1e00 2023-07-18T03:40:25.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.458+0000 7f20217fa700 1 -- 172.21.15.49:0/3401033435 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18249 (secure 0 0 0) 0x7f2014057050 con 0x7f20240b1e00 2023-07-18T03:40:25.453 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:25.453 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":38,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:40:23.979978+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:39:54.412334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-18T03:39:55.841218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:25.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.460+0000 7f202a3ff700 1 -- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f200406ad70 msgr2=0x7f200406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.460+0000 7f202a3ff700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f200406ad70 0x7f200406d220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f20240b2980 tx=0x7f200c008040 comp rx=0 tx=0).stop 2023-07-18T03:40:25.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.460+0000 7f202a3ff700 1 -- 172.21.15.49:0/3401033435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20240b1e00 msgr2=0x7f202414f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.460+0000 7f202a3ff700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20240b1e00 0x7f202414f650 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f201400c4a0 tx=0x7f201400c860 comp rx=0 tx=0).stop 2023-07-18T03:40:25.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.460+0000 7f202a3ff700 1 -- 172.21.15.49:0/3401033435 shutdown_connections 2023-07-18T03:40:25.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.460+0000 7f202a3ff700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f202415c950 0x7f20240c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.460+0000 7f202a3ff700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f200406ad70 0x7f200406d220 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.461+0000 7f202a3ff700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20240b1e00 0x7f202414f650 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.461+0000 7f202a3ff700 1 --2- 172.21.15.49:0/3401033435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240d1220 0x7f20240c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.461+0000 7f202a3ff700 1 -- 172.21.15.49:0/3401033435 >> 172.21.15.49:0/3401033435 conn(0x7f202409c1d0 msgr2=0x7f20240b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:25.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.461+0000 7f202a3ff700 1 -- 172.21.15.49:0/3401033435 shutdown_connections 2023-07-18T03:40:25.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.461+0000 7f202a3ff700 1 -- 172.21.15.49:0/3401033435 wait complete. 2023-07-18T03:40:25.474 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-18T03:40:25.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:40:25.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.685+0000 7f0459dff700 1 Processor -- start 2023-07-18T03:40:25.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.685+0000 7f0459dff700 1 -- start start 2023-07-18T03:40:25.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.686+0000 7f0459dff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f04541500d0 0x7f0454070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:25.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.686+0000 7f0459dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f04540cea50 0x7f04540ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:25.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.686+0000 7f0459dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04540cdf90 con 0x7f0454070890 2023-07-18T03:40:25.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.686+0000 7f0459dff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f045415a2f0 con 0x7f04541500d0 2023-07-18T03:40:25.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.686+0000 7f0459dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f045415a5d0 con 0x7f04540cea50 2023-07-18T03:40:25.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.686+0000 7f04537fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f04541500d0 0x7f0454070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:25.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.687+0000 7f04537fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f04541500d0 0x7f0454070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46808/0 (socket says 172.21.15.49:46808) 2023-07-18T03:40:25.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.687+0000 7f04537fe700 1 -- 172.21.15.49:0/857181205 learned_addr learned my addr 172.21.15.49:0/857181205 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:25.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.687+0000 7f0452ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0454070890 0x7f04541597c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36150/0 (socket says 172.21.15.49:36150) 2023-07-18T03:40:25.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.687+0000 7f0453fff700 1 --2- 172.21.15.49:0/857181205 >> v2:172.21.15.107:3300/0 conn(0x7f04540cea50 0x7f04540ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:25.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.687+0000 7f04537fe700 1 -- 172.21.15.49:0/857181205 >> v2:172.21.15.107:3300/0 conn(0x7f04540cea50 msgr2=0x7f04540ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:25.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.687+0000 7f04537fe700 1 --2- 172.21.15.49:0/857181205 >> v2:172.21.15.107:3300/0 conn(0x7f04540cea50 0x7f04540ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.687+0000 7f04537fe700 1 -- 172.21.15.49:0/857181205 >> v1:172.21.15.49:6789/0 conn(0x7f0454070890 legacy=0x7f04541597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.687+0000 7f04537fe700 1 -- 172.21.15.49:0/857181205 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04540ce250 con 0x7f04541500d0 2023-07-18T03:40:25.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.687+0000 7f04527fc700 1 -- 172.21.15.49:0/857181205 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672753015 0 0) 0x7f04540cdf90 con 0x7f0454070890 2023-07-18T03:40:25.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.687+0000 7f04537fe700 1 --2- 172.21.15.49:0/857181205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04541500d0 0x7f0454070350 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f045406f670 tx=0x7f0444009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a7462a29448888 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.688+0000 7f04527fc700 1 -- 172.21.15.49:0/857181205 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444011830 con 0x7f04541500d0 2023-07-18T03:40:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.688+0000 7f04527fc700 1 -- 172.21.15.49:0/857181205 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0444011990 con 0x7f04541500d0 2023-07-18T03:40:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.688+0000 7f04527fc700 1 -- 172.21.15.49:0/857181205 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444011d20 con 0x7f04541500d0 2023-07-18T03:40:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.688+0000 7f0459dff700 1 -- 172.21.15.49:0/857181205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04541500d0 msgr2=0x7f0454070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.688+0000 7f0459dff700 1 --2- 172.21.15.49:0/857181205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04541500d0 0x7f0454070350 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f045406f670 tx=0x7f0444009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.688+0000 7f0459dff700 1 -- 172.21.15.49:0/857181205 shutdown_connections 2023-07-18T03:40:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.688+0000 7f0459dff700 1 --2- 172.21.15.49:0/857181205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04541500d0 0x7f0454070350 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.688+0000 7f0459dff700 1 --2- 172.21.15.49:0/857181205 >> v2:172.21.15.107:3300/0 conn(0x7f04540cea50 0x7f04540ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.688+0000 7f0459dff700 1 -- 172.21.15.49:0/857181205 >> 172.21.15.49:0/857181205 conn(0x7f0454069f30 msgr2=0x7f045406a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:25.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.688+0000 7f0459dff700 1 -- 172.21.15.49:0/857181205 shutdown_connections 2023-07-18T03:40:25.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.688+0000 7f0459dff700 1 -- 172.21.15.49:0/857181205 wait complete. 2023-07-18T03:40:25.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.690+0000 7f0459dff700 1 Processor -- start 2023-07-18T03:40:25.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.690+0000 7f0459dff700 1 -- start start 2023-07-18T03:40:25.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.691+0000 7f0459dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454070890 0x7f045413bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:25.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.691+0000 7f0459dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04540cea50 0x7f04541e4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:25.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.691+0000 7f0459dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04541500d0 0x7f0454151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:25.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.691+0000 7f0459dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0454151e60 con 0x7f04540cea50 2023-07-18T03:40:25.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.691+0000 7f0459dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0454152140 con 0x7f0454070890 2023-07-18T03:40:25.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.691+0000 7f0459dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0454152420 con 0x7f04541500d0 2023-07-18T03:40:25.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.691+0000 7f0452ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04540cea50 0x7f04541e4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:25.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.692+0000 7f0452ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04540cea50 0x7f04541e4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46816/0 (socket says 172.21.15.49:46816) 2023-07-18T03:40:25.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.692+0000 7f0453fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04541500d0 0x7f0454151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:25.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.692+0000 7f04537fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454070890 0x7f045413bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:25.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.692+0000 7f0452ffd700 1 -- 172.21.15.49:0/1633553528 learned_addr learned my addr 172.21.15.49:0/1633553528 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:25.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.692+0000 7f0452ffd700 1 -- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04541500d0 msgr2=0x7f0454151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.692+0000 7f0452ffd700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04541500d0 0x7f0454151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.692+0000 7f0452ffd700 1 -- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454070890 msgr2=0x7f045413bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.692+0000 7f0452ffd700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454070890 0x7f045413bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.692+0000 7f0452ffd700 1 -- 172.21.15.49:0/1633553528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0444009550 con 0x7f04540cea50 2023-07-18T03:40:25.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.692+0000 7f04537fe700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454070890 0x7f045413bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:25.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.692+0000 7f0452ffd700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04540cea50 0x7f04541e4d00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f04480011e0 tx=0x7f044800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:25.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.692+0000 7f043bfff700 1 -- 172.21.15.49:0/1633553528 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0448004c80 con 0x7f04540cea50 2023-07-18T03:40:25.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.693+0000 7f043bfff700 1 -- 172.21.15.49:0/1633553528 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0448004e00 con 0x7f04540cea50 2023-07-18T03:40:25.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.693+0000 7f043bfff700 1 -- 172.21.15.49:0/1633553528 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0448007310 con 0x7f04540cea50 2023-07-18T03:40:25.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.693+0000 7f0459dff700 1 -- 172.21.15.49:0/1633553528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0454152700 con 0x7f04540cea50 2023-07-18T03:40:25.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.693+0000 7f0459dff700 1 -- 172.21.15.49:0/1633553528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04541e5240 con 0x7f04540cea50 2023-07-18T03:40:25.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.694+0000 7f0459dff700 1 -- 172.21.15.49:0/1633553528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045404e280 con 0x7f04540cea50 2023-07-18T03:40:25.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.695+0000 7f043bfff700 1 -- 172.21.15.49:0/1633553528 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f0448007470 con 0x7f04540cea50 2023-07-18T03:40:25.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.696+0000 7f043bfff700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f044006ad70 0x7f044006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:25.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.696+0000 7f043bfff700 1 -- 172.21.15.49:0/1633553528 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f04480891d0 con 0x7f04540cea50 2023-07-18T03:40:25.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.696+0000 7f04537fe700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f044006ad70 0x7f044006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:25.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.697+0000 7f04537fe700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f044006ad70 0x7f044006d220 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0444014040 tx=0x7f0444008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:25.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.700+0000 7f043bfff700 1 -- 172.21.15.49:0/1633553528 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0448054170 con 0x7f04540cea50 2023-07-18T03:40:25.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.952+0000 7f0459dff700 1 -- 172.21.15.49:0/1633553528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0454070130 con 0x7f04540cea50 2023-07-18T03:40:25.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.953+0000 7f043bfff700 1 -- 172.21.15.49:0/1633553528 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f0448057790 con 0x7f04540cea50 2023-07-18T03:40:25.947 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:25.947 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1095518881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1095518881},{"type":"v1","addr":"172.21.15.49:6834","nonce":1095518881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/843626338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":843626338},{"type":"v1","addr":"172.21.15.107:6835","nonce":843626338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3238680863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3238680863},{"type":"v1","addr":"172.21.15.107:6837","nonce":3238680863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:39:56.844870+0000","modified":"2023-07-18T03:40:00.356383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5172":{"gid":5172,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6835/3161575771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":3161575771},{"type":"v1","addr":"172.21.15.49:6835","nonce":3161575771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-18T03:40:25.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.955+0000 7f0459dff700 1 -- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f044006ad70 msgr2=0x7f044006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.955+0000 7f0459dff700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f044006ad70 0x7f044006d220 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0444014040 tx=0x7f0444008030 comp rx=0 tx=0).stop 2023-07-18T03:40:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.955+0000 7f0459dff700 1 -- 172.21.15.49:0/1633553528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04540cea50 msgr2=0x7f04541e4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.955+0000 7f0459dff700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04540cea50 0x7f04541e4d00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f04480011e0 tx=0x7f044800ca30 comp rx=0 tx=0).stop 2023-07-18T03:40:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.955+0000 7f0459dff700 1 -- 172.21.15.49:0/1633553528 shutdown_connections 2023-07-18T03:40:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.955+0000 7f0459dff700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04541500d0 0x7f0454151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.955+0000 7f0459dff700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454070890 0x7f045413bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.955+0000 7f0459dff700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f044006ad70 0x7f044006d220 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.955+0000 7f0459dff700 1 --2- 172.21.15.49:0/1633553528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04540cea50 0x7f04541e4d00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:25.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.955+0000 7f0459dff700 1 -- 172.21.15.49:0/1633553528 >> 172.21.15.49:0/1633553528 conn(0x7f0454069f30 msgr2=0x7f04540ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:25.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.955+0000 7f0459dff700 1 -- 172.21.15.49:0/1633553528 shutdown_connections 2023-07-18T03:40:25.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:25.955+0000 7f0459dff700 1 -- 172.21.15.49:0/1633553528 wait complete. 2023-07-18T03:40:25.951 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 19 2023-07-18T03:40:25.968 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:40:26.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.179+0000 7fafc9fff700 1 Processor -- start 2023-07-18T03:40:26.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.180+0000 7fafc9fff700 1 -- start start 2023-07-18T03:40:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.181+0000 7fafc9fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fafc415c950 0x7fafc40b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.181+0000 7fafc9fff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc40d2ce0 con 0x7fafc40d1220 2023-07-18T03:40:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.181+0000 7fafc9fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc40b3350 con 0x7fafc40b1e00 2023-07-18T03:40:26.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.181+0000 7fafc9fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc40b3630 con 0x7fafc415c950 2023-07-18T03:40:26.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.181+0000 7fafc37fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fafc415c950 0x7fafc40b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:26.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.181+0000 7fafc3fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fafc40d1220 0x7fafc40afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36152/0 (socket says 172.21.15.49:36152) 2023-07-18T03:40:26.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.181+0000 7fafc3fff700 1 -- 172.21.15.49:0/625498932 learned_addr learned my addr 172.21.15.49:0/625498932 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:26.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.181+0000 7fafc37fe700 1 -- 172.21.15.49:0/625498932 >> v1:172.21.15.107:6789/0 conn(0x7fafc40b1e00 legacy=0x7fafc40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:26.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.182+0000 7fafc37fe700 1 -- 172.21.15.49:0/625498932 >> v1:172.21.15.49:6789/0 conn(0x7fafc40d1220 legacy=0x7fafc40afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:26.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.182+0000 7fafc37fe700 1 -- 172.21.15.49:0/625498932 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40b3910 con 0x7fafc415c950 2023-07-18T03:40:26.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.182+0000 7fafc2ffd700 1 -- 172.21.15.49:0/625498932 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888593206 0 0) 0x7fafc40d2ce0 con 0x7fafc40d1220 2023-07-18T03:40:26.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.182+0000 7fafc37fe700 1 --2- 172.21.15.49:0/625498932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc415c950 0x7fafc40b43f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009820 tx=0x7fafb4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b3e9f58ebd98a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:26.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.182+0000 7fafc2ffd700 1 -- 172.21.15.49:0/625498932 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb4005e30 con 0x7fafc415c950 2023-07-18T03:40:26.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.182+0000 7fafc2ffd700 1 -- 172.21.15.49:0/625498932 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb4006030 con 0x7fafc415c950 2023-07-18T03:40:26.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.182+0000 7fafc2ffd700 1 -- 172.21.15.49:0/625498932 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb40063e0 con 0x7fafc415c950 2023-07-18T03:40:26.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.183+0000 7fafc9fff700 1 -- 172.21.15.49:0/625498932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc415c950 msgr2=0x7fafc40b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:26.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.183+0000 7fafc9fff700 1 --2- 172.21.15.49:0/625498932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc415c950 0x7fafc40b43f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009820 tx=0x7fafb4009b30 comp rx=0 tx=0).stop 2023-07-18T03:40:26.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.183+0000 7fafc9fff700 1 -- 172.21.15.49:0/625498932 shutdown_connections 2023-07-18T03:40:26.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.183+0000 7fafc9fff700 1 --2- 172.21.15.49:0/625498932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc415c950 0x7fafc40b43f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.183+0000 7fafc9fff700 1 -- 172.21.15.49:0/625498932 >> 172.21.15.49:0/625498932 conn(0x7fafc409c1d0 msgr2=0x7fafc40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:26.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.183+0000 7fafc9fff700 1 -- 172.21.15.49:0/625498932 shutdown_connections 2023-07-18T03:40:26.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.183+0000 7fafc9fff700 1 -- 172.21.15.49:0/625498932 wait complete. 2023-07-18T03:40:26.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.185+0000 7fafc9fff700 1 Processor -- start 2023-07-18T03:40:26.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.185+0000 7fafc9fff700 1 -- start start 2023-07-18T03:40:26.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.186+0000 7fafc9fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc40b1e00 0x7fafc4157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:26.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.186+0000 7fafc9fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafc40d1220 0x7fafc40b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:26.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.186+0000 7fafc9fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc415c950 0x7fafc40b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:26.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.186+0000 7fafc9fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc415dab0 con 0x7fafc415c950 2023-07-18T03:40:26.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.186+0000 7fafc9fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc415ce30 con 0x7fafc40d1220 2023-07-18T03:40:26.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.186+0000 7fafc9fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc415d110 con 0x7fafc40b1e00 2023-07-18T03:40:26.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.186+0000 7fafc3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc415c950 0x7fafc40b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:26.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.186+0000 7fafc3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc415c950 0x7fafc40b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46824/0 (socket says 172.21.15.49:46824) 2023-07-18T03:40:26.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.186+0000 7fafc3fff700 1 -- 172.21.15.49:0/528412580 learned_addr learned my addr 172.21.15.49:0/528412580 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:26.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.186+0000 7fafc37fe700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc40b1e00 0x7fafc4157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:26.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.186+0000 7fafbbfff700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafc40d1220 0x7fafc40b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:26.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.187+0000 7fafc3fff700 1 -- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc40b1e00 msgr2=0x7fafc4157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:26.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.187+0000 7fafc3fff700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc40b1e00 0x7fafc4157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.187+0000 7fafc3fff700 1 -- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafc40d1220 msgr2=0x7fafc40b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:26.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.187+0000 7fafc3fff700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafc40d1220 0x7fafc40b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.187+0000 7fafc3fff700 1 -- 172.21.15.49:0/528412580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb4009430 con 0x7fafc415c950 2023-07-18T03:40:26.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.187+0000 7fafc3fff700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc415c950 0x7fafc40b07d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00cc70 tx=0x7fafbc00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:26.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.187+0000 7fafc0ff9700 1 -- 172.21.15.49:0/528412580 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00aaa0 con 0x7fafc415c950 2023-07-18T03:40:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.187+0000 7fafc0ff9700 1 -- 172.21.15.49:0/528412580 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc00ac20 con 0x7fafc415c950 2023-07-18T03:40:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.187+0000 7fafc9fff700 1 -- 172.21.15.49:0/528412580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc415d3f0 con 0x7fafc415c950 2023-07-18T03:40:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.187+0000 7fafc0ff9700 1 -- 172.21.15.49:0/528412580 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc0070d0 con 0x7fafc415c950 2023-07-18T03:40:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.187+0000 7fafc9fff700 1 -- 172.21.15.49:0/528412580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc41578a0 con 0x7fafc415c950 2023-07-18T03:40:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.188+0000 7fafc9fff700 1 -- 172.21.15.49:0/528412580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc415dbf0 con 0x7fafc415c950 2023-07-18T03:40:26.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.189+0000 7fafc0ff9700 1 -- 172.21.15.49:0/528412580 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fafbc007230 con 0x7fafc415c950 2023-07-18T03:40:26.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.190+0000 7fafc0ff9700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fafa406ad70 0x7fafa406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:26.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.191+0000 7fafc0ff9700 1 -- 172.21.15.49:0/528412580 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fafbc089240 con 0x7fafc415c950 2023-07-18T03:40:26.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.191+0000 7fafc37fe700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fafa406ad70 0x7fafa406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:26.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.192+0000 7fafc37fe700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fafa406ad70 0x7fafa406d220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fafb4013040 tx=0x7fafb4008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:26.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.195+0000 7fafc0ff9700 1 -- 172.21.15.49:0/528412580 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fafc415dbf0 con 0x7fafc415c950 2023-07-18T03:40:26.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.448+0000 7fafc9fff700 1 -- 172.21.15.49:0/528412580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafc415dbf0 con 0x7fafc415c950 2023-07-18T03:40:26.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.449+0000 7fafc0ff9700 1 -- 172.21.15.49:0/528412580 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fafc415dbf0 con 0x7fafc415c950 2023-07-18T03:40:26.443 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:26.443 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1095518881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1095518881},{"type":"v1","addr":"172.21.15.49:6834","nonce":1095518881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/843626338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":843626338},{"type":"v1","addr":"172.21.15.107:6835","nonce":843626338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3238680863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3238680863},{"type":"v1","addr":"172.21.15.107:6837","nonce":3238680863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:39:56.844870+0000","modified":"2023-07-18T03:40:00.356383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5172":{"gid":5172,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6835/3161575771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":3161575771},{"type":"v1","addr":"172.21.15.49:6835","nonce":3161575771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-18T03:40:26.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.451+0000 7fafc9fff700 1 -- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fafa406ad70 msgr2=0x7fafa406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:26.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.451+0000 7fafc9fff700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fafa406ad70 0x7fafa406d220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fafb4013040 tx=0x7fafb4008070 comp rx=0 tx=0).stop 2023-07-18T03:40:26.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.451+0000 7fafc9fff700 1 -- 172.21.15.49:0/528412580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc415c950 msgr2=0x7fafc40b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:26.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.451+0000 7fafc9fff700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc415c950 0x7fafc40b07d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00cc70 tx=0x7fafbc00cf80 comp rx=0 tx=0).stop 2023-07-18T03:40:26.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.451+0000 7fafc9fff700 1 -- 172.21.15.49:0/528412580 shutdown_connections 2023-07-18T03:40:26.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.451+0000 7fafc9fff700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fafa406ad70 0x7fafa406d220 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.451+0000 7fafc9fff700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc415c950 0x7fafc40b07d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.451+0000 7fafc9fff700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc40b1e00 0x7fafc4157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.451+0000 7fafc9fff700 1 --2- 172.21.15.49:0/528412580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafc40d1220 0x7fafc40b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.451+0000 7fafc9fff700 1 -- 172.21.15.49:0/528412580 >> 172.21.15.49:0/528412580 conn(0x7fafc409c1d0 msgr2=0x7fafc40b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:26.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.452+0000 7fafc9fff700 1 -- 172.21.15.49:0/528412580 shutdown_connections 2023-07-18T03:40:26.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.452+0000 7fafc9fff700 1 -- 172.21.15.49:0/528412580 wait complete. 2023-07-18T03:40:26.449 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 19 2023-07-18T03:40:26.465 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:26.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.676+0000 7f708d7ff700 1 Processor -- start 2023-07-18T03:40:26.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.676+0000 7f708d7ff700 1 -- start start 2023-07-18T03:40:26.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.677+0000 7f708d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70880b4510 0x7f70880b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:26.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.677+0000 7f708d7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f70880b5d90 0x7f70880cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:26.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.677+0000 7f708d7ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70880b7f50 con 0x7f70880d0b10 2023-07-18T03:40:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.677+0000 7f708d7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70880b48e0 con 0x7f70880b5d90 2023-07-18T03:40:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.677+0000 7f708d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70880b7340 con 0x7f70880b4510 2023-07-18T03:40:26.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.677+0000 7f70877fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f70880d0b10 0x7f70880b83e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36154/0 (socket says 172.21.15.49:36154) 2023-07-18T03:40:26.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.677+0000 7f70877fe700 1 -- 172.21.15.49:0/2637189647 learned_addr learned my addr 172.21.15.49:0/2637189647 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.678+0000 7f707ffff700 1 --2- 172.21.15.49:0/2637189647 >> v2:172.21.15.107:3300/0 conn(0x7f70880b5d90 0x7f70880cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.678+0000 7f7086ffd700 1 --2- 172.21.15.49:0/2637189647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70880b4510 0x7f70880b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.678+0000 7f70867fc700 1 -- 172.21.15.49:0/2637189647 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893684555 0 0) 0x7f70880b7f50 con 0x7f70880d0b10 2023-07-18T03:40:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.678+0000 7f70867fc700 1 -- 172.21.15.49:0/2637189647 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70740035b0 con 0x7f70880d0b10 2023-07-18T03:40:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.678+0000 7f707ffff700 1 -- 172.21.15.49:0/2637189647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70880b4510 msgr2=0x7f70880b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.678+0000 7f707ffff700 1 --2- 172.21.15.49:0/2637189647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70880b4510 0x7f70880b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.678+0000 7f707ffff700 1 -- 172.21.15.49:0/2637189647 >> v1:172.21.15.49:6789/0 conn(0x7f70880d0b10 legacy=0x7f70880b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:26.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.678+0000 7f707ffff700 1 -- 172.21.15.49:0/2637189647 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880b7620 con 0x7f70880b5d90 2023-07-18T03:40:26.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.678+0000 7f707ffff700 1 --2- 172.21.15.49:0/2637189647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70880b5d90 0x7f70880cd590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7070009840 tx=0x7f7070009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea203ac12b658114 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:26.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.679+0000 7f70867fc700 1 -- 172.21.15.49:0/2637189647 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070011830 con 0x7f70880b5d90 2023-07-18T03:40:26.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.679+0000 7f70867fc700 1 -- 172.21.15.49:0/2637189647 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7070011990 con 0x7f70880b5d90 2023-07-18T03:40:26.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.679+0000 7f70867fc700 1 -- 172.21.15.49:0/2637189647 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070011d20 con 0x7f70880b5d90 2023-07-18T03:40:26.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.679+0000 7f708d7ff700 1 -- 172.21.15.49:0/2637189647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70880b5d90 msgr2=0x7f70880cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:26.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.679+0000 7f708d7ff700 1 --2- 172.21.15.49:0/2637189647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70880b5d90 0x7f70880cd590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7070009840 tx=0x7f7070009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:26.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.679+0000 7f708d7ff700 1 -- 172.21.15.49:0/2637189647 shutdown_connections 2023-07-18T03:40:26.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.679+0000 7f708d7ff700 1 --2- 172.21.15.49:0/2637189647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70880b4510 0x7f70880b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.679+0000 7f708d7ff700 1 --2- 172.21.15.49:0/2637189647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70880b5d90 0x7f70880cd590 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.679+0000 7f708d7ff700 1 -- 172.21.15.49:0/2637189647 >> 172.21.15.49:0/2637189647 conn(0x7f708809c1d0 msgr2=0x7f70880d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:26.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.679+0000 7f708d7ff700 1 -- 172.21.15.49:0/2637189647 shutdown_connections 2023-07-18T03:40:26.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.679+0000 7f708d7ff700 1 -- 172.21.15.49:0/2637189647 wait complete. 2023-07-18T03:40:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.681+0000 7f708d7ff700 1 Processor -- start 2023-07-18T03:40:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.681+0000 7f708d7ff700 1 -- start start 2023-07-18T03:40:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.682+0000 7f708d7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70880b4510 0x7f7088156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.682+0000 7f708d7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70880b5d90 0x7f7088152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.682+0000 7f708d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70880d0b10 0x7f7088158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.682+0000 7f708d7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70881573b0 con 0x7f70880b4510 2023-07-18T03:40:26.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.682+0000 7f708d7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7088152770 con 0x7f70880b5d90 2023-07-18T03:40:26.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.682+0000 7f708d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7088153b90 con 0x7f70880d0b10 2023-07-18T03:40:26.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.682+0000 7f7086ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70880b4510 0x7f7088156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:26.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.682+0000 7f7086ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70880b4510 0x7f7088156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46828/0 (socket says 172.21.15.49:46828) 2023-07-18T03:40:26.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.682+0000 7f7086ffd700 1 -- 172.21.15.49:0/1238585060 learned_addr learned my addr 172.21.15.49:0/1238585060 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:26.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f707ffff700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70880b5d90 0x7f7088152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:26.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f70877fe700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70880d0b10 0x7f7088158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:26.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f7086ffd700 1 -- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70880d0b10 msgr2=0x7f7088158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:26.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f7086ffd700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70880d0b10 0x7f7088158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f7086ffd700 1 -- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70880b5d90 msgr2=0x7f7088152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:26.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f7086ffd700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70880b5d90 0x7f7088152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f7086ffd700 1 -- 172.21.15.49:0/1238585060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70700094a0 con 0x7f70880b4510 2023-07-18T03:40:26.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f70877fe700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70880d0b10 0x7f7088158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:26.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f7086ffd700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70880b4510 0x7f7088156100 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f707800e930 tx=0x7f707800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:26.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f707f7fe700 1 -- 172.21.15.49:0/1238585060 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078009650 con 0x7f70880b4510 2023-07-18T03:40:26.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f707f7fe700 1 -- 172.21.15.49:0/1238585060 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707800ee80 con 0x7f70880b4510 2023-07-18T03:40:26.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f707f7fe700 1 -- 172.21.15.49:0/1238585060 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800ca50 con 0x7f70880b4510 2023-07-18T03:40:26.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.683+0000 7f708d7ff700 1 -- 172.21.15.49:0/1238585060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7088156b10 con 0x7f70880b4510 2023-07-18T03:40:26.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.684+0000 7f708d7ff700 1 -- 172.21.15.49:0/1238585060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70881570d0 con 0x7f70880b4510 2023-07-18T03:40:26.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.685+0000 7f708d7ff700 1 -- 172.21.15.49:0/1238585060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7088154070 con 0x7f70880b4510 2023-07-18T03:40:26.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.685+0000 7f707f7fe700 1 -- 172.21.15.49:0/1238585060 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f707800cbb0 con 0x7f70880b4510 2023-07-18T03:40:26.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.687+0000 7f707f7fe700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f706806ad70 0x7f706806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:26.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.687+0000 7f707f7fe700 1 -- 172.21.15.49:0/1238585060 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f7078089960 con 0x7f70880b4510 2023-07-18T03:40:26.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.687+0000 7f707ffff700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f706806ad70 0x7f706806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:26.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.688+0000 7f707ffff700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f706806ad70 0x7f706806d220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7070000c00 tx=0x7f7070008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:26.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.691+0000 7f707f7fe700 1 -- 172.21.15.49:0/1238585060 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7078058390 con 0x7f70880b4510 2023-07-18T03:40:26.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.898+0000 7f708d7ff700 1 -- 172.21.15.49:0/1238585060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f708804e280 con 0x7f70880b4510 2023-07-18T03:40:26.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.900+0000 7f707f7fe700 1 -- 172.21.15.49:0/1238585060 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18249 (secure 0 0 0) 0x7f7078057f20 con 0x7f70880b4510 2023-07-18T03:40:26.894 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:26.895 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":38,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:40:23.979978+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:39:54.412334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-18T03:39:55.841218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.902+0000 7f708d7ff700 1 -- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f706806ad70 msgr2=0x7f706806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.902+0000 7f708d7ff700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f706806ad70 0x7f706806d220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7070000c00 tx=0x7f7070008030 comp rx=0 tx=0).stop 2023-07-18T03:40:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.902+0000 7f708d7ff700 1 -- 172.21.15.49:0/1238585060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70880b4510 msgr2=0x7f7088156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.902+0000 7f708d7ff700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70880b4510 0x7f7088156100 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f707800e930 tx=0x7f707800ec40 comp rx=0 tx=0).stop 2023-07-18T03:40:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.902+0000 7f708d7ff700 1 -- 172.21.15.49:0/1238585060 shutdown_connections 2023-07-18T03:40:26.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.902+0000 7f708d7ff700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70880d0b10 0x7f7088158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.902+0000 7f708d7ff700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f706806ad70 0x7f706806d220 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.902+0000 7f708d7ff700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70880b4510 0x7f7088156100 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.902+0000 7f708d7ff700 1 --2- 172.21.15.49:0/1238585060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70880b5d90 0x7f7088152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.902+0000 7f708d7ff700 1 -- 172.21.15.49:0/1238585060 >> 172.21.15.49:0/1238585060 conn(0x7f708809c1d0 msgr2=0x7f708806aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.903+0000 7f708d7ff700 1 -- 172.21.15.49:0/1238585060 shutdown_connections 2023-07-18T03:40:26.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:26.903+0000 7f708d7ff700 1 -- 172.21.15.49:0/1238585060 wait complete. 2023-07-18T03:40:26.916 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-18T03:40:27.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.128+0000 7f44a1fff700 1 Processor -- start 2023-07-18T03:40:27.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.128+0000 7f44a1fff700 1 -- start start 2023-07-18T03:40:27.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.129+0000 7f44a1fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f449c0b12f0 0x7f449c0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f44a1fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f449c0adea0 0x7f449c0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f44a1fff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449c15ce70 con 0x7f449c0c92d0 2023-07-18T03:40:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f44a1fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f449c15da60 con 0x7f449c0adea0 2023-07-18T03:40:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f44a1fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c15d0c0 con 0x7f449c0b12f0 2023-07-18T03:40:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f449b7fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f449c0c92d0 0x7f449c0c96a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36166/0 (socket says 172.21.15.49:36166) 2023-07-18T03:40:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f449b7fe700 1 -- 172.21.15.49:0/520897784 learned_addr learned my addr 172.21.15.49:0/520897784 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:27.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f449bfff700 1 --2- 172.21.15.49:0/520897784 >> v2:172.21.15.49:3300/0 conn(0x7f449c0adea0 0x7f449c0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f449affd700 1 --2- 172.21.15.49:0/520897784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f449c0b12f0 0x7f449c0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f449a7fc700 1 -- 172.21.15.49:0/520897784 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326423931 0 0) 0x7f449c15ce70 con 0x7f449c0c92d0 2023-07-18T03:40:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f449a7fc700 1 -- 172.21.15.49:0/520897784 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44840035b0 con 0x7f449c0c92d0 2023-07-18T03:40:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f449bfff700 1 -- 172.21.15.49:0/520897784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f449c0b12f0 msgr2=0x7f449c0ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f449bfff700 1 --2- 172.21.15.49:0/520897784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f449c0b12f0 0x7f449c0ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f449bfff700 1 -- 172.21.15.49:0/520897784 >> v1:172.21.15.49:6789/0 conn(0x7f449c0c92d0 legacy=0x7f449c0c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f449bfff700 1 -- 172.21.15.49:0/520897784 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c15d3a0 con 0x7f449c0adea0 2023-07-18T03:40:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.130+0000 7f449affd700 1 --2- 172.21.15.49:0/520897784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f449c0b12f0 0x7f449c0ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.131+0000 7f449bfff700 1 --2- 172.21.15.49:0/520897784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449c0adea0 0x7f449c0cc690 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4494009d90 tx=0x7f449400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8e1a648c96b7c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.131+0000 7f449a7fc700 1 -- 172.21.15.49:0/520897784 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449400e070 con 0x7f449c0adea0 2023-07-18T03:40:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.131+0000 7f449a7fc700 1 -- 172.21.15.49:0/520897784 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449400bb20 con 0x7f449c0adea0 2023-07-18T03:40:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.131+0000 7f449a7fc700 1 -- 172.21.15.49:0/520897784 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494012430 con 0x7f449c0adea0 2023-07-18T03:40:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.131+0000 7f44a1fff700 1 -- 172.21.15.49:0/520897784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449c0adea0 msgr2=0x7f449c0cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.131+0000 7f44a1fff700 1 --2- 172.21.15.49:0/520897784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449c0adea0 0x7f449c0cc690 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4494009d90 tx=0x7f449400b330 comp rx=0 tx=0).stop 2023-07-18T03:40:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.132+0000 7f44a1fff700 1 -- 172.21.15.49:0/520897784 shutdown_connections 2023-07-18T03:40:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.132+0000 7f44a1fff700 1 --2- 172.21.15.49:0/520897784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449c0adea0 0x7f449c0cc690 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.132+0000 7f44a1fff700 1 --2- 172.21.15.49:0/520897784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f449c0b12f0 0x7f449c0ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.132+0000 7f44a1fff700 1 -- 172.21.15.49:0/520897784 >> 172.21.15.49:0/520897784 conn(0x7f449c0b05d0 msgr2=0x7f449c0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.132+0000 7f44a1fff700 1 -- 172.21.15.49:0/520897784 shutdown_connections 2023-07-18T03:40:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.132+0000 7f44a1fff700 1 -- 172.21.15.49:0/520897784 wait complete. 2023-07-18T03:40:27.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.134+0000 7f44a1fff700 1 Processor -- start 2023-07-18T03:40:27.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.134+0000 7f44a1fff700 1 -- start start 2023-07-18T03:40:27.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.135+0000 7f44a1fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f449c0adea0 0x7f449c14c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.135+0000 7f44a1fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f449c0b12f0 0x7f449c0705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.135+0000 7f44a1fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449c0c92d0 0x7f449c0c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.135+0000 7f44a1fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c06b9c0 con 0x7f449c0c92d0 2023-07-18T03:40:27.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.135+0000 7f44a1fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c06b650 con 0x7f449c0b12f0 2023-07-18T03:40:27.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.135+0000 7f44a1fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c1f3340 con 0x7f449c0adea0 2023-07-18T03:40:27.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.135+0000 7f449bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449c0c92d0 0x7f449c0c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.135+0000 7f449bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449c0c92d0 0x7f449c0c5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46848/0 (socket says 172.21.15.49:46848) 2023-07-18T03:40:27.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.135+0000 7f449bfff700 1 -- 172.21.15.49:0/856391100 learned_addr learned my addr 172.21.15.49:0/856391100 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:27.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.135+0000 7f449affd700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f449c0b12f0 0x7f449c0705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.135+0000 7f449b7fe700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f449c0adea0 0x7f449c14c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.136+0000 7f449bfff700 1 -- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f449c0adea0 msgr2=0x7f449c14c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.136+0000 7f449bfff700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f449c0adea0 0x7f449c14c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.136+0000 7f449bfff700 1 -- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f449c0b12f0 msgr2=0x7f449c0705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.136+0000 7f449bfff700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f449c0b12f0 0x7f449c0705d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.136+0000 7f449bfff700 1 -- 172.21.15.49:0/856391100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4494009a70 con 0x7f449c0c92d0 2023-07-18T03:40:27.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.136+0000 7f449bfff700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449c0c92d0 0x7f449c0c5b90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4494014040 tx=0x7f4494004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:27.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.136+0000 7f4483fff700 1 -- 172.21.15.49:0/856391100 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494012430 con 0x7f449c0c92d0 2023-07-18T03:40:27.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.136+0000 7f4483fff700 1 -- 172.21.15.49:0/856391100 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4494020070 con 0x7f449c0c92d0 2023-07-18T03:40:27.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.136+0000 7f44a1fff700 1 -- 172.21.15.49:0/856391100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449c071050 con 0x7f449c0c92d0 2023-07-18T03:40:27.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.136+0000 7f4483fff700 1 -- 172.21.15.49:0/856391100 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494003ea0 con 0x7f449c0c92d0 2023-07-18T03:40:27.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.136+0000 7f44a1fff700 1 -- 172.21.15.49:0/856391100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449c071610 con 0x7f449c0c92d0 2023-07-18T03:40:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.137+0000 7f44a1fff700 1 -- 172.21.15.49:0/856391100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449c06bb00 con 0x7f449c0c92d0 2023-07-18T03:40:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.138+0000 7f4483fff700 1 -- 172.21.15.49:0/856391100 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f4494004040 con 0x7f449c0c92d0 2023-07-18T03:40:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.139+0000 7f4483fff700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f448806ad70 0x7f448806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.139+0000 7f4483fff700 1 -- 172.21.15.49:0/856391100 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f4494022070 con 0x7f449c0c92d0 2023-07-18T03:40:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.140+0000 7f449b7fe700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f448806ad70 0x7f448806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.141+0000 7f449b7fe700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f448806ad70 0x7f448806d220 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f448c007fd0 tx=0x7f448c007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.144+0000 7f4483fff700 1 -- 172.21.15.49:0/856391100 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f449c06bb00 con 0x7f449c0c92d0 2023-07-18T03:40:27.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.398+0000 7f44a1fff700 1 -- 172.21.15.49:0/856391100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f449c06bb00 con 0x7f449c0c92d0 2023-07-18T03:40:27.393 INFO:tasks.ceph.mon.a.smithi049.stderr:2023-07-18T03:40:27.399+0000 7ff4cc5f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-18T03:40:27.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.404+0000 7f4483fff700 1 -- 172.21.15.49:0/856391100 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) v1 ==== 161+0+0 (secure 0 0 0) 0x7f449c06bb00 con 0x7f449c0c92d0 2023-07-18T03:40:27.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.406+0000 7f44a1fff700 1 -- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f448806ad70 msgr2=0x7f448806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.406+0000 7f44a1fff700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f448806ad70 0x7f448806d220 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f448c007fd0 tx=0x7f448c007dc0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.406+0000 7f44a1fff700 1 -- 172.21.15.49:0/856391100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449c0c92d0 msgr2=0x7f449c0c5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.406+0000 7f44a1fff700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449c0c92d0 0x7f449c0c5b90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4494014040 tx=0x7f4494004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.407+0000 7f44a1fff700 1 -- 172.21.15.49:0/856391100 shutdown_connections 2023-07-18T03:40:27.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.407+0000 7f44a1fff700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f448806ad70 0x7f448806d220 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.407+0000 7f44a1fff700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449c0c92d0 0x7f449c0c5b90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.407+0000 7f44a1fff700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f449c0adea0 0x7f449c14c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.407+0000 7f44a1fff700 1 --2- 172.21.15.49:0/856391100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f449c0b12f0 0x7f449c0705d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.407+0000 7f44a1fff700 1 -- 172.21.15.49:0/856391100 >> 172.21.15.49:0/856391100 conn(0x7f449c0b05d0 msgr2=0x7f449c0ae670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:27.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.407+0000 7f44a1fff700 1 -- 172.21.15.49:0/856391100 shutdown_connections 2023-07-18T03:40:27.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.407+0000 7f44a1fff700 1 -- 172.21.15.49:0/856391100 wait complete. 2023-07-18T03:40:27.403 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-18T03:40:27.408 INFO:tasks.ceph.mds.c.smithi049.stderr: -5644> 2023-07-18T03:40:03.032+0000 7f60b99f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-18T03:40:27.421 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-18T03:40:27.465 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-07-18T03:40:27.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.637+0000 7f11da1ff700 1 Processor -- start 2023-07-18T03:40:27.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.637+0000 7f11da1ff700 1 -- start start 2023-07-18T03:40:27.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.638+0000 7f11da1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f11d40d1e40 0x7f11d40cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.638+0000 7f11da1ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d40cd5a0 con 0x7f11d40c71b0 2023-07-18T03:40:27.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.638+0000 7f11da1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d40cd7f0 con 0x7f11d415c860 2023-07-18T03:40:27.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.638+0000 7f11da1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d4146740 con 0x7f11d40d1e40 2023-07-18T03:40:27.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.639+0000 7f11d3fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f11d40c71b0 0x7f11d40b49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36196/0 (socket says 172.21.15.49:36196) 2023-07-18T03:40:27.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.639+0000 7f11d3fff700 1 -- 172.21.15.49:0/3211714462 learned_addr learned my addr 172.21.15.49:0/3211714462 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:27.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.639+0000 7f11d2ffd700 1 --2- 172.21.15.49:0/3211714462 >> v2:172.21.15.107:3300/0 conn(0x7f11d40d1e40 0x7f11d40cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.639+0000 7f11d27fc700 1 -- 172.21.15.49:0/3211714462 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319064746 0 0) 0x7f11d40cd5a0 con 0x7f11d40c71b0 2023-07-18T03:40:27.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.639+0000 7f11d27fc700 1 -- 172.21.15.49:0/3211714462 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11bc0035b0 con 0x7f11d40c71b0 2023-07-18T03:40:27.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.639+0000 7f11d27fc700 1 -- 172.21.15.49:0/3211714462 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 940188706 0 0) 0x7f11d40cd7f0 con 0x7f11d415c860 2023-07-18T03:40:27.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.639+0000 7f11d27fc700 1 -- 172.21.15.49:0/3211714462 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11d40cd5a0 con 0x7f11d415c860 2023-07-18T03:40:27.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.639+0000 7f11d2ffd700 1 -- 172.21.15.49:0/3211714462 >> v1:172.21.15.107:6789/0 conn(0x7f11d415c860 legacy=0x7f11d40c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.639+0000 7f11d2ffd700 1 -- 172.21.15.49:0/3211714462 >> v1:172.21.15.49:6789/0 conn(0x7f11d40c71b0 legacy=0x7f11d40b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.639+0000 7f11d2ffd700 1 -- 172.21.15.49:0/3211714462 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d40cdad0 con 0x7f11d40d1e40 2023-07-18T03:40:27.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.640+0000 7f11d2ffd700 1 --2- 172.21.15.49:0/3211714462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d40d1e40 0x7f11d40cf2a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009840 tx=0x7f11c8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed68648f8ba645a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:27.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.640+0000 7f11d27fc700 1 -- 172.21.15.49:0/3211714462 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8011830 con 0x7f11d40d1e40 2023-07-18T03:40:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.640+0000 7f11d27fc700 1 -- 172.21.15.49:0/3211714462 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c8011990 con 0x7f11d40d1e40 2023-07-18T03:40:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.640+0000 7f11d27fc700 1 -- 172.21.15.49:0/3211714462 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8011d20 con 0x7f11d40d1e40 2023-07-18T03:40:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.640+0000 7f11da1ff700 1 -- 172.21.15.49:0/3211714462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d40d1e40 msgr2=0x7f11d40cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.640+0000 7f11da1ff700 1 --2- 172.21.15.49:0/3211714462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d40d1e40 0x7f11d40cf2a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009840 tx=0x7f11c8009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.640+0000 7f11da1ff700 1 -- 172.21.15.49:0/3211714462 shutdown_connections 2023-07-18T03:40:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.640+0000 7f11da1ff700 1 --2- 172.21.15.49:0/3211714462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d40d1e40 0x7f11d40cf2a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.640+0000 7f11da1ff700 1 -- 172.21.15.49:0/3211714462 >> 172.21.15.49:0/3211714462 conn(0x7f11d4065bd0 msgr2=0x7f11d4065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:27.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.641+0000 7f11da1ff700 1 -- 172.21.15.49:0/3211714462 shutdown_connections 2023-07-18T03:40:27.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.641+0000 7f11da1ff700 1 -- 172.21.15.49:0/3211714462 wait complete. 2023-07-18T03:40:27.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.642+0000 7f11da1ff700 1 Processor -- start 2023-07-18T03:40:27.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.642+0000 7f11da1ff700 1 -- start start 2023-07-18T03:40:27.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.643+0000 7f11da1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d40c71b0 0x7f11d40cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.643+0000 7f11da1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11d40d1e40 0x7f11d40b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.643+0000 7f11da1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d415c860 0x7f11d41532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.643+0000 7f11da1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40cb170 con 0x7f11d415c860 2023-07-18T03:40:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.643+0000 7f11da1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40cc6a0 con 0x7f11d40c71b0 2023-07-18T03:40:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.643+0000 7f11da1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40cab60 con 0x7f11d40d1e40 2023-07-18T03:40:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.644+0000 7f11d3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d415c860 0x7f11d41532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.644+0000 7f11d3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d415c860 0x7f11d41532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46862/0 (socket says 172.21.15.49:46862) 2023-07-18T03:40:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.644+0000 7f11d2ffd700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11d40d1e40 0x7f11d40b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.644+0000 7f11d3fff700 1 -- 172.21.15.49:0/2795818795 learned_addr learned my addr 172.21.15.49:0/2795818795 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.644+0000 7f11d37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d40c71b0 0x7f11d40cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.644+0000 7f11d3fff700 1 -- 172.21.15.49:0/2795818795 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11d40d1e40 msgr2=0x7f11d40b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.644+0000 7f11d3fff700 1 --2- 172.21.15.49:0/2795818795 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11d40d1e40 0x7f11d40b73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.644+0000 7f11d3fff700 1 -- 172.21.15.49:0/2795818795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d40c71b0 msgr2=0x7f11d40cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.644+0000 7f11d3fff700 1 --2- 172.21.15.49:0/2795818795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d40c71b0 0x7f11d40cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.644+0000 7f11d3fff700 1 -- 172.21.15.49:0/2795818795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c80094a0 con 0x7f11d415c860 2023-07-18T03:40:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.644+0000 7f11d3fff700 1 --2- 172.21.15.49:0/2795818795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d415c860 0x7f11d41532a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00cb40 tx=0x7f11cc00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.645+0000 7f11bbfff700 1 -- 172.21.15.49:0/2795818795 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc00a9e0 con 0x7f11d415c860 2023-07-18T03:40:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.645+0000 7f11bbfff700 1 -- 172.21.15.49:0/2795818795 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc00ab40 con 0x7f11d415c860 2023-07-18T03:40:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.645+0000 7f11da1ff700 1 -- 172.21.15.49:0/2795818795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d40cc3c0 con 0x7f11d415c860 2023-07-18T03:40:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.645+0000 7f11bbfff700 1 -- 172.21.15.49:0/2795818795 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc012400 con 0x7f11d415c860 2023-07-18T03:40:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.645+0000 7f11da1ff700 1 -- 172.21.15.49:0/2795818795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d4156120 con 0x7f11d415c860 2023-07-18T03:40:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.646+0000 7f11da1ff700 1 -- 172.21.15.49:0/2795818795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d40cb2e0 con 0x7f11d415c860 2023-07-18T03:40:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.647+0000 7f11bbfff700 1 -- 172.21.15.49:0/2795818795 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f11cc01b3b0 con 0x7f11d415c860 2023-07-18T03:40:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.648+0000 7f11bbfff700 1 --2- 172.21.15.49:0/2795818795 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f11c006ad70 0x7f11c006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.648+0000 7f11bbfff700 1 -- 172.21.15.49:0/2795818795 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f11cc089270 con 0x7f11d415c860 2023-07-18T03:40:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.649+0000 7f11d37fe700 1 --2- 172.21.15.49:0/2795818795 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f11c006ad70 0x7f11c006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.649+0000 7f11d37fe700 1 --2- 172.21.15.49:0/2795818795 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f11c006ad70 0x7f11c006d220 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f11c40011e0 tx=0x7f11c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:27.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.653+0000 7f11bbfff700 1 -- 172.21.15.49:0/2795818795 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f11cc054240 con 0x7f11d415c860 2023-07-18T03:40:27.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:27.910+0000 7f11da1ff700 1 -- 172.21.15.49:0/2795818795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f11d4074350 con 0x7f11d415c860 2023-07-18T03:40:28.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.405+0000 7f11bbfff700 1 -- 172.21.15.49:0/2795818795 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f11cc057860 con 0x7f11d415c860 2023-07-18T03:40:28.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.407+0000 7f11da1ff700 1 -- 172.21.15.49:0/2795818795 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f11c006ad70 msgr2=0x7f11c006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:28.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.407+0000 7f11da1ff700 1 --2- 172.21.15.49:0/2795818795 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f11c006ad70 0x7f11c006d220 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f11c40011e0 tx=0x7f11c4009040 comp rx=0 tx=0).stop 2023-07-18T03:40:28.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.407+0000 7f11da1ff700 1 -- 172.21.15.49:0/2795818795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d415c860 msgr2=0x7f11d41532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:28.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.407+0000 7f11da1ff700 1 --2- 172.21.15.49:0/2795818795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d415c860 0x7f11d41532a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00cb40 tx=0x7f11cc00ce50 comp rx=0 tx=0).stop 2023-07-18T03:40:28.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.407+0000 7f11da1ff700 1 -- 172.21.15.49:0/2795818795 shutdown_connections 2023-07-18T03:40:28.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.407+0000 7f11da1ff700 1 --2- 172.21.15.49:0/2795818795 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f11c006ad70 0x7f11c006d220 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:28.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.407+0000 7f11da1ff700 1 --2- 172.21.15.49:0/2795818795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d415c860 0x7f11d41532a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:28.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.407+0000 7f11da1ff700 1 --2- 172.21.15.49:0/2795818795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d40c71b0 0x7f11d40cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:28.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.407+0000 7f11da1ff700 1 --2- 172.21.15.49:0/2795818795 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11d40d1e40 0x7f11d40b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:28.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.408+0000 7f11da1ff700 1 -- 172.21.15.49:0/2795818795 >> 172.21.15.49:0/2795818795 conn(0x7f11d4065bd0 msgr2=0x7f11d40d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:28.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.408+0000 7f11da1ff700 1 -- 172.21.15.49:0/2795818795 shutdown_connections 2023-07-18T03:40:28.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.408+0000 7f11da1ff700 1 -- 172.21.15.49:0/2795818795 wait complete. 2023-07-18T03:40:28.422 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-18T03:40:28.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.678+0000 7f6a559ff700 1 Processor -- start 2023-07-18T03:40:28.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.678+0000 7f6a559ff700 1 -- start start 2023-07-18T03:40:28.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.679+0000 7f6a559ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6a5006b750 0x7f6a500cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:28.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.679+0000 7f6a559ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a500cc800 0x7f6a500b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:28.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.679+0000 7f6a559ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a500ad830 con 0x7f6a5015c8f0 2023-07-18T03:40:28.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.679+0000 7f6a559ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a500b0f70 con 0x7f6a5006b750 2023-07-18T03:40:28.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.679+0000 7f6a559ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500cd090 con 0x7f6a500cc800 2023-07-18T03:40:28.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.680+0000 7f6a4f7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a500cc800 0x7f6a500b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:28.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.680+0000 7f6a4f7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a500cc800 0x7f6a500b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:34686/0 (socket says 172.21.15.49:34686) 2023-07-18T03:40:28.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.680+0000 7f6a4f7fe700 1 -- 172.21.15.49:0/2650301633 learned_addr learned my addr 172.21.15.49:0/2650301633 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:28.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.680+0000 7f6a4effd700 1 --1- 172.21.15.49:0/2650301633 >> v1:172.21.15.107:6789/0 conn(0x7f6a5015c8f0 0x7f6a5006b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:38834/0 (socket says 172.21.15.49:38834) 2023-07-18T03:40:28.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.680+0000 7f6a4e7fc700 1 --2- 172.21.15.49:0/2650301633 >> v2:172.21.15.107:3300/0 conn(0x7f6a5006b750 0x7f6a500cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:28.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.680+0000 7f6a4f7fe700 1 -- 172.21.15.49:0/2650301633 >> v2:172.21.15.107:3300/0 conn(0x7f6a5006b750 msgr2=0x7f6a500cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:28.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.680+0000 7f6a4f7fe700 1 --2- 172.21.15.49:0/2650301633 >> v2:172.21.15.107:3300/0 conn(0x7f6a5006b750 0x7f6a500cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:28.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.680+0000 7f6a4f7fe700 1 -- 172.21.15.49:0/2650301633 >> v1:172.21.15.107:6789/0 conn(0x7f6a5015c8f0 legacy=0x7f6a5006b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:28.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.680+0000 7f6a4f7fe700 1 -- 172.21.15.49:0/2650301633 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a500b0640 con 0x7f6a500cc800 2023-07-18T03:40:28.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.681+0000 7f6a4f7fe700 1 --2- 172.21.15.49:0/2650301633 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a500cc800 0x7f6a500b1400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009e50 tx=0x7f6a4800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3afb01bf7e743fa7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:28.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.681+0000 7f6a4dffb700 1 -- 172.21.15.49:0/2650301633 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800e070 con 0x7f6a500cc800 2023-07-18T03:40:28.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.681+0000 7f6a4dffb700 1 -- 172.21.15.49:0/2650301633 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4800b910 con 0x7f6a500cc800 2023-07-18T03:40:28.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.681+0000 7f6a4dffb700 1 -- 172.21.15.49:0/2650301633 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800bcc0 con 0x7f6a500cc800 2023-07-18T03:40:28.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.681+0000 7f6a559ff700 1 -- 172.21.15.49:0/2650301633 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a500cc800 msgr2=0x7f6a500b1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:28.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.681+0000 7f6a559ff700 1 --2- 172.21.15.49:0/2650301633 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a500cc800 0x7f6a500b1400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009e50 tx=0x7f6a4800b330 comp rx=0 tx=0).stop 2023-07-18T03:40:28.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.681+0000 7f6a559ff700 1 -- 172.21.15.49:0/2650301633 shutdown_connections 2023-07-18T03:40:28.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.681+0000 7f6a559ff700 1 --2- 172.21.15.49:0/2650301633 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a500cc800 0x7f6a500b1400 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:28.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.681+0000 7f6a559ff700 1 --2- 172.21.15.49:0/2650301633 >> v2:172.21.15.107:3300/0 conn(0x7f6a5006b750 0x7f6a500cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:28.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.681+0000 7f6a559ff700 1 -- 172.21.15.49:0/2650301633 >> 172.21.15.49:0/2650301633 conn(0x7f6a500664f0 msgr2=0x7f6a50099f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:28.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.681+0000 7f6a559ff700 1 -- 172.21.15.49:0/2650301633 shutdown_connections 2023-07-18T03:40:28.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.682+0000 7f6a559ff700 1 -- 172.21.15.49:0/2650301633 wait complete. 2023-07-18T03:40:28.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.683+0000 7f6a559ff700 1 Processor -- start 2023-07-18T03:40:28.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.683+0000 7f6a559ff700 1 -- start start 2023-07-18T03:40:28.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.684+0000 7f6a559ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a5006b750 0x7f6a50158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:28.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.684+0000 7f6a559ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a500cc800 0x7f6a5015cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:28.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a559ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a5015c8f0 0x7f6a500b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:28.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a559ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500c6930 con 0x7f6a5015c8f0 2023-07-18T03:40:28.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a559ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500c5cb0 con 0x7f6a500cc800 2023-07-18T03:40:28.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a559ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500c5f90 con 0x7f6a5006b750 2023-07-18T03:40:28.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a4f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a5015c8f0 0x7f6a500b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:28.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a4f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a5015c8f0 0x7f6a500b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46868/0 (socket says 172.21.15.49:46868) 2023-07-18T03:40:28.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a4f7fe700 1 -- 172.21.15.49:0/3733428673 learned_addr learned my addr 172.21.15.49:0/3733428673 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:28.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a4e7fc700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a500cc800 0x7f6a5015cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:28.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a4effd700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a5006b750 0x7f6a50158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:28.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a4f7fe700 1 -- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a5006b750 msgr2=0x7f6a50158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:28.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a4f7fe700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a5006b750 0x7f6a50158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:28.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a4f7fe700 1 -- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a500cc800 msgr2=0x7f6a5015cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:28.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a4f7fe700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a500cc800 0x7f6a5015cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:28.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.685+0000 7f6a4f7fe700 1 -- 172.21.15.49:0/3733428673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a48009b30 con 0x7f6a5015c8f0 2023-07-18T03:40:28.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.686+0000 7f6a4f7fe700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a5015c8f0 0x7f6a500b47d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6a500cd1d0 tx=0x7f6a48004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:28.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.686+0000 7f6a37fff700 1 -- 172.21.15.49:0/3733428673 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48004800 con 0x7f6a5015c8f0 2023-07-18T03:40:28.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.686+0000 7f6a37fff700 1 -- 172.21.15.49:0/3733428673 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a48004960 con 0x7f6a5015c8f0 2023-07-18T03:40:28.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.686+0000 7f6a559ff700 1 -- 172.21.15.49:0/3733428673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a500c6270 con 0x7f6a5015c8f0 2023-07-18T03:40:28.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.686+0000 7f6a37fff700 1 -- 172.21.15.49:0/3733428673 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4801c680 con 0x7f6a5015c8f0 2023-07-18T03:40:28.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.686+0000 7f6a559ff700 1 -- 172.21.15.49:0/3733428673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5015d440 con 0x7f6a5015c8f0 2023-07-18T03:40:28.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.687+0000 7f6a559ff700 1 -- 172.21.15.49:0/3733428673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a500c6a70 con 0x7f6a5015c8f0 2023-07-18T03:40:28.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.688+0000 7f6a37fff700 1 -- 172.21.15.49:0/3733428673 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f6a4801c7e0 con 0x7f6a5015c8f0 2023-07-18T03:40:28.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.689+0000 7f6a37fff700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6a3c06ad70 0x7f6a3c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:28.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.689+0000 7f6a37fff700 1 -- 172.21.15.49:0/3733428673 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f6a4808b2c0 con 0x7f6a5015c8f0 2023-07-18T03:40:28.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.690+0000 7f6a4effd700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6a3c06ad70 0x7f6a3c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:28.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.690+0000 7f6a4effd700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6a3c06ad70 0x7f6a3c06d220 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6a40000f60 tx=0x7f6a40007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:28.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.694+0000 7f6a37fff700 1 -- 172.21.15.49:0/3733428673 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6a500c6a70 con 0x7f6a5015c8f0 2023-07-18T03:40:28.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:28.903+0000 7f6a559ff700 1 -- 172.21.15.49:0/3733428673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6a500c6a70 con 0x7f6a5015c8f0 2023-07-18T03:40:29.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.407+0000 7f6a37fff700 1 -- 172.21.15.49:0/3733428673 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6a500c6a70 con 0x7f6a5015c8f0 2023-07-18T03:40:29.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.413+0000 7f6a559ff700 1 -- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6a3c06ad70 msgr2=0x7f6a3c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:29.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.413+0000 7f6a559ff700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6a3c06ad70 0x7f6a3c06d220 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6a40000f60 tx=0x7f6a40007ed0 comp rx=0 tx=0).stop 2023-07-18T03:40:29.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.413+0000 7f6a559ff700 1 -- 172.21.15.49:0/3733428673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a5015c8f0 msgr2=0x7f6a500b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:29.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.413+0000 7f6a559ff700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a5015c8f0 0x7f6a500b47d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6a500cd1d0 tx=0x7f6a48004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:29.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.413+0000 7f6a559ff700 1 -- 172.21.15.49:0/3733428673 shutdown_connections 2023-07-18T03:40:29.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.413+0000 7f6a559ff700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6a3c06ad70 0x7f6a3c06d220 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:29.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.413+0000 7f6a559ff700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a5015c8f0 0x7f6a500b47d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:29.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.413+0000 7f6a559ff700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a5006b750 0x7f6a50158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:29.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.413+0000 7f6a559ff700 1 --2- 172.21.15.49:0/3733428673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a500cc800 0x7f6a5015cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:29.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.413+0000 7f6a559ff700 1 -- 172.21.15.49:0/3733428673 >> 172.21.15.49:0/3733428673 conn(0x7f6a500664f0 msgr2=0x7f6a5006da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:29.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.413+0000 7f6a559ff700 1 -- 172.21.15.49:0/3733428673 shutdown_connections 2023-07-18T03:40:29.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.414+0000 7f6a559ff700 1 -- 172.21.15.49:0/3733428673 wait complete. 2023-07-18T03:40:29.411 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2023-07-18T03:40:29.429 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-18T03:40:29.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.650+0000 7f952cdff700 1 Processor -- start 2023-07-18T03:40:29.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.650+0000 7f952cdff700 1 -- start start 2023-07-18T03:40:29.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.651+0000 7f952cdff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9528158af0 0x7f95280ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.651+0000 7f952cdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95280b0d80 0x7f952806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.651+0000 7f952cdff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9528071220 con 0x7f95280cc540 2023-07-18T03:40:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.651+0000 7f952cdff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95280ce7c0 con 0x7f9528158af0 2023-07-18T03:40:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.651+0000 7f952cdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95280caa50 con 0x7f95280b0d80 2023-07-18T03:40:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.652+0000 7f952659c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9528158af0 0x7f95280ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.652+0000 7f9526d9d700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f95280cc540 0x7f95280cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35516/0 (socket says 172.21.15.49:35516) 2023-07-18T03:40:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.652+0000 7f9526d9d700 1 -- 172.21.15.49:0/1975614325 learned_addr learned my addr 172.21.15.49:0/1975614325 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:29.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.652+0000 7f9525d9b700 1 --2- 172.21.15.49:0/1975614325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95280b0d80 0x7f952806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:29.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.652+0000 7f952659c700 1 -- 172.21.15.49:0/1975614325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95280b0d80 msgr2=0x7f952806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:29.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.652+0000 7f952659c700 1 --2- 172.21.15.49:0/1975614325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95280b0d80 0x7f952806b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:29.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.652+0000 7f952659c700 1 -- 172.21.15.49:0/1975614325 >> v1:172.21.15.49:6789/0 conn(0x7f95280cc540 legacy=0x7f95280cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:29.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.652+0000 7f952659c700 1 -- 172.21.15.49:0/1975614325 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95280cad30 con 0x7f9528158af0 2023-07-18T03:40:29.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.652+0000 7f952559a700 1 -- 172.21.15.49:0/1975614325 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605316598 0 0) 0x7f9528071220 con 0x7f95280cc540 2023-07-18T03:40:29.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.652+0000 7f952659c700 1 --2- 172.21.15.49:0/1975614325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9528158af0 0x7f95280ce190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f951800bae0 tx=0x7f951800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0ce34ae80bd467 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:29.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.653+0000 7f952559a700 1 -- 172.21.15.49:0/1975614325 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951800dd30 con 0x7f9528158af0 2023-07-18T03:40:29.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.653+0000 7f952559a700 1 -- 172.21.15.49:0/1975614325 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951800de90 con 0x7f9528158af0 2023-07-18T03:40:29.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.653+0000 7f952559a700 1 -- 172.21.15.49:0/1975614325 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518010690 con 0x7f9528158af0 2023-07-18T03:40:29.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.653+0000 7f952cdff700 1 -- 172.21.15.49:0/1975614325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9528158af0 msgr2=0x7f95280ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:29.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.653+0000 7f952cdff700 1 --2- 172.21.15.49:0/1975614325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9528158af0 0x7f95280ce190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f951800bae0 tx=0x7f951800bdf0 comp rx=0 tx=0).stop 2023-07-18T03:40:29.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.653+0000 7f952cdff700 1 -- 172.21.15.49:0/1975614325 shutdown_connections 2023-07-18T03:40:29.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.653+0000 7f952cdff700 1 --2- 172.21.15.49:0/1975614325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9528158af0 0x7f95280ce190 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:29.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.653+0000 7f952cdff700 1 --2- 172.21.15.49:0/1975614325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95280b0d80 0x7f952806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:29.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.653+0000 7f952cdff700 1 -- 172.21.15.49:0/1975614325 >> 172.21.15.49:0/1975614325 conn(0x7f95280cea00 msgr2=0x7f95280cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:29.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.653+0000 7f952cdff700 1 -- 172.21.15.49:0/1975614325 shutdown_connections 2023-07-18T03:40:29.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.654+0000 7f952cdff700 1 -- 172.21.15.49:0/1975614325 wait complete. 2023-07-18T03:40:29.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.655+0000 7f952cdff700 1 Processor -- start 2023-07-18T03:40:29.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.655+0000 7f952cdff700 1 -- start start 2023-07-18T03:40:29.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.656+0000 7f952cdff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95280b0d80 0x7f952814ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:29.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.656+0000 7f952cdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95280cc540 0x7f95281e1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:29.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.656+0000 7f952cdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9528158af0 0x7f95280b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:29.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.656+0000 7f952cdff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95280b3b30 con 0x7f95280b0d80 2023-07-18T03:40:29.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.656+0000 7f952cdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952814b3a0 con 0x7f9528158af0 2023-07-18T03:40:29.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.656+0000 7f952cdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95280b4260 con 0x7f95280cc540 2023-07-18T03:40:29.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.657+0000 7f952659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95280b0d80 0x7f952814ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:29.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.657+0000 7f952659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95280b0d80 0x7f952814ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32938/0 (socket says 172.21.15.49:32938) 2023-07-18T03:40:29.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.657+0000 7f952659c700 1 -- 172.21.15.49:0/2235779930 learned_addr learned my addr 172.21.15.49:0/2235779930 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:29.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.657+0000 7f9525d9b700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95280cc540 0x7f95281e1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:29.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.657+0000 7f9526d9d700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9528158af0 0x7f95280b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:29.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.657+0000 7f952659c700 1 -- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95280cc540 msgr2=0x7f95281e1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:29.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.657+0000 7f952659c700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95280cc540 0x7f95281e1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:29.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.657+0000 7f952659c700 1 -- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9528158af0 msgr2=0x7f95280b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:29.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.657+0000 7f952659c700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9528158af0 0x7f95280b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:29.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.657+0000 7f952659c700 1 -- 172.21.15.49:0/2235779930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951800b740 con 0x7f95280b0d80 2023-07-18T03:40:29.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.657+0000 7f9526d9d700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9528158af0 0x7f95280b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:29.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.657+0000 7f952659c700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95280b0d80 0x7f952814ae60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f951800b2c0 tx=0x7f95180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:29.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.658+0000 7f9516ffd700 1 -- 172.21.15.49:0/2235779930 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951801d040 con 0x7f95280b0d80 2023-07-18T03:40:29.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.658+0000 7f9516ffd700 1 -- 172.21.15.49:0/2235779930 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9518010c70 con 0x7f95280b0d80 2023-07-18T03:40:29.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.658+0000 7f952cdff700 1 -- 172.21.15.49:0/2235779930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95280b4540 con 0x7f95280b0d80 2023-07-18T03:40:29.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.658+0000 7f9516ffd700 1 -- 172.21.15.49:0/2235779930 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95180036a0 con 0x7f95280b0d80 2023-07-18T03:40:29.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.658+0000 7f952cdff700 1 -- 172.21.15.49:0/2235779930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95280b3330 con 0x7f95280b0d80 2023-07-18T03:40:29.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.659+0000 7f952cdff700 1 -- 172.21.15.49:0/2235779930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9528074350 con 0x7f95280b0d80 2023-07-18T03:40:29.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.660+0000 7f9516ffd700 1 -- 172.21.15.49:0/2235779930 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f9518003820 con 0x7f95280b0d80 2023-07-18T03:40:29.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.661+0000 7f9516ffd700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f950806ad70 0x7f950806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:29.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.661+0000 7f9516ffd700 1 -- 172.21.15.49:0/2235779930 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5739+0+0 (secure 0 0 0) 0x7f951808ae30 con 0x7f95280b0d80 2023-07-18T03:40:29.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.661+0000 7f9525d9b700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f950806ad70 0x7f950806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:29.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.662+0000 7f9525d9b700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f950806ad70 0x7f950806d220 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f951c006b20 tx=0x7f951c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:29.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.665+0000 7f9516ffd700 1 -- 172.21.15.49:0/2235779930 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9518055fb0 con 0x7f95280b0d80 2023-07-18T03:40:29.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:29.874+0000 7f952cdff700 1 -- 172.21.15.49:0/2235779930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f95280ad510 con 0x7f95280b0d80 2023-07-18T03:40:30.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.410+0000 7f9516ffd700 1 -- 172.21.15.49:0/2235779930 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) v1 ==== 169+0+0 (secure 0 0 0) 0x7f95180595d0 con 0x7f95280b0d80 2023-07-18T03:40:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.416+0000 7f952cdff700 1 -- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f950806ad70 msgr2=0x7f950806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.416+0000 7f952cdff700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f950806ad70 0x7f950806d220 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f951c006b20 tx=0x7f951c008040 comp rx=0 tx=0).stop 2023-07-18T03:40:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.416+0000 7f952cdff700 1 -- 172.21.15.49:0/2235779930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95280b0d80 msgr2=0x7f952814ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.416+0000 7f952cdff700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95280b0d80 0x7f952814ae60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f951800b2c0 tx=0x7f95180047c0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.416+0000 7f952cdff700 1 -- 172.21.15.49:0/2235779930 shutdown_connections 2023-07-18T03:40:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.416+0000 7f952cdff700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9528158af0 0x7f95280b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.416+0000 7f952cdff700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f950806ad70 0x7f950806d220 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.416+0000 7f952cdff700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95280b0d80 0x7f952814ae60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.416+0000 7f952cdff700 1 --2- 172.21.15.49:0/2235779930 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95280cc540 0x7f95281e1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.416+0000 7f952cdff700 1 -- 172.21.15.49:0/2235779930 >> 172.21.15.49:0/2235779930 conn(0x7f95280cea00 msgr2=0x7f952806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:30.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.417+0000 7f952cdff700 1 -- 172.21.15.49:0/2235779930 shutdown_connections 2023-07-18T03:40:30.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.417+0000 7f952cdff700 1 -- 172.21.15.49:0/2235779930 wait complete. 2023-07-18T03:40:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2023-07-18T03:40:30.433 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-18T03:40:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.653+0000 7f499759d700 1 Processor -- start 2023-07-18T03:40:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.653+0000 7f499759d700 1 -- start start 2023-07-18T03:40:30.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.654+0000 7f499759d700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f49900d3850 0x7f49900cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:30.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.654+0000 7f499759d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900cfc90 0x7f49900d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:30.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.654+0000 7f499759d700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49900d00f0 con 0x7f4990156fe0 2023-07-18T03:40:30.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.654+0000 7f499759d700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49900d3430 con 0x7f49900d3850 2023-07-18T03:40:30.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.654+0000 7f499759d700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900cde10 con 0x7f49900cfc90 2023-07-18T03:40:30.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.655+0000 7f499659b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4990156fe0 0x7f49900cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35524/0 (socket says 172.21.15.49:35524) 2023-07-18T03:40:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.655+0000 7f499659b700 1 -- 172.21.15.49:0/1425792211 learned_addr learned my addr 172.21.15.49:0/1425792211 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.655+0000 7f4995d9a700 1 --2- 172.21.15.49:0/1425792211 >> v2:172.21.15.49:3300/0 conn(0x7f49900d3850 0x7f49900cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.655+0000 7f4996d9c700 1 --2- 172.21.15.49:0/1425792211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900cfc90 0x7f49900d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.655+0000 7f4995d9a700 1 -- 172.21.15.49:0/1425792211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900cfc90 msgr2=0x7f49900d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.655+0000 7f4995d9a700 1 --2- 172.21.15.49:0/1425792211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900cfc90 0x7f49900d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.655+0000 7f4995d9a700 1 -- 172.21.15.49:0/1425792211 >> v1:172.21.15.49:6789/0 conn(0x7f4990156fe0 legacy=0x7f49900cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:30.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.655+0000 7f4995d9a700 1 -- 172.21.15.49:0/1425792211 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900d2a90 con 0x7f49900d3850 2023-07-18T03:40:30.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.655+0000 7f4996d9c700 1 --2- 172.21.15.49:0/1425792211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900cfc90 0x7f49900d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:30.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.655+0000 7f4995d9a700 1 --2- 172.21.15.49:0/1425792211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49900d3850 0x7f49900cf750 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4980009840 tx=0x7f4980009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954d5d12bf00052f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:30.653 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:40:30.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.656+0000 7f4995599700 1 -- 172.21.15.49:0/1425792211 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980011830 con 0x7f49900d3850 2023-07-18T03:40:30.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.656+0000 7f4995599700 1 -- 172.21.15.49:0/1425792211 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4980011990 con 0x7f49900d3850 2023-07-18T03:40:30.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.656+0000 7f4995599700 1 -- 172.21.15.49:0/1425792211 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980011d20 con 0x7f49900d3850 2023-07-18T03:40:30.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.656+0000 7f499759d700 1 -- 172.21.15.49:0/1425792211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49900d3850 msgr2=0x7f49900cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:30.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.656+0000 7f499759d700 1 --2- 172.21.15.49:0/1425792211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49900d3850 0x7f49900cf750 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4980009840 tx=0x7f4980009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:30.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.656+0000 7f499759d700 1 -- 172.21.15.49:0/1425792211 shutdown_connections 2023-07-18T03:40:30.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.656+0000 7f499759d700 1 --2- 172.21.15.49:0/1425792211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900cfc90 0x7f49900d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.656+0000 7f499759d700 1 --2- 172.21.15.49:0/1425792211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49900d3850 0x7f49900cf750 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.656+0000 7f499759d700 1 -- 172.21.15.49:0/1425792211 >> 172.21.15.49:0/1425792211 conn(0x7f4990065bd0 msgr2=0x7f4990065fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:30.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.656+0000 7f499759d700 1 -- 172.21.15.49:0/1425792211 shutdown_connections 2023-07-18T03:40:30.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.656+0000 7f499759d700 1 -- 172.21.15.49:0/1425792211 wait complete. 2023-07-18T03:40:30.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.658+0000 7f499759d700 1 Processor -- start 2023-07-18T03:40:30.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.658+0000 7f499759d700 1 -- start start 2023-07-18T03:40:30.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.659+0000 7f499759d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900cfc90 0x7f49900cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:30.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.659+0000 7f499759d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49900d3850 0x7f4990148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:30.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.659+0000 7f499759d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4990156fe0 0x7f4990144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:30.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.659+0000 7f499759d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49901486c0 con 0x7f49900d3850 2023-07-18T03:40:30.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.659+0000 7f499759d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49900cc7f0 con 0x7f4990156fe0 2023-07-18T03:40:30.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.659+0000 7f499759d700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4990145850 con 0x7f49900cfc90 2023-07-18T03:40:30.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.659+0000 7f4995d9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49900d3850 0x7f4990148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:30.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.659+0000 7f4995d9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49900d3850 0x7f4990148180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32950/0 (socket says 172.21.15.49:32950) 2023-07-18T03:40:30.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.660+0000 7f4995d9a700 1 -- 172.21.15.49:0/3000784751 learned_addr learned my addr 172.21.15.49:0/3000784751 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:30.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.660+0000 7f499659b700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900cfc90 0x7f49900cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:30.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.660+0000 7f4996d9c700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4990156fe0 0x7f4990144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:30.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.660+0000 7f4995d9a700 1 -- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900cfc90 msgr2=0x7f49900cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:30.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.660+0000 7f4995d9a700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900cfc90 0x7f49900cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.660+0000 7f4995d9a700 1 -- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4990156fe0 msgr2=0x7f4990144db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:30.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.660+0000 7f4995d9a700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4990156fe0 0x7f4990144db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.660+0000 7f4995d9a700 1 -- 172.21.15.49:0/3000784751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49800094a0 con 0x7f49900d3850 2023-07-18T03:40:30.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.660+0000 7f4995d9a700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49900d3850 0x7f4990148180 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f49800090b0 tx=0x7f4980004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:30.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.660+0000 7f4986ffd700 1 -- 172.21.15.49:0/3000784751 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800036a0 con 0x7f49900d3850 2023-07-18T03:40:30.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.661+0000 7f4986ffd700 1 -- 172.21.15.49:0/3000784751 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498001e070 con 0x7f49900d3850 2023-07-18T03:40:30.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.661+0000 7f4986ffd700 1 -- 172.21.15.49:0/3000784751 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980003a60 con 0x7f49900d3850 2023-07-18T03:40:30.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.661+0000 7f499759d700 1 -- 172.21.15.49:0/3000784751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49900cbf50 con 0x7f49900d3850 2023-07-18T03:40:30.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.661+0000 7f499759d700 1 -- 172.21.15.49:0/3000784751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49900cc510 con 0x7f49900d3850 2023-07-18T03:40:30.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.662+0000 7f499759d700 1 -- 172.21.15.49:0/3000784751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499004e210 con 0x7f49900d3850 2023-07-18T03:40:30.667 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:40:30.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.663+0000 7f4986ffd700 1 -- 172.21.15.49:0/3000784751 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f4980003bc0 con 0x7f49900d3850 2023-07-18T03:40:30.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.665+0000 7f4986ffd700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f497406ad70 0x7f497406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:30.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.665+0000 7f4986ffd700 1 -- 172.21.15.49:0/3000784751 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f498008a2a0 con 0x7f49900d3850 2023-07-18T03:40:30.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.665+0000 7f499659b700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f497406ad70 0x7f497406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:30.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.666+0000 7f499659b700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f497406ad70 0x7f497406d220 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f498c005910 tx=0x7f498c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:30.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.670+0000 7f4986ffd700 1 -- 172.21.15.49:0/3000784751 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4980055610 con 0x7f49900d3850 2023-07-18T03:40:30.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.880+0000 7f499759d700 1 -- 172.21.15.49:0/3000784751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f49900b55b0 con 0x7f49900d3850 2023-07-18T03:40:30.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.884+0000 7f4986ffd700 1 -- 172.21.15.49:0/3000784751 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4980058c30 con 0x7f49900d3850 2023-07-18T03:40:30.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.886+0000 7f499759d700 1 -- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f497406ad70 msgr2=0x7f497406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:30.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.886+0000 7f499759d700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f497406ad70 0x7f497406d220 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f498c005910 tx=0x7f498c009040 comp rx=0 tx=0).stop 2023-07-18T03:40:30.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.886+0000 7f499759d700 1 -- 172.21.15.49:0/3000784751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49900d3850 msgr2=0x7f4990148180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:30.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.886+0000 7f499759d700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49900d3850 0x7f4990148180 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f49800090b0 tx=0x7f4980004750 comp rx=0 tx=0).stop 2023-07-18T03:40:30.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.886+0000 7f499759d700 1 -- 172.21.15.49:0/3000784751 shutdown_connections 2023-07-18T03:40:30.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.886+0000 7f499759d700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4990156fe0 0x7f4990144db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.886+0000 7f499759d700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900cfc90 0x7f49900cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.887+0000 7f499759d700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f497406ad70 0x7f497406d220 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.887+0000 7f499759d700 1 --2- 172.21.15.49:0/3000784751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49900d3850 0x7f4990148180 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.887+0000 7f499759d700 1 -- 172.21.15.49:0/3000784751 >> 172.21.15.49:0/3000784751 conn(0x7f4990065bd0 msgr2=0x7f499015bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:30.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.887+0000 7f499759d700 1 -- 172.21.15.49:0/3000784751 shutdown_connections 2023-07-18T03:40:30.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:30.887+0000 7f499759d700 1 -- 172.21.15.49:0/3000784751 wait complete. 2023-07-18T03:40:30.899 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2023-07-18T03:40:31.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.111+0000 7fce5ffff700 1 Processor -- start 2023-07-18T03:40:31.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.111+0000 7fce5ffff700 1 -- start start 2023-07-18T03:40:31.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.112+0000 7fce5ffff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fce580b4510 0x7fce580b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:31.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.112+0000 7fce5ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce580b5d90 0x7fce580cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:31.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.112+0000 7fce5ffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fce580d0b10 0x7fce580d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:31.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.112+0000 7fce5ffff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce580b48e0 con 0x7fce580d0b10 2023-07-18T03:40:31.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.112+0000 7fce5ffff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce580d14d0 con 0x7fce580b4510 2023-07-18T03:40:31.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.112+0000 7fce5ffff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce580b7f50 con 0x7fce580b5d90 2023-07-18T03:40:31.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.112+0000 7fce5e59c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fce580d0b10 0x7fce580d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:31.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.112+0000 7fce5e59c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fce580d0b10 0x7fce580d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32952/0 (socket says 172.21.15.49:32952) 2023-07-18T03:40:31.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.112+0000 7fce5e59c700 1 -- 172.21.15.49:0/1292025897 learned_addr learned my addr 172.21.15.49:0/1292025897 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:31.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.113+0000 7fce5dd9b700 1 --2- 172.21.15.49:0/1292025897 >> v2:172.21.15.107:3300/0 conn(0x7fce580b4510 0x7fce580b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:31.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.113+0000 7fce5d59a700 1 --2- 172.21.15.49:0/1292025897 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce580b5d90 0x7fce580cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:31.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.113+0000 7fce5e59c700 1 -- 172.21.15.49:0/1292025897 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce580b5d90 msgr2=0x7fce580cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:31.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.113+0000 7fce5e59c700 1 --2- 172.21.15.49:0/1292025897 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce580b5d90 0x7fce580cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.113+0000 7fce5e59c700 1 -- 172.21.15.49:0/1292025897 >> v2:172.21.15.107:3300/0 conn(0x7fce580b4510 msgr2=0x7fce580b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.113+0000 7fce5e59c700 1 --2- 172.21.15.49:0/1292025897 >> v2:172.21.15.107:3300/0 conn(0x7fce580b4510 0x7fce580b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.113+0000 7fce5e59c700 1 -- 172.21.15.49:0/1292025897 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce580b7340 con 0x7fce580d0b10 2023-07-18T03:40:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.113+0000 7fce5e59c700 1 --2- 172.21.15.49:0/1292025897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce580d0b10 0x7fce580d0f90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fce5808bd80 tx=0x7fce5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d98e3f12afaaec server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.114+0000 7fce5cd99700 1 -- 172.21.15.49:0/1292025897 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5000e070 con 0x7fce580d0b10 2023-07-18T03:40:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.114+0000 7fce5cd99700 1 -- 172.21.15.49:0/1292025897 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5000b910 con 0x7fce580d0b10 2023-07-18T03:40:31.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.114+0000 7fce5cd99700 1 -- 172.21.15.49:0/1292025897 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5000bcc0 con 0x7fce580d0b10 2023-07-18T03:40:31.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.114+0000 7fce5ffff700 1 -- 172.21.15.49:0/1292025897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce580d0b10 msgr2=0x7fce580d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:31.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.114+0000 7fce5ffff700 1 --2- 172.21.15.49:0/1292025897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce580d0b10 0x7fce580d0f90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fce5808bd80 tx=0x7fce5000b330 comp rx=0 tx=0).stop 2023-07-18T03:40:31.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.114+0000 7fce5ffff700 1 -- 172.21.15.49:0/1292025897 shutdown_connections 2023-07-18T03:40:31.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.114+0000 7fce5ffff700 1 --2- 172.21.15.49:0/1292025897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce580d0b10 0x7fce580d0f90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:31.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.114+0000 7fce5ffff700 1 --2- 172.21.15.49:0/1292025897 >> v2:172.21.15.107:3300/0 conn(0x7fce580b4510 0x7fce580b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:31.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.114+0000 7fce5ffff700 1 --2- 172.21.15.49:0/1292025897 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce580b5d90 0x7fce580cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:31.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.114+0000 7fce5ffff700 1 -- 172.21.15.49:0/1292025897 >> 172.21.15.49:0/1292025897 conn(0x7fce5809c1d0 msgr2=0x7fce580d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:31.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.114+0000 7fce5ffff700 1 -- 172.21.15.49:0/1292025897 shutdown_connections 2023-07-18T03:40:31.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.114+0000 7fce5ffff700 1 -- 172.21.15.49:0/1292025897 wait complete. 2023-07-18T03:40:31.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.116+0000 7fce5ffff700 1 Processor -- start 2023-07-18T03:40:31.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.116+0000 7fce5ffff700 1 -- start start 2023-07-18T03:40:31.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.117+0000 7fce5ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce580b4510 0x7fce5814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:31.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.117+0000 7fce5ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce580b5d90 0x7fce580b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:31.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.117+0000 7fce5ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce580d0b10 0x7fce580b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:31.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.117+0000 7fce5ffff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce580a09b0 con 0x7fce580d0b10 2023-07-18T03:40:31.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.117+0000 7fce5ffff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce580c9430 con 0x7fce580b4510 2023-07-18T03:40:31.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.117+0000 7fce5ffff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5809dd10 con 0x7fce580b5d90 2023-07-18T03:40:31.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.118+0000 7fce5e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce580d0b10 0x7fce580b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:31.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.118+0000 7fce5e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce580d0b10 0x7fce580b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32956/0 (socket says 172.21.15.49:32956) 2023-07-18T03:40:31.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.118+0000 7fce5e59c700 1 -- 172.21.15.49:0/894421711 learned_addr learned my addr 172.21.15.49:0/894421711 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:31.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.118+0000 7fce5dd9b700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce580b4510 0x7fce5814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:31.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.118+0000 7fce5d59a700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce580b5d90 0x7fce580b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:31.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.118+0000 7fce5e59c700 1 -- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce580b5d90 msgr2=0x7fce580b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:31.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.118+0000 7fce5e59c700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce580b5d90 0x7fce580b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:31.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.118+0000 7fce5e59c700 1 -- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce580b4510 msgr2=0x7fce5814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:31.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.118+0000 7fce5e59c700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce580b4510 0x7fce5814ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:31.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.118+0000 7fce5e59c700 1 -- 172.21.15.49:0/894421711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce50009b30 con 0x7fce580d0b10 2023-07-18T03:40:31.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.118+0000 7fce5e59c700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce580d0b10 0x7fce580b31d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fce5000b680 tx=0x7fce50004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:31.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.119+0000 7fce4e7fc700 1 -- 172.21.15.49:0/894421711 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50004450 con 0x7fce580d0b10 2023-07-18T03:40:31.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.119+0000 7fce4e7fc700 1 -- 172.21.15.49:0/894421711 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce500045b0 con 0x7fce580d0b10 2023-07-18T03:40:31.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.119+0000 7fce4e7fc700 1 -- 172.21.15.49:0/894421711 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50004960 con 0x7fce580d0b10 2023-07-18T03:40:31.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.119+0000 7fce5ffff700 1 -- 172.21.15.49:0/894421711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce580c9150 con 0x7fce580d0b10 2023-07-18T03:40:31.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.119+0000 7fce5ffff700 1 -- 172.21.15.49:0/894421711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce580b06d0 con 0x7fce580d0b10 2023-07-18T03:40:31.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.120+0000 7fce5ffff700 1 -- 172.21.15.49:0/894421711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5804e280 con 0x7fce580d0b10 2023-07-18T03:40:31.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.121+0000 7fce4e7fc700 1 -- 172.21.15.49:0/894421711 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fce50025460 con 0x7fce580d0b10 2023-07-18T03:40:31.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.122+0000 7fce4e7fc700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fce3c06ad70 0x7fce3c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:31.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.123+0000 7fce4e7fc700 1 -- 172.21.15.49:0/894421711 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fce5008ab30 con 0x7fce580d0b10 2023-07-18T03:40:31.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.123+0000 7fce5dd9b700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fce3c06ad70 0x7fce3c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:31.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.124+0000 7fce5dd9b700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fce3c06ad70 0x7fce3c06d220 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fce54009aa0 tx=0x7fce540091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.127+0000 7fce4e7fc700 1 -- 172.21.15.49:0/894421711 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fce50055f30 con 0x7fce580d0b10 2023-07-18T03:40:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:31.415+0000 7fce5ffff700 1 -- 172.21.15.49:0/894421711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7fce580a0b80 con 0x7fce580d0b10 2023-07-18T03:40:32.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.424+0000 7fce4e7fc700 1 -- 172.21.15.49:0/894421711 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v130) v1 ==== 181+0+0 (secure 0 0 0) 0x7fce500200f0 con 0x7fce580d0b10 2023-07-18T03:40:32.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.427+0000 7fce5ffff700 1 -- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fce3c06ad70 msgr2=0x7fce3c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:32.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.427+0000 7fce5ffff700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fce3c06ad70 0x7fce3c06d220 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fce54009aa0 tx=0x7fce540091c0 comp rx=0 tx=0).stop 2023-07-18T03:40:32.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.427+0000 7fce5ffff700 1 -- 172.21.15.49:0/894421711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce580d0b10 msgr2=0x7fce580b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:32.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.427+0000 7fce5ffff700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce580d0b10 0x7fce580b31d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fce5000b680 tx=0x7fce50004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:32.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.427+0000 7fce5ffff700 1 -- 172.21.15.49:0/894421711 shutdown_connections 2023-07-18T03:40:32.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.427+0000 7fce5ffff700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fce3c06ad70 0x7fce3c06d220 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:32.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.427+0000 7fce5ffff700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce580d0b10 0x7fce580b31d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:32.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.427+0000 7fce5ffff700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce580b4510 0x7fce5814ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:32.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.427+0000 7fce5ffff700 1 --2- 172.21.15.49:0/894421711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce580b5d90 0x7fce580b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:32.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.427+0000 7fce5ffff700 1 -- 172.21.15.49:0/894421711 >> 172.21.15.49:0/894421711 conn(0x7fce5809c1d0 msgr2=0x7fce58160830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:32.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.428+0000 7fce5ffff700 1 -- 172.21.15.49:0/894421711 shutdown_connections 2023-07-18T03:40:32.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.428+0000 7fce5ffff700 1 -- 172.21.15.49:0/894421711 wait complete. 2023-07-18T03:40:32.443 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... ok 2023-07-18T03:40:32.444 INFO:tasks.cephfs_test_runner:Starting test: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-07-18T03:40:32.444 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2023-07-18T03:40:32.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.661+0000 7f7a6b5ff700 1 Processor -- start 2023-07-18T03:40:32.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.661+0000 7f7a6b5ff700 1 -- start start 2023-07-18T03:40:32.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.662+0000 7f7a6b5ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7a640b4510 0x7f7a640b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:32.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.662+0000 7f7a6b5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7a640b5d90 0x7f7a640cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:32.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.662+0000 7f7a6b5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a640b7f50 con 0x7f7a640d0b10 2023-07-18T03:40:32.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.662+0000 7f7a6b5ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a640b48e0 con 0x7f7a640b4510 2023-07-18T03:40:32.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.662+0000 7f7a6b5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a640b7340 con 0x7f7a640b5d90 2023-07-18T03:40:32.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.662+0000 7f7a6939b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7a640b4510 0x7f7a640b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:32.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.662+0000 7f7a6939b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7a640b4510 0x7f7a640b6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32966/0 (socket says 172.21.15.49:32966) 2023-07-18T03:40:32.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.662+0000 7f7a6939b700 1 -- 172.21.15.49:0/3268350531 learned_addr learned my addr 172.21.15.49:0/3268350531 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.663+0000 7f7a68b9a700 1 --2- 172.21.15.49:0/3268350531 >> v2:172.21.15.107:3300/0 conn(0x7f7a640b5d90 0x7f7a640cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.663+0000 7f7a6939b700 1 -- 172.21.15.49:0/3268350531 >> v2:172.21.15.107:3300/0 conn(0x7f7a640b5d90 msgr2=0x7f7a640cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.663+0000 7f7a6939b700 1 --2- 172.21.15.49:0/3268350531 >> v2:172.21.15.107:3300/0 conn(0x7f7a640b5d90 0x7f7a640cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.663+0000 7f7a6939b700 1 -- 172.21.15.49:0/3268350531 >> v1:172.21.15.107:6789/0 conn(0x7f7a640d0b10 legacy=0x7f7a640b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:32.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.663+0000 7f7a6939b700 1 -- 172.21.15.49:0/3268350531 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a640b7620 con 0x7f7a640b4510 2023-07-18T03:40:32.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.663+0000 7f7a68b9a700 1 --2- 172.21.15.49:0/3268350531 >> v2:172.21.15.107:3300/0 conn(0x7f7a640b5d90 0x7f7a640cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:32.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.663+0000 7f7a6939b700 1 --2- 172.21.15.49:0/3268350531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a640b4510 0x7f7a640b6c80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7a640b5b30 tx=0x7f7a60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff39ce325b48929 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:32.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.663+0000 7f7a5bfff700 1 -- 172.21.15.49:0/3268350531 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60005e30 con 0x7f7a640b4510 2023-07-18T03:40:32.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.663+0000 7f7a5bfff700 1 -- 172.21.15.49:0/3268350531 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a60006030 con 0x7f7a640b4510 2023-07-18T03:40:32.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.663+0000 7f7a5bfff700 1 -- 172.21.15.49:0/3268350531 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a600063e0 con 0x7f7a640b4510 2023-07-18T03:40:32.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.664+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3268350531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a640b4510 msgr2=0x7f7a640b6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:32.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.664+0000 7f7a6b5ff700 1 --2- 172.21.15.49:0/3268350531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a640b4510 0x7f7a640b6c80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7a640b5b30 tx=0x7f7a60009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:32.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.664+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3268350531 shutdown_connections 2023-07-18T03:40:32.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.664+0000 7f7a6b5ff700 1 --2- 172.21.15.49:0/3268350531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a640b4510 0x7f7a640b6c80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:32.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.664+0000 7f7a6b5ff700 1 --2- 172.21.15.49:0/3268350531 >> v2:172.21.15.107:3300/0 conn(0x7f7a640b5d90 0x7f7a640cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:32.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.664+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3268350531 >> 172.21.15.49:0/3268350531 conn(0x7f7a6409c1d0 msgr2=0x7f7a640d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:32.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.664+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3268350531 shutdown_connections 2023-07-18T03:40:32.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.664+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3268350531 wait complete. 2023-07-18T03:40:32.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.666+0000 7f7a6b5ff700 1 Processor -- start 2023-07-18T03:40:32.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.666+0000 7f7a6b5ff700 1 -- start start 2023-07-18T03:40:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.667+0000 7f7a6b5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a640b4510 0x7f7a64154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.667+0000 7f7a6b5ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a640b5d90 0x7f7a640b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.667+0000 7f7a6b5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a640d0b10 0x7f7a641f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.667+0000 7f7a6b5ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a640b0440 con 0x7f7a640b5d90 2023-07-18T03:40:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.667+0000 7f7a6b5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a640b3460 con 0x7f7a640d0b10 2023-07-18T03:40:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.667+0000 7f7a6b5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a640b3740 con 0x7f7a640b4510 2023-07-18T03:40:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.667+0000 7f7a68b9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a640b5d90 0x7f7a640b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.667+0000 7f7a68b9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a640b5d90 0x7f7a640b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32982/0 (socket says 172.21.15.49:32982) 2023-07-18T03:40:32.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.667+0000 7f7a68b9a700 1 -- 172.21.15.49:0/3197454987 learned_addr learned my addr 172.21.15.49:0/3197454987 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:32.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a69b9c700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a640d0b10 0x7f7a641f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:32.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a6939b700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a640b4510 0x7f7a64154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:32.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a68b9a700 1 -- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a640b4510 msgr2=0x7f7a64154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:32.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a68b9a700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a640b4510 0x7f7a64154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:32.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a68b9a700 1 -- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a640d0b10 msgr2=0x7f7a641f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:32.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a68b9a700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a640d0b10 0x7f7a641f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:32.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a68b9a700 1 -- 172.21.15.49:0/3197454987 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a60009500 con 0x7f7a640b5d90 2023-07-18T03:40:32.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a68b9a700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a640b5d90 0x7f7a640b2b60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400c7f0 tx=0x7f7a5400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:32.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a59ffb700 1 -- 172.21.15.49:0/3197454987 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54011800 con 0x7f7a640b5d90 2023-07-18T03:40:32.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a59ffb700 1 -- 172.21.15.49:0/3197454987 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a54011960 con 0x7f7a640b5d90 2023-07-18T03:40:32.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3197454987 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a640b0690 con 0x7f7a640b5d90 2023-07-18T03:40:32.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a59ffb700 1 -- 172.21.15.49:0/3197454987 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54011d10 con 0x7f7a640b5d90 2023-07-18T03:40:32.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.668+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3197454987 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a64155230 con 0x7f7a640b5d90 2023-07-18T03:40:32.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.669+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3197454987 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a64072410 con 0x7f7a640b5d90 2023-07-18T03:40:32.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.670+0000 7f7a59ffb700 1 -- 172.21.15.49:0/3197454987 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f7a54018360 con 0x7f7a640b5d90 2023-07-18T03:40:32.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.671+0000 7f7a59ffb700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7a4806ad70 0x7f7a4806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:32.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.671+0000 7f7a59ffb700 1 -- 172.21.15.49:0/3197454987 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7a54089330 con 0x7f7a640b5d90 2023-07-18T03:40:32.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.672+0000 7f7a6939b700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7a4806ad70 0x7f7a4806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:32.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.672+0000 7f7a6939b700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7a4806ad70 0x7f7a4806d220 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a640b5b30 tx=0x7f7a60008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:32.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.676+0000 7f7a59ffb700 1 -- 172.21.15.49:0/3197454987 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7a54057f90 con 0x7f7a640b5d90 2023-07-18T03:40:32.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:32.966+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3197454987 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7f7a6404e280 con 0x7f7a640b5d90 2023-07-18T03:40:33.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.426+0000 7f7a59ffb700 1 -- 172.21.15.49:0/3197454987 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v131) v1 ==== 175+0+0 (secure 0 0 0) 0x7f7a54057b20 con 0x7f7a640b5d90 2023-07-18T03:40:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.429+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7a4806ad70 msgr2=0x7f7a4806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.429+0000 7f7a6b5ff700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7a4806ad70 0x7f7a4806d220 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a640b5b30 tx=0x7f7a60008070 comp rx=0 tx=0).stop 2023-07-18T03:40:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.429+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3197454987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a640b5d90 msgr2=0x7f7a640b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.429+0000 7f7a6b5ff700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a640b5d90 0x7f7a640b2b60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400c7f0 tx=0x7f7a5400cbb0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.429+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3197454987 shutdown_connections 2023-07-18T03:40:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.429+0000 7f7a6b5ff700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a640d0b10 0x7f7a641f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.429+0000 7f7a6b5ff700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a640b4510 0x7f7a64154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.429+0000 7f7a6b5ff700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7a4806ad70 0x7f7a4806d220 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.429+0000 7f7a6b5ff700 1 --2- 172.21.15.49:0/3197454987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a640b5d90 0x7f7a640b2b60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.429+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3197454987 >> 172.21.15.49:0/3197454987 conn(0x7f7a6409c1d0 msgr2=0x7f7a640d3580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.429+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3197454987 shutdown_connections 2023-07-18T03:40:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.429+0000 7f7a6b5ff700 1 -- 172.21.15.49:0/3197454987 wait complete. 2023-07-18T03:40:33.444 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-18T03:40:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.661+0000 7f6b1a3ff700 1 Processor -- start 2023-07-18T03:40:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.661+0000 7f6b1a3ff700 1 -- start start 2023-07-18T03:40:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.662+0000 7f6b1a3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6b140d0b10 0x7f6b140d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:33.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.662+0000 7f6b1a3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b140b0e20 0x7f6b140b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.662+0000 7f6b1a3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b140b8e30 con 0x7f6b140b8a60 2023-07-18T03:40:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.662+0000 7f6b1a3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b140d14d0 con 0x7f6b140d0b10 2023-07-18T03:40:33.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.662+0000 7f6b1a3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b140afaa0 con 0x7f6b140b0e20 2023-07-18T03:40:33.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.663+0000 7f6b1899c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b140b0e20 0x7f6b140b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:33.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.663+0000 7f6b13fff700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f6b140b8a60 0x7f6b140b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:34658/0 (socket says 172.21.15.49:34658) 2023-07-18T03:40:33.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.663+0000 7f6b13fff700 1 -- 172.21.15.49:0/205629128 learned_addr learned my addr 172.21.15.49:0/205629128 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:33.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.663+0000 7f6b137fe700 1 --2- 172.21.15.49:0/205629128 >> v2:172.21.15.107:3300/0 conn(0x7f6b140d0b10 0x7f6b140d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:33.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.663+0000 7f6b12ffd700 1 -- 172.21.15.49:0/205629128 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614653812 0 0) 0x7f6b140b8e30 con 0x7f6b140b8a60 2023-07-18T03:40:33.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.663+0000 7f6b12ffd700 1 -- 172.21.15.49:0/205629128 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6afc0035b0 con 0x7f6b140b8a60 2023-07-18T03:40:33.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.663+0000 7f6b1899c700 1 -- 172.21.15.49:0/205629128 >> v2:172.21.15.107:3300/0 conn(0x7f6b140d0b10 msgr2=0x7f6b140d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:33.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.663+0000 7f6b1899c700 1 --2- 172.21.15.49:0/205629128 >> v2:172.21.15.107:3300/0 conn(0x7f6b140d0b10 0x7f6b140d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.663+0000 7f6b1899c700 1 -- 172.21.15.49:0/205629128 >> v1:172.21.15.107:6789/0 conn(0x7f6b140b8a60 legacy=0x7f6b140b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:33.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.663+0000 7f6b1899c700 1 -- 172.21.15.49:0/205629128 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b140afd80 con 0x7f6b140b0e20 2023-07-18T03:40:33.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.663+0000 7f6b137fe700 1 --2- 172.21.15.49:0/205629128 >> v2:172.21.15.107:3300/0 conn(0x7f6b140d0b10 0x7f6b140d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:33.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.664+0000 7f6b1899c700 1 --2- 172.21.15.49:0/205629128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b140b0e20 0x7f6b140b12a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009d90 tx=0x7f6b0c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=714681e352dab2b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:33.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.664+0000 7f6b12ffd700 1 -- 172.21.15.49:0/205629128 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00e070 con 0x7f6b140b0e20 2023-07-18T03:40:33.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.664+0000 7f6b12ffd700 1 -- 172.21.15.49:0/205629128 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c00bb20 con 0x7f6b140b0e20 2023-07-18T03:40:33.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.664+0000 7f6b12ffd700 1 -- 172.21.15.49:0/205629128 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00be90 con 0x7f6b140b0e20 2023-07-18T03:40:33.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.664+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/205629128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b140b0e20 msgr2=0x7f6b140b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:33.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.664+0000 7f6b1a3ff700 1 --2- 172.21.15.49:0/205629128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b140b0e20 0x7f6b140b12a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009d90 tx=0x7f6b0c00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:33.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.665+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/205629128 shutdown_connections 2023-07-18T03:40:33.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.665+0000 7f6b1a3ff700 1 --2- 172.21.15.49:0/205629128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b140b0e20 0x7f6b140b12a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.665+0000 7f6b1a3ff700 1 --2- 172.21.15.49:0/205629128 >> v2:172.21.15.107:3300/0 conn(0x7f6b140d0b10 0x7f6b140d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.665+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/205629128 >> 172.21.15.49:0/205629128 conn(0x7f6b1409c1b0 msgr2=0x7f6b140d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:33.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.665+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/205629128 shutdown_connections 2023-07-18T03:40:33.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.665+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/205629128 wait complete. 2023-07-18T03:40:33.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.667+0000 7f6b1a3ff700 1 Processor -- start 2023-07-18T03:40:33.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.667+0000 7f6b1a3ff700 1 -- start start 2023-07-18T03:40:33.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1a3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b140b0e20 0x7f6b1415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:33.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1a3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b140b8a60 0x7f6b140cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:33.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1a3ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140d0b10 0x7f6b140cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:33.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1a3ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b140cdb60 con 0x7f6b140d0b10 2023-07-18T03:40:33.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1a3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b140cd570 con 0x7f6b140b8a60 2023-07-18T03:40:33.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1a3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b140ccaa0 con 0x7f6b140b0e20 2023-07-18T03:40:33.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1899c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140d0b10 0x7f6b140cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:33.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1899c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140d0b10 0x7f6b140cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32984/0 (socket says 172.21.15.49:32984) 2023-07-18T03:40:33.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1899c700 1 -- 172.21.15.49:0/2155886946 learned_addr learned my addr 172.21.15.49:0/2155886946 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:33.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b13fff700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b140b0e20 0x7f6b1415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:33.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b137fe700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b140b8a60 0x7f6b140cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:33.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1899c700 1 -- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b140b0e20 msgr2=0x7f6b1415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:33.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1899c700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b140b0e20 0x7f6b1415bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1899c700 1 -- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b140b8a60 msgr2=0x7f6b140cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:33.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.669+0000 7f6b1899c700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b140b8a60 0x7f6b140cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.670+0000 7f6b1899c700 1 -- 172.21.15.49:0/2155886946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b080094a0 con 0x7f6b140d0b10 2023-07-18T03:40:33.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.670+0000 7f6b137fe700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b140b8a60 0x7f6b140cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:33.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.670+0000 7f6b13fff700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b140b0e20 0x7f6b1415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:33.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.670+0000 7f6b1899c700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140d0b10 0x7f6b140cc420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c014040 tx=0x7f6b0c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:33.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.670+0000 7f6b10ff9700 1 -- 172.21.15.49:0/2155886946 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c004960 con 0x7f6b140d0b10 2023-07-18T03:40:33.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.670+0000 7f6b10ff9700 1 -- 172.21.15.49:0/2155886946 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c020070 con 0x7f6b140d0b10 2023-07-18T03:40:33.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.670+0000 7f6b10ff9700 1 -- 172.21.15.49:0/2155886946 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0276c0 con 0x7f6b140d0b10 2023-07-18T03:40:33.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.670+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/2155886946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c009a70 con 0x7f6b140d0b10 2023-07-18T03:40:33.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.670+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/2155886946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b140cd780 con 0x7f6b140d0b10 2023-07-18T03:40:33.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.671+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/2155886946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b14074350 con 0x7f6b140d0b10 2023-07-18T03:40:33.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.672+0000 7f6b10ff9700 1 -- 172.21.15.49:0/2155886946 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f6b0c012430 con 0x7f6b140d0b10 2023-07-18T03:40:33.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.673+0000 7f6b10ff9700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6b0006ad70 0x7f6b0006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:33.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.673+0000 7f6b10ff9700 1 -- 172.21.15.49:0/2155886946 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6b0c022070 con 0x7f6b140d0b10 2023-07-18T03:40:33.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.674+0000 7f6b13fff700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6b0006ad70 0x7f6b0006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:33.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.674+0000 7f6b13fff700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6b0006ad70 0x7f6b0006d220 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6b04005910 tx=0x7f6b04009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:33.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.677+0000 7f6b10ff9700 1 -- 172.21.15.49:0/2155886946 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6b0c056270 con 0x7f6b140d0b10 2023-07-18T03:40:33.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.888+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/2155886946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6b140b08f0 con 0x7f6b140d0b10 2023-07-18T03:40:33.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.892+0000 7f6b10ff9700 1 -- 172.21.15.49:0/2155886946 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6b0c02c0c0 con 0x7f6b140d0b10 2023-07-18T03:40:33.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.894+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6b0006ad70 msgr2=0x7f6b0006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:33.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.894+0000 7f6b1a3ff700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6b0006ad70 0x7f6b0006d220 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6b04005910 tx=0x7f6b04009040 comp rx=0 tx=0).stop 2023-07-18T03:40:33.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.894+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/2155886946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140d0b10 msgr2=0x7f6b140cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:33.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.894+0000 7f6b1a3ff700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140d0b10 0x7f6b140cc420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c014040 tx=0x7f6b0c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.894+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/2155886946 shutdown_connections 2023-07-18T03:40:33.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.894+0000 7f6b1a3ff700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f6b0006ad70 0x7f6b0006d220 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.894+0000 7f6b1a3ff700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140d0b10 0x7f6b140cc420 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.894+0000 7f6b1a3ff700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b140b0e20 0x7f6b1415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.894+0000 7f6b1a3ff700 1 --2- 172.21.15.49:0/2155886946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b140b8a60 0x7f6b140cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:33.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.894+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/2155886946 >> 172.21.15.49:0/2155886946 conn(0x7f6b1409c1b0 msgr2=0x7f6b140b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:33.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.894+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/2155886946 shutdown_connections 2023-07-18T03:40:33.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:33.895+0000 7f6b1a3ff700 1 -- 172.21.15.49:0/2155886946 wait complete. 2023-07-18T03:40:33.907 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-18T03:40:33.907 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-18T03:40:34.007 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-18T03:40:34.112 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-18T03:40:34.112 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:40:34.112 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:34.157 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345544k,nr_inodes=4086386,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-18T03:40:34.157 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:40:34.158 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:34.201 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345444k,nr_inodes=4086361,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-18T03:40:34.202 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:40:34.202 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:34.245 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345544k,nr_inodes=4086386,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-18T03:40:34.246 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:40:34.246 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:34.289 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345444k,nr_inodes=4086361,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-18T03:40:34.472 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-18T03:40:34.473 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:40:34.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.685+0000 7fc6963ff700 1 Processor -- start 2023-07-18T03:40:34.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.685+0000 7fc6963ff700 1 -- start start 2023-07-18T03:40:34.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.687+0000 7fc6963ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc6900b3e90 0x7fc6900cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:34.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.687+0000 7fc6963ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69015b430 con 0x7fc69015e890 2023-07-18T03:40:34.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.687+0000 7fc6963ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6900ce110 con 0x7fc69015a8c0 2023-07-18T03:40:34.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.687+0000 7fc6963ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6900cf380 con 0x7fc6900b3e90 2023-07-18T03:40:34.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.687+0000 7fc68f7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc6900b3e90 0x7fc6900cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:34.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.687+0000 7fc68ffff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc69015e890 0x7fc6900b3780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35536/0 (socket says 172.21.15.49:35536) 2023-07-18T03:40:34.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.687+0000 7fc68ffff700 1 -- 172.21.15.49:0/1069655949 learned_addr learned my addr 172.21.15.49:0/1069655949 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:34.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.687+0000 7fc68f7fe700 1 -- 172.21.15.49:0/1069655949 >> v1:172.21.15.107:6789/0 conn(0x7fc69015a8c0 legacy=0x7fc69015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:34.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.687+0000 7fc68f7fe700 1 -- 172.21.15.49:0/1069655949 >> v1:172.21.15.49:6789/0 conn(0x7fc69015e890 legacy=0x7fc6900b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:34.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.687+0000 7fc68f7fe700 1 -- 172.21.15.49:0/1069655949 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69015b660 con 0x7fc6900b3e90 2023-07-18T03:40:34.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.687+0000 7fc68effd700 1 -- 172.21.15.49:0/1069655949 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469909679 0 0) 0x7fc69015b430 con 0x7fc69015e890 2023-07-18T03:40:34.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.688+0000 7fc68f7fe700 1 --2- 172.21.15.49:0/1069655949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6900b3e90 0x7fc6900cee40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc684009840 tx=0x7fc684009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97407a66b7487d77 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:34.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.688+0000 7fc68effd700 1 -- 172.21.15.49:0/1069655949 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684011830 con 0x7fc6900b3e90 2023-07-18T03:40:34.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.688+0000 7fc68effd700 1 -- 172.21.15.49:0/1069655949 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc684011990 con 0x7fc6900b3e90 2023-07-18T03:40:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.688+0000 7fc68effd700 1 -- 172.21.15.49:0/1069655949 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684011d20 con 0x7fc6900b3e90 2023-07-18T03:40:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.688+0000 7fc6963ff700 1 -- 172.21.15.49:0/1069655949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6900b3e90 msgr2=0x7fc6900cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:34.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.688+0000 7fc6963ff700 1 --2- 172.21.15.49:0/1069655949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6900b3e90 0x7fc6900cee40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc684009840 tx=0x7fc684009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.689+0000 7fc6963ff700 1 -- 172.21.15.49:0/1069655949 shutdown_connections 2023-07-18T03:40:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.689+0000 7fc6963ff700 1 --2- 172.21.15.49:0/1069655949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6900b3e90 0x7fc6900cee40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:34.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.689+0000 7fc6963ff700 1 -- 172.21.15.49:0/1069655949 >> 172.21.15.49:0/1069655949 conn(0x7fc690069f30 msgr2=0x7fc69006a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.689+0000 7fc6963ff700 1 -- 172.21.15.49:0/1069655949 shutdown_connections 2023-07-18T03:40:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.689+0000 7fc6963ff700 1 -- 172.21.15.49:0/1069655949 wait complete. 2023-07-18T03:40:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.690+0000 7fc6963ff700 1 Processor -- start 2023-07-18T03:40:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.691+0000 7fc6963ff700 1 -- start start 2023-07-18T03:40:34.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.691+0000 7fc6963ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6900b3e90 0x7fc690152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:34.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.691+0000 7fc6963ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc69015a8c0 0x7fc690151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:34.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.691+0000 7fc6963ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc69015e890 0x7fc690071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:34.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.691+0000 7fc6963ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc690151fc0 con 0x7fc69015e890 2023-07-18T03:40:34.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.691+0000 7fc6963ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6901e3c20 con 0x7fc69015a8c0 2023-07-18T03:40:34.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.691+0000 7fc6963ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6901e3f00 con 0x7fc6900b3e90 2023-07-18T03:40:34.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.692+0000 7fc69499c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc69015e890 0x7fc690071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:34.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.692+0000 7fc68f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc69015a8c0 0x7fc690151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:34.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.692+0000 7fc69499c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc69015e890 0x7fc690071300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33012/0 (socket says 172.21.15.49:33012) 2023-07-18T03:40:34.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.692+0000 7fc69499c700 1 -- 172.21.15.49:0/3014003007 learned_addr learned my addr 172.21.15.49:0/3014003007 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:34.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.692+0000 7fc68ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6900b3e90 0x7fc690152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:34.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.692+0000 7fc69499c700 1 -- 172.21.15.49:0/3014003007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6900b3e90 msgr2=0x7fc690152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:34.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.692+0000 7fc69499c700 1 --2- 172.21.15.49:0/3014003007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6900b3e90 0x7fc690152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:34.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.692+0000 7fc69499c700 1 -- 172.21.15.49:0/3014003007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc69015a8c0 msgr2=0x7fc690151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:34.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.692+0000 7fc69499c700 1 --2- 172.21.15.49:0/3014003007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc69015a8c0 0x7fc690151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:34.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.692+0000 7fc69499c700 1 -- 172.21.15.49:0/3014003007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840094a0 con 0x7fc69015e890 2023-07-18T03:40:34.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.693+0000 7fc69499c700 1 --2- 172.21.15.49:0/3014003007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc69015e890 0x7fc690071300 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc68800cc50 tx=0x7fc68800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:34.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.693+0000 7fc68cff9700 1 -- 172.21.15.49:0/3014003007 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688003c00 con 0x7fc69015e890 2023-07-18T03:40:34.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.693+0000 7fc68cff9700 1 -- 172.21.15.49:0/3014003007 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc688003d60 con 0x7fc69015e890 2023-07-18T03:40:34.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.693+0000 7fc68cff9700 1 -- 172.21.15.49:0/3014003007 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68800aab0 con 0x7fc69015e890 2023-07-18T03:40:34.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.693+0000 7fc6963ff700 1 -- 172.21.15.49:0/3014003007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6901e41e0 con 0x7fc69015e890 2023-07-18T03:40:34.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.693+0000 7fc6963ff700 1 -- 172.21.15.49:0/3014003007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6901e47a0 con 0x7fc69015e890 2023-07-18T03:40:34.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.694+0000 7fc6963ff700 1 -- 172.21.15.49:0/3014003007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69004e280 con 0x7fc69015e890 2023-07-18T03:40:34.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.695+0000 7fc68cff9700 1 -- 172.21.15.49:0/3014003007 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fc68800ac10 con 0x7fc69015e890 2023-07-18T03:40:34.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.696+0000 7fc68cff9700 1 --2- 172.21.15.49:0/3014003007 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc67c06ad70 0x7fc67c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:34.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.696+0000 7fc68cff9700 1 -- 172.21.15.49:0/3014003007 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc6880890f0 con 0x7fc69015e890 2023-07-18T03:40:34.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.697+0000 7fc68ffff700 1 --2- 172.21.15.49:0/3014003007 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc67c06ad70 0x7fc67c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:34.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.697+0000 7fc68ffff700 1 --2- 172.21.15.49:0/3014003007 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc67c06ad70 0x7fc67c06d220 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc680007c40 tx=0x7fc680007b90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:34.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.700+0000 7fc68cff9700 1 -- 172.21.15.49:0/3014003007 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc6880544d0 con 0x7fc69015e890 2023-07-18T03:40:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.955+0000 7fc6963ff700 1 -- 172.21.15.49:0/3014003007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc69006d240 con 0x7fc69015e890 2023-07-18T03:40:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.955+0000 7fc68cff9700 1 -- 172.21.15.49:0/3014003007 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fc688057af0 con 0x7fc69015e890 2023-07-18T03:40:34.950 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:34.950 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1095518881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1095518881},{"type":"v1","addr":"172.21.15.49:6834","nonce":1095518881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/843626338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":843626338},{"type":"v1","addr":"172.21.15.107:6835","nonce":843626338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3238680863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3238680863},{"type":"v1","addr":"172.21.15.107:6837","nonce":3238680863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":5407,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/576088231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":576088231},{"type":"v1","addr":"172.21.15.49:6835","nonce":576088231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[]} 2023-07-18T03:40:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.958+0000 7fc6963ff700 1 -- 172.21.15.49:0/3014003007 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc67c06ad70 msgr2=0x7fc67c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.958+0000 7fc6963ff700 1 --2- 172.21.15.49:0/3014003007 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc67c06ad70 0x7fc67c06d220 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc680007c40 tx=0x7fc680007b90 comp rx=0 tx=0).stop 2023-07-18T03:40:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.958+0000 7fc6963ff700 1 -- 172.21.15.49:0/3014003007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc69015e890 msgr2=0x7fc690071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.958+0000 7fc6963ff700 1 --2- 172.21.15.49:0/3014003007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc69015e890 0x7fc690071300 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc68800cc50 tx=0x7fc68800cf60 comp rx=0 tx=0).stop 2023-07-18T03:40:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.958+0000 7fc6963ff700 1 -- 172.21.15.49:0/3014003007 shutdown_connections 2023-07-18T03:40:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.958+0000 7fc6963ff700 1 --2- 172.21.15.49:0/3014003007 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc67c06ad70 0x7fc67c06d220 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.958+0000 7fc6963ff700 1 --2- 172.21.15.49:0/3014003007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc69015e890 0x7fc690071300 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.958+0000 7fc6963ff700 1 --2- 172.21.15.49:0/3014003007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6900b3e90 0x7fc690152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.958+0000 7fc6963ff700 1 --2- 172.21.15.49:0/3014003007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc69015a8c0 0x7fc690151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.958+0000 7fc6963ff700 1 -- 172.21.15.49:0/3014003007 >> 172.21.15.49:0/3014003007 conn(0x7fc690069f30 msgr2=0x7fc69015ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.958+0000 7fc6963ff700 1 -- 172.21.15.49:0/3014003007 shutdown_connections 2023-07-18T03:40:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:34.958+0000 7fc6963ff700 1 -- 172.21.15.49:0/3014003007 wait complete. 2023-07-18T03:40:34.955 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 21 2023-07-18T03:40:34.970 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-18T03:40:34.971 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-18T03:40:34.971 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-18T03:40:34.971 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:40:34.972 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-18T03:40:34.972 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-18T03:40:34.972 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-18T03:40:34.972 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:40:34.973 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-18T03:40:34.973 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-18T03:40:34.973 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-18T03:40:34.973 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:40:34.974 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-18T03:40:34.974 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-18T03:40:34.974 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-18T03:40:34.974 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:40:34.987 INFO:tasks.ceph.mds.a:Stopped 2023-07-18T03:40:34.988 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-18T03:40:34.989 INFO:tasks.ceph.mds.c:Stopped 2023-07-18T03:40:34.989 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-18T03:40:34.991 INFO:tasks.ceph.mds.b:Stopped 2023-07-18T03:40:34.992 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-18T03:40:34.993 INFO:tasks.ceph.mds.d:Stopped 2023-07-18T03:40:34.994 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-18T03:40:35.033 INFO:tasks.ceph.mds.a:Started 2023-07-18T03:40:35.034 DEBUG:teuthology.parallel:result is None 2023-07-18T03:40:35.036 INFO:tasks.ceph.mds.c:Started 2023-07-18T03:40:35.036 DEBUG:teuthology.parallel:result is None 2023-07-18T03:40:35.038 INFO:tasks.ceph.mds.b:Started 2023-07-18T03:40:35.039 DEBUG:teuthology.parallel:result is None 2023-07-18T03:40:35.042 INFO:tasks.ceph.mds.d:Started 2023-07-18T03:40:35.043 DEBUG:teuthology.parallel:result is None 2023-07-18T03:40:35.043 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-18T03:40:35.188 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2023-07-18T03:40:35.189 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2023-07-18T03:40:35.189 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2023-07-18T03:40:35.190 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-07-18T03:40:35.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.263+0000 7f9f6f9ff700 1 Processor -- start 2023-07-18T03:40:35.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.264+0000 7f9f6f9ff700 1 -- start start 2023-07-18T03:40:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.265+0000 7f9f6f9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f680b8660 0x7f9f680cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.265+0000 7f9f6f9ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f680d02a0 con 0x7f9f680b4f70 2023-07-18T03:40:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.265+0000 7f9f6f9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f680cfce0 con 0x7f9f680cd970 2023-07-18T03:40:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.265+0000 7f9f6f9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f680cffc0 con 0x7f9f680b8660 2023-07-18T03:40:35.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.265+0000 7f9f6df9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9f680b4f70 0x7f9f680d12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35546/0 (socket says 172.21.15.49:35546) 2023-07-18T03:40:35.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.265+0000 7f9f6df9c700 1 -- 172.21.15.49:0/1377507846 learned_addr learned my addr 172.21.15.49:0/1377507846 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:35.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.265+0000 7f9f6d79b700 1 --2- 172.21.15.49:0/1377507846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f680b8660 0x7f9f680cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.266+0000 7f9f5ffff700 1 -- 172.21.15.49:0/1377507846 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352731671 0 0) 0x7f9f680cfce0 con 0x7f9f680cd970 2023-07-18T03:40:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.266+0000 7f9f6d79b700 1 -- 172.21.15.49:0/1377507846 >> v1:172.21.15.107:6789/0 conn(0x7f9f680cd970 legacy=0x7f9f680b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.266+0000 7f9f6d79b700 1 -- 172.21.15.49:0/1377507846 >> v1:172.21.15.49:6789/0 conn(0x7f9f680b4f70 legacy=0x7f9f680d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.266+0000 7f9f6d79b700 1 -- 172.21.15.49:0/1377507846 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f680d0ac0 con 0x7f9f680b8660 2023-07-18T03:40:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.266+0000 7f9f6d79b700 1 --2- 172.21.15.49:0/1377507846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f680b8660 0x7f9f680cd430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009dc0 tx=0x7f9f6400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f7bd25ed673cb2f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.266+0000 7f9f5ffff700 1 -- 172.21.15.49:0/1377507846 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6400e070 con 0x7f9f680b8660 2023-07-18T03:40:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.266+0000 7f9f5ffff700 1 -- 172.21.15.49:0/1377507846 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6400bad0 con 0x7f9f680b8660 2023-07-18T03:40:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.266+0000 7f9f5ffff700 1 -- 172.21.15.49:0/1377507846 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6400be80 con 0x7f9f680b8660 2023-07-18T03:40:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.267+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/1377507846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f680b8660 msgr2=0x7f9f680cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.267+0000 7f9f6f9ff700 1 --2- 172.21.15.49:0/1377507846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f680b8660 0x7f9f680cd430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009dc0 tx=0x7f9f6400b330 comp rx=0 tx=0).stop 2023-07-18T03:40:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.267+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/1377507846 shutdown_connections 2023-07-18T03:40:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.267+0000 7f9f6f9ff700 1 --2- 172.21.15.49:0/1377507846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f680b8660 0x7f9f680cd430 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.267+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/1377507846 >> 172.21.15.49:0/1377507846 conn(0x7f9f68066860 msgr2=0x7f9f6806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.267+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/1377507846 shutdown_connections 2023-07-18T03:40:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.267+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/1377507846 wait complete. 2023-07-18T03:40:35.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.269+0000 7f9f6f9ff700 1 Processor -- start 2023-07-18T03:40:35.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.269+0000 7f9f6f9ff700 1 -- start start 2023-07-18T03:40:35.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6f9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f680b4f70 0x7f9f68149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:35.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6f9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f680b8660 0x7f9f681591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:35.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6f9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f680cd970 0x7f9f681f1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:35.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6f9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f68159710 con 0x7f9f680cd970 2023-07-18T03:40:35.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6f9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f68159940 con 0x7f9f680b4f70 2023-07-18T03:40:35.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6f9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f68154b20 con 0x7f9f680b8660 2023-07-18T03:40:35.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6df9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f680cd970 0x7f9f681f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:35.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6d79b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f680b4f70 0x7f9f68149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:35.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6d79b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f680b4f70 0x7f9f68149310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:37710/0 (socket says 172.21.15.49:37710) 2023-07-18T03:40:35.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6cf9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f680b8660 0x7f9f681591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:35.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6d79b700 1 -- 172.21.15.49:0/2774131344 learned_addr learned my addr 172.21.15.49:0/2774131344 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:35.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6d79b700 1 -- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f680b8660 msgr2=0x7f9f681591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:35.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6d79b700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f680b8660 0x7f9f681591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:35.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6d79b700 1 -- 172.21.15.49:0/2774131344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f680cd970 msgr2=0x7f9f681f1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:35.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6d79b700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f680cd970 0x7f9f681f1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:35.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6d79b700 1 -- 172.21.15.49:0/2774131344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f64009a70 con 0x7f9f680b4f70 2023-07-18T03:40:35.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.270+0000 7f9f6df9c700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f680cd970 0x7f9f681f1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:35.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.271+0000 7f9f6d79b700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f680b4f70 0x7f9f68149310 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400b370 tx=0x7f9f64004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:35.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.271+0000 7f9f5dffb700 1 -- 172.21.15.49:0/2774131344 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f64012630 con 0x7f9f680b4f70 2023-07-18T03:40:35.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.271+0000 7f9f5dffb700 1 -- 172.21.15.49:0/2774131344 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f64012790 con 0x7f9f680b4f70 2023-07-18T03:40:35.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.271+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/2774131344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f681541f0 con 0x7f9f680b4f70 2023-07-18T03:40:35.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.271+0000 7f9f5dffb700 1 -- 172.21.15.49:0/2774131344 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f64012b40 con 0x7f9f680b4f70 2023-07-18T03:40:35.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.272+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/2774131344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f681547b0 con 0x7f9f680b4f70 2023-07-18T03:40:35.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.273+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/2774131344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f68074350 con 0x7f9f680b4f70 2023-07-18T03:40:35.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.274+0000 7f9f5dffb700 1 -- 172.21.15.49:0/2774131344 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f9f64012ca0 con 0x7f9f680b4f70 2023-07-18T03:40:35.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.275+0000 7f9f5dffb700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9f4c06ad70 0x7f9f4c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:35.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.275+0000 7f9f5dffb700 1 -- 172.21.15.49:0/2774131344 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9f6408b230 con 0x7f9f680b4f70 2023-07-18T03:40:35.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.275+0000 7f9f6cf9a700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9f4c06ad70 0x7f9f4c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:35.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.276+0000 7f9f6cf9a700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9f4c06ad70 0x7f9f4c06d220 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9f680d2870 tx=0x7f9f58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:35.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.279+0000 7f9f5dffb700 1 -- 172.21.15.49:0/2774131344 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9f640565d0 con 0x7f9f680b4f70 2023-07-18T03:40:35.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.503+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/2774131344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9f680b2150 con 0x7f9f680b4f70 2023-07-18T03:40:35.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.920+0000 7f9f5dffb700 1 -- 172.21.15.49:0/2774131344 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9f64059bf0 con 0x7f9f680b4f70 2023-07-18T03:40:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.924+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9f4c06ad70 msgr2=0x7f9f4c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.924+0000 7f9f6f9ff700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9f4c06ad70 0x7f9f4c06d220 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9f680d2870 tx=0x7f9f58009040 comp rx=0 tx=0).stop 2023-07-18T03:40:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.924+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f680b4f70 msgr2=0x7f9f68149310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.924+0000 7f9f6f9ff700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f680b4f70 0x7f9f68149310 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400b370 tx=0x7f9f64004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.924+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/2774131344 shutdown_connections 2023-07-18T03:40:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.924+0000 7f9f6f9ff700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9f4c06ad70 0x7f9f4c06d220 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.924+0000 7f9f6f9ff700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f680cd970 0x7f9f681f1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.924+0000 7f9f6f9ff700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f680b4f70 0x7f9f68149310 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.924+0000 7f9f6f9ff700 1 --2- 172.21.15.49:0/2774131344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f680b8660 0x7f9f681591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.924+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/2774131344 >> 172.21.15.49:0/2774131344 conn(0x7f9f68066860 msgr2=0x7f9f680c93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.924+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/2774131344 shutdown_connections 2023-07-18T03:40:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:35.924+0000 7f9f6f9ff700 1 -- 172.21.15.49:0/2774131344 wait complete. 2023-07-18T03:40:35.920 INFO:teuthology.orchestra.run.smithi049.stderr: removed all blocklist entries 2023-07-18T03:40:35.940 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-18T03:40:36.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.151+0000 7fc5c8bff700 1 Processor -- start 2023-07-18T03:40:36.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.152+0000 7fc5c8bff700 1 -- start start 2023-07-18T03:40:36.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.153+0000 7fc5c8bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc5c41500d0 0x7fc5c4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:36.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.153+0000 7fc5c8bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5c40cea50 0x7fc5c40ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:36.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.153+0000 7fc5c8bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c40cdf90 con 0x7fc5c4070890 2023-07-18T03:40:36.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.153+0000 7fc5c8bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5c415a2f0 con 0x7fc5c41500d0 2023-07-18T03:40:36.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.153+0000 7fc5c8bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c415a5d0 con 0x7fc5c40cea50 2023-07-18T03:40:36.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.154+0000 7fc5c259c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc5c41500d0 0x7fc5c4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:36.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.154+0000 7fc5c259c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc5c41500d0 0x7fc5c4070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:37726/0 (socket says 172.21.15.49:37726) 2023-07-18T03:40:36.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.154+0000 7fc5c259c700 1 -- 172.21.15.49:0/3532459833 learned_addr learned my addr 172.21.15.49:0/3532459833 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:36.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.154+0000 7fc5c2d9d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5c40cea50 0x7fc5c40ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:36.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.154+0000 7fc5bbfff700 1 --1- 172.21.15.49:0/3532459833 >> v1:172.21.15.107:6789/0 conn(0x7fc5c4070890 0x7fc5c41597c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:34704/0 (socket says 172.21.15.49:34704) 2023-07-18T03:40:36.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.154+0000 7fc5c259c700 1 -- 172.21.15.49:0/3532459833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5c40cea50 msgr2=0x7fc5c40ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:36.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.154+0000 7fc5c259c700 1 --2- 172.21.15.49:0/3532459833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5c40cea50 0x7fc5c40ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.154+0000 7fc5c259c700 1 -- 172.21.15.49:0/3532459833 >> v1:172.21.15.107:6789/0 conn(0x7fc5c4070890 legacy=0x7fc5c41597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:36.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.154+0000 7fc5c259c700 1 -- 172.21.15.49:0/3532459833 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c40ce250 con 0x7fc5c41500d0 2023-07-18T03:40:36.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.154+0000 7fc5c2d9d700 1 --2- 172.21.15.49:0/3532459833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5c40cea50 0x7fc5c40ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:36.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.154+0000 7fc5c259c700 1 --2- 172.21.15.49:0/3532459833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5c41500d0 0x7fc5c4070350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400bb90 tx=0x7fc5b400bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dd8b785c6906811 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:36.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.155+0000 7fc5c1d9b700 1 -- 172.21.15.49:0/3532459833 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400dd30 con 0x7fc5c41500d0 2023-07-18T03:40:36.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.155+0000 7fc5c1d9b700 1 -- 172.21.15.49:0/3532459833 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b400de90 con 0x7fc5c41500d0 2023-07-18T03:40:36.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.155+0000 7fc5c1d9b700 1 -- 172.21.15.49:0/3532459833 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4010690 con 0x7fc5c41500d0 2023-07-18T03:40:36.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.155+0000 7fc5c8bff700 1 -- 172.21.15.49:0/3532459833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5c41500d0 msgr2=0x7fc5c4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:36.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.155+0000 7fc5c8bff700 1 --2- 172.21.15.49:0/3532459833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5c41500d0 0x7fc5c4070350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400bb90 tx=0x7fc5b400bea0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.155+0000 7fc5c8bff700 1 -- 172.21.15.49:0/3532459833 shutdown_connections 2023-07-18T03:40:36.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.155+0000 7fc5c8bff700 1 --2- 172.21.15.49:0/3532459833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5c40cea50 0x7fc5c40ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.155+0000 7fc5c8bff700 1 --2- 172.21.15.49:0/3532459833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5c41500d0 0x7fc5c4070350 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.155+0000 7fc5c8bff700 1 -- 172.21.15.49:0/3532459833 >> 172.21.15.49:0/3532459833 conn(0x7fc5c4069f30 msgr2=0x7fc5c406a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:36.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.155+0000 7fc5c8bff700 1 -- 172.21.15.49:0/3532459833 shutdown_connections 2023-07-18T03:40:36.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.155+0000 7fc5c8bff700 1 -- 172.21.15.49:0/3532459833 wait complete. 2023-07-18T03:40:36.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.157+0000 7fc5c8bff700 1 Processor -- start 2023-07-18T03:40:36.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.157+0000 7fc5c8bff700 1 -- start start 2023-07-18T03:40:36.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.158+0000 7fc5c8bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5c4070890 0x7fc5c4140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:36.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.158+0000 7fc5c8bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c40cea50 0x7fc5c413e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:36.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.158+0000 7fc5c8bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5c41500d0 0x7fc5c413ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:36.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.158+0000 7fc5c8bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c413fef0 con 0x7fc5c40cea50 2023-07-18T03:40:36.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.158+0000 7fc5c8bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c41408c0 con 0x7fc5c41500d0 2023-07-18T03:40:36.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.158+0000 7fc5c8bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c413e7a0 con 0x7fc5c4070890 2023-07-18T03:40:36.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.158+0000 7fc5bbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c40cea50 0x7fc5c413e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:36.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.159+0000 7fc5bbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c40cea50 0x7fc5c413e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33052/0 (socket says 172.21.15.49:33052) 2023-07-18T03:40:36.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.159+0000 7fc5bbfff700 1 -- 172.21.15.49:0/359794569 learned_addr learned my addr 172.21.15.49:0/359794569 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:36.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.159+0000 7fc5c2d9d700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5c41500d0 0x7fc5c413ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:36.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.159+0000 7fc5c259c700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5c4070890 0x7fc5c4140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:36.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.159+0000 7fc5bbfff700 1 -- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5c4070890 msgr2=0x7fc5c4140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:36.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.159+0000 7fc5bbfff700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5c4070890 0x7fc5c4140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.159+0000 7fc5bbfff700 1 -- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5c41500d0 msgr2=0x7fc5c413ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:36.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.159+0000 7fc5bbfff700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5c41500d0 0x7fc5c413ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.159+0000 7fc5bbfff700 1 -- 172.21.15.49:0/359794569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc009a70 con 0x7fc5c40cea50 2023-07-18T03:40:36.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.159+0000 7fc5c259c700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5c4070890 0x7fc5c4140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:36.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.159+0000 7fc5bbfff700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c40cea50 0x7fc5c413e260 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c520 tx=0x7fc5ac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:36.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.159+0000 7fc5baffd700 1 -- 172.21.15.49:0/359794569 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac005cb0 con 0x7fc5c40cea50 2023-07-18T03:40:36.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.160+0000 7fc5c8bff700 1 -- 172.21.15.49:0/359794569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b400b7f0 con 0x7fc5c40cea50 2023-07-18T03:40:36.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.160+0000 7fc5baffd700 1 -- 172.21.15.49:0/359794569 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ac005e10 con 0x7fc5c40cea50 2023-07-18T03:40:36.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.160+0000 7fc5baffd700 1 -- 172.21.15.49:0/359794569 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac0072d0 con 0x7fc5c40cea50 2023-07-18T03:40:36.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.160+0000 7fc5c8bff700 1 -- 172.21.15.49:0/359794569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c413f5c0 con 0x7fc5c40cea50 2023-07-18T03:40:36.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.161+0000 7fc5c8bff700 1 -- 172.21.15.49:0/359794569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c404e280 con 0x7fc5c40cea50 2023-07-18T03:40:36.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.162+0000 7fc5baffd700 1 -- 172.21.15.49:0/359794569 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fc5ac007430 con 0x7fc5c40cea50 2023-07-18T03:40:36.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.163+0000 7fc5baffd700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc5a406ad70 0x7fc5a406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:36.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.163+0000 7fc5baffd700 1 -- 172.21.15.49:0/359794569 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc5ac0885e0 con 0x7fc5c40cea50 2023-07-18T03:40:36.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.163+0000 7fc5c259c700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc5a406ad70 0x7fc5a406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:36.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.164+0000 7fc5c259c700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc5a406ad70 0x7fc5a406d220 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400bee0 tx=0x7fc5b4013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:36.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.168+0000 7fc5baffd700 1 -- 172.21.15.49:0/359794569 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc5ac053920 con 0x7fc5c40cea50 2023-07-18T03:40:36.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.460+0000 7fc5c8bff700 1 -- 172.21.15.49:0/359794569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc5c4070130 con 0x7fc5c40cea50 2023-07-18T03:40:36.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.461+0000 7fc5baffd700 1 -- 172.21.15.49:0/359794569 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc5ac056f40 con 0x7fc5c40cea50 2023-07-18T03:40:36.455 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:36.455 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:40:36.455 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_dump": [ 2023-07-18T03:40:36.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.455 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.a", 2023-07-18T03:40:36.456 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD+CLZk0A9wAxAAPyxogWvN53TNCPyBYAgGgQ==", 2023-07-18T03:40:36.456 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.456 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:40:36.456 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-07-18T03:40:36.456 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-07-18T03:40:36.456 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.457 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.457 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.b", 2023-07-18T03:40:36.457 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD+CLZkWB96GxAAepmZ6nF9qh75BBmZTY1W4w==", 2023-07-18T03:40:36.457 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.457 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:40:36.457 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-07-18T03:40:36.458 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-07-18T03:40:36.458 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.458 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.458 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.c", 2023-07-18T03:40:36.458 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD+CLZkCKiYDxAAhmsGhQF1Uusa7eSwZKWIWw==", 2023-07-18T03:40:36.458 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.459 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:40:36.459 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-07-18T03:40:36.459 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-07-18T03:40:36.459 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.459 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.460 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.d", 2023-07-18T03:40:36.460 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD+CLZkKJwHKBAAusYdoX3eXJdWmNNsa7RuZA==", 2023-07-18T03:40:36.460 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.460 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:40:36.460 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-07-18T03:40:36.460 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-07-18T03:40:36.460 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.461 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.461 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.0", 2023-07-18T03:40:36.461 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAACbZkV1wVMhAAqSqjoEtRctkKCcFiUUOTcw==", 2023-07-18T03:40:36.461 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.461 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:40:36.462 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:40:36.462 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.463 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.464 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.1", 2023-07-18T03:40:36.464 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQADCbZk9S6TNhAAOl9IxSR58LnxX58i/sjMvg==", 2023-07-18T03:40:36.464 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.464 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:40:36.464 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:40:36.464 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.465 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.465 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.2", 2023-07-18T03:40:36.465 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAGCbZke4glOxAAASCGrrHrKsam5fv4gjdZ/A==", 2023-07-18T03:40:36.465 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.465 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:40:36.465 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:40:36.466 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.466 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.466 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.3", 2023-07-18T03:40:36.466 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAKCbZkP4iFAxAA/nPkDfrSOZeJm2QyEAGNiQ==", 2023-07-18T03:40:36.466 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.467 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:40:36.467 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:40:36.467 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.467 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.4", 2023-07-18T03:40:36.468 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAQCbZkxYX2GRAA1Y4sxGA6H05aa5jydM38WQ==", 2023-07-18T03:40:36.468 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.468 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:40:36.468 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:40:36.468 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.468 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.469 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.5", 2023-07-18T03:40:36.469 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQATCbZkt8qVIBAAeaCS5rmMJEgwZUjh3siVTw==", 2023-07-18T03:40:36.469 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.469 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:40:36.469 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:40:36.469 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.469 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.470 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.6", 2023-07-18T03:40:36.470 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAWCbZkK/qRKBAA7wpJMzxg/K+YMxEJHUeRPw==", 2023-07-18T03:40:36.470 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.470 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:40:36.470 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:40:36.471 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.471 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.471 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.7", 2023-07-18T03:40:36.471 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQAZCbZk2ltrMhAATvhB8r+uw5+LDGlMtJzEbw==", 2023-07-18T03:40:36.471 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.472 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-07-18T03:40:36.472 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-07-18T03:40:36.472 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.472 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.472 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.0", 2023-07-18T03:40:36.473 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD+CLZkI5fMMhAAisfdJO7kEPCDAngS1ywP+g==", 2023-07-18T03:40:36.473 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.473 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:40:36.473 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow r", 2023-07-18T03:40:36.473 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-18T03:40:36.473 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.474 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.1", 2023-07-18T03:40:36.474 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD/CLZkgQfPBhAAIkgoL4P4YFEqTZqP/8hYHQ==", 2023-07-18T03:40:36.474 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.475 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:40:36.475 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-07-18T03:40:36.475 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-07-18T03:40:36.475 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-07-18T03:40:36.475 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.475 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.2", 2023-07-18T03:40:36.476 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD+CLZk1mmtOhAAYC8ALN9saaob9naZYRC6Vg==", 2023-07-18T03:40:36.476 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:40:36.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-07-18T03:40:36.477 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-07-18T03:40:36.477 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-07-18T03:40:36.477 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.477 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.3", 2023-07-18T03:40:36.477 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD/CLZk1OaMDhAARMs8iCxSvTEiQomODnCtIg==", 2023-07-18T03:40:36.478 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-07-18T03:40:36.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-07-18T03:40:36.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-07-18T03:40:36.479 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-07-18T03:40:36.479 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.479 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.admin", 2023-07-18T03:40:36.479 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD9CLZk9BpYBhAAGOejpn6CQyH1hCyRmbyKrQ==", 2023-07-18T03:40:36.479 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-07-18T03:40:36.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-07-18T03:40:36.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-07-18T03:40:36.480 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.480 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.481 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.x", 2023-07-18T03:40:36.481 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD9CLZk6hvxNhAAMALkYp+kAehBGNNCw/BJfg==", 2023-07-18T03:40:36.481 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-07-18T03:40:36.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2023-07-18T03:40:36.481 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.481 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:36.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:36.482 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.y", 2023-07-18T03:40:36.482 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQD9CLZkATFALhAAqRgQzaCzvqgqsvTwhZ/+rw==", 2023-07-18T03:40:36.482 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-07-18T03:40:36.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-07-18T03:40:36.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2023-07-18T03:40:36.483 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-07-18T03:40:36.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:36.483 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-07-18T03:40:36.483 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:40:36.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.463+0000 7fc5c8bff700 1 -- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc5a406ad70 msgr2=0x7fc5a406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:36.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.463+0000 7fc5c8bff700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc5a406ad70 0x7fc5a406d220 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400bee0 tx=0x7fc5b4013000 comp rx=0 tx=0).stop 2023-07-18T03:40:36.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.463+0000 7fc5c8bff700 1 -- 172.21.15.49:0/359794569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c40cea50 msgr2=0x7fc5c413e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:36.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.463+0000 7fc5c8bff700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c40cea50 0x7fc5c413e260 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c520 tx=0x7fc5ac00c8e0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.463+0000 7fc5c8bff700 1 -- 172.21.15.49:0/359794569 shutdown_connections 2023-07-18T03:40:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.463+0000 7fc5c8bff700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5c41500d0 0x7fc5c413ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.463+0000 7fc5c8bff700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5c4070890 0x7fc5c4140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.463+0000 7fc5c8bff700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc5a406ad70 0x7fc5a406d220 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.463+0000 7fc5c8bff700 1 --2- 172.21.15.49:0/359794569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5c40cea50 0x7fc5c413e260 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.463+0000 7fc5c8bff700 1 -- 172.21.15.49:0/359794569 >> 172.21.15.49:0/359794569 conn(0x7fc5c4069f30 msgr2=0x7fc5c40ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.463+0000 7fc5c8bff700 1 -- 172.21.15.49:0/359794569 shutdown_connections 2023-07-18T03:40:36.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.463+0000 7fc5c8bff700 1 -- 172.21.15.49:0/359794569 wait complete. 2023-07-18T03:40:36.487 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:36.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.699+0000 7f72c75ff700 1 Processor -- start 2023-07-18T03:40:36.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.699+0000 7f72c75ff700 1 -- start start 2023-07-18T03:40:36.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.700+0000 7f72c75ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f72c00cf510 0x7f72c006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:36.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.700+0000 7f72c75ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72c006d960 0x7f72c0070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:36.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.700+0000 7f72c75ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c00cd320 con 0x7f72c015c710 2023-07-18T03:40:36.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.700+0000 7f72c75ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c00cf980 con 0x7f72c00cf510 2023-07-18T03:40:36.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.700+0000 7f72c75ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c006dde0 con 0x7f72c006d960 2023-07-18T03:40:36.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.701+0000 7f72c4b9a700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f72c00cf510 0x7f72c006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:36.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.701+0000 7f72c5b9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72c006d960 0x7f72c0070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:36.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.701+0000 7f72c539b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f72c015c710 0x7f72c0069f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:34720/0 (socket says 172.21.15.49:34720) 2023-07-18T03:40:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.701+0000 7f72c4b9a700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f72c00cf510 0x7f72c006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:37748/0 (socket says 172.21.15.49:37748) 2023-07-18T03:40:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.701+0000 7f72c539b700 1 -- 172.21.15.49:0/2235061988 learned_addr learned my addr 172.21.15.49:0/2235061988 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.702+0000 7f72c4b9a700 1 -- 172.21.15.49:0/2235061988 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72c006d960 msgr2=0x7f72c0070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.702+0000 7f72c4b9a700 1 --2- 172.21.15.49:0/2235061988 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72c006d960 0x7f72c0070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.702+0000 7f72c4b9a700 1 -- 172.21.15.49:0/2235061988 >> v1:172.21.15.107:6789/0 conn(0x7f72c015c710 legacy=0x7f72c0069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.702+0000 7f72c4b9a700 1 -- 172.21.15.49:0/2235061988 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c00ce750 con 0x7f72c00cf510 2023-07-18T03:40:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.702+0000 7f72c4b9a700 1 --2- 172.21.15.49:0/2235061988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c00cf510 0x7f72c006d420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009840 tx=0x7f72b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7f56dea443495c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:36.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.702+0000 7f72b7fff700 1 -- 172.21.15.49:0/2235061988 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0011830 con 0x7f72c00cf510 2023-07-18T03:40:36.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.702+0000 7f72b7fff700 1 -- 172.21.15.49:0/2235061988 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0011990 con 0x7f72c00cf510 2023-07-18T03:40:36.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.702+0000 7f72b7fff700 1 -- 172.21.15.49:0/2235061988 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0011d20 con 0x7f72c00cf510 2023-07-18T03:40:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.703+0000 7f72c75ff700 1 -- 172.21.15.49:0/2235061988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c00cf510 msgr2=0x7f72c006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.703+0000 7f72c75ff700 1 --2- 172.21.15.49:0/2235061988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c00cf510 0x7f72c006d420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009840 tx=0x7f72b0009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.703+0000 7f72c75ff700 1 -- 172.21.15.49:0/2235061988 shutdown_connections 2023-07-18T03:40:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.703+0000 7f72c75ff700 1 --2- 172.21.15.49:0/2235061988 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72c006d960 0x7f72c0070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.703+0000 7f72c75ff700 1 --2- 172.21.15.49:0/2235061988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c00cf510 0x7f72c006d420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.703+0000 7f72c75ff700 1 -- 172.21.15.49:0/2235061988 >> 172.21.15.49:0/2235061988 conn(0x7f72c00b16c0 msgr2=0x7f72c00b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:36.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.703+0000 7f72c75ff700 1 -- 172.21.15.49:0/2235061988 shutdown_connections 2023-07-18T03:40:36.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.703+0000 7f72c75ff700 1 -- 172.21.15.49:0/2235061988 wait complete. 2023-07-18T03:40:36.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.705+0000 7f72c75ff700 1 Processor -- start 2023-07-18T03:40:36.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.705+0000 7f72c75ff700 1 -- start start 2023-07-18T03:40:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c75ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c006d960 0x7f72c00b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c75ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72c00cf510 0x7f72c015e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:36.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c75ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72c015edc0 0x7f72c01f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:36.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c75ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c01525b0 con 0x7f72c00cf510 2023-07-18T03:40:36.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c75ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0142ec0 con 0x7f72c006d960 2023-07-18T03:40:36.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c75ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c015fb90 con 0x7f72c015edc0 2023-07-18T03:40:36.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c4b9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72c00cf510 0x7f72c015e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:36.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c4b9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72c00cf510 0x7f72c015e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33062/0 (socket says 172.21.15.49:33062) 2023-07-18T03:40:36.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c4b9a700 1 -- 172.21.15.49:0/503652902 learned_addr learned my addr 172.21.15.49:0/503652902 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c5b9c700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72c015edc0 0x7f72c01f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c539b700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c006d960 0x7f72c00b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c4b9a700 1 -- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72c015edc0 msgr2=0x7f72c01f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c4b9a700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72c015edc0 0x7f72c01f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c4b9a700 1 -- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c006d960 msgr2=0x7f72c00b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c4b9a700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c006d960 0x7f72c00b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.706+0000 7f72c4b9a700 1 -- 172.21.15.49:0/503652902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b00094a0 con 0x7f72c00cf510 2023-07-18T03:40:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.707+0000 7f72c4b9a700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72c00cf510 0x7f72c015e880 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009fd0 tx=0x7f72b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.707+0000 7f72b5ffb700 1 -- 172.21.15.49:0/503652902 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b00036a0 con 0x7f72c00cf510 2023-07-18T03:40:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.707+0000 7f72b5ffb700 1 -- 172.21.15.49:0/503652902 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b001f070 con 0x7f72c00cf510 2023-07-18T03:40:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.707+0000 7f72b5ffb700 1 -- 172.21.15.49:0/503652902 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0003a60 con 0x7f72c00cf510 2023-07-18T03:40:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.707+0000 7f72c75ff700 1 -- 172.21.15.49:0/503652902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c015f5d0 con 0x7f72c00cf510 2023-07-18T03:40:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.707+0000 7f72c75ff700 1 -- 172.21.15.49:0/503652902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c0158880 con 0x7f72c00cf510 2023-07-18T03:40:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.708+0000 7f72c75ff700 1 -- 172.21.15.49:0/503652902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c00703e0 con 0x7f72c00cf510 2023-07-18T03:40:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.709+0000 7f72b5ffb700 1 -- 172.21.15.49:0/503652902 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f72b0003bc0 con 0x7f72c00cf510 2023-07-18T03:40:36.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.710+0000 7f72b5ffb700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f72a406ad70 0x7f72a406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:36.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.710+0000 7f72b5ffb700 1 -- 172.21.15.49:0/503652902 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f72b0021070 con 0x7f72c00cf510 2023-07-18T03:40:36.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.711+0000 7f72c539b700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f72a406ad70 0x7f72a406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:36.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.712+0000 7f72c539b700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f72a406ad70 0x7f72a406d220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f72bc005910 tx=0x7f72bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:36.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.715+0000 7f72b5ffb700 1 -- 172.21.15.49:0/503652902 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f72b0055d60 con 0x7f72c00cf510 2023-07-18T03:40:36.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.925+0000 7f72c75ff700 1 -- 172.21.15.49:0/503652902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72c006aaa0 con 0x7f72c00cf510 2023-07-18T03:40:36.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.926+0000 7f72b5ffb700 1 -- 172.21.15.49:0/503652902 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+14545 (secure 0 0 0) 0x7f72b0059380 con 0x7f72c00cf510 2023-07-18T03:40:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:36.920 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":45,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:40:35.925120+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:36.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.928+0000 7f72c75ff700 1 -- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f72a406ad70 msgr2=0x7f72a406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:36.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.928+0000 7f72c75ff700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f72a406ad70 0x7f72a406d220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f72bc005910 tx=0x7f72bc009040 comp rx=0 tx=0).stop 2023-07-18T03:40:36.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.928+0000 7f72c75ff700 1 -- 172.21.15.49:0/503652902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72c00cf510 msgr2=0x7f72c015e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:36.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.928+0000 7f72c75ff700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72c00cf510 0x7f72c015e880 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009fd0 tx=0x7f72b0004750 comp rx=0 tx=0).stop 2023-07-18T03:40:36.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.928+0000 7f72c75ff700 1 -- 172.21.15.49:0/503652902 shutdown_connections 2023-07-18T03:40:36.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.928+0000 7f72c75ff700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f72c015edc0 0x7f72c01f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.928+0000 7f72c75ff700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c006d960 0x7f72c00b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.928+0000 7f72c75ff700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f72a406ad70 0x7f72a406d220 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.928+0000 7f72c75ff700 1 --2- 172.21.15.49:0/503652902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72c00cf510 0x7f72c015e880 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:36.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.928+0000 7f72c75ff700 1 -- 172.21.15.49:0/503652902 >> 172.21.15.49:0/503652902 conn(0x7f72c00b16c0 msgr2=0x7f72c00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:36.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.928+0000 7f72c75ff700 1 -- 172.21.15.49:0/503652902 shutdown_connections 2023-07-18T03:40:36.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:36.929+0000 7f72c75ff700 1 -- 172.21.15.49:0/503652902 wait complete. 2023-07-18T03:40:36.942 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:40:36.943 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:40:37.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.156+0000 7f14a9bff700 1 Processor -- start 2023-07-18T03:40:37.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.157+0000 7f14a9bff700 1 -- start start 2023-07-18T03:40:37.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.158+0000 7f14a9bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14a40cbe40 0x7f14a40cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.158+0000 7f14a9bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f14a40cdf80 0x7f14a406ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.158+0000 7f14a9bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a40ce8c0 con 0x7f14a4158bb0 2023-07-18T03:40:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.158+0000 7f14a9bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a406b5b0 con 0x7f14a40cdf80 2023-07-18T03:40:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.158+0000 7f14a9bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a40ce400 con 0x7f14a40cbe40 2023-07-18T03:40:37.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.159+0000 7f14a37fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f14a4158bb0 0x7f14a40cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35550/0 (socket says 172.21.15.49:35550) 2023-07-18T03:40:37.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.159+0000 7f14a37fe700 1 -- 172.21.15.49:0/1844876085 learned_addr learned my addr 172.21.15.49:0/1844876085 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.159+0000 7f149bfff700 1 --2- 172.21.15.49:0/1844876085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14a40cbe40 0x7f14a40cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.159+0000 7f14a3fff700 1 --2- 172.21.15.49:0/1844876085 >> v2:172.21.15.107:3300/0 conn(0x7f14a40cdf80 0x7f14a406ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.159+0000 7f14a2ffd700 1 -- 172.21.15.49:0/1844876085 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029486763 0 0) 0x7f14a40ce8c0 con 0x7f14a4158bb0 2023-07-18T03:40:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.159+0000 7f14a2ffd700 1 -- 172.21.15.49:0/1844876085 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14900035b0 con 0x7f14a4158bb0 2023-07-18T03:40:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.159+0000 7f149bfff700 1 -- 172.21.15.49:0/1844876085 >> v2:172.21.15.107:3300/0 conn(0x7f14a40cdf80 msgr2=0x7f14a406ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.159+0000 7f149bfff700 1 --2- 172.21.15.49:0/1844876085 >> v2:172.21.15.107:3300/0 conn(0x7f14a40cdf80 0x7f14a406ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.159+0000 7f14a2ffd700 1 -- 172.21.15.49:0/1844876085 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4293052398 0 0) 0x7f14900035b0 con 0x7f14a4158bb0 2023-07-18T03:40:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.159+0000 7f149bfff700 1 -- 172.21.15.49:0/1844876085 >> v1:172.21.15.49:6789/0 conn(0x7f14a4158bb0 legacy=0x7f14a40cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.160+0000 7f149bfff700 1 -- 172.21.15.49:0/1844876085 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a40cc2b0 con 0x7f14a40cbe40 2023-07-18T03:40:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.160+0000 7f149bfff700 1 --2- 172.21.15.49:0/1844876085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14a40cbe40 0x7f14a40cda40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f148c009840 tx=0x7f148c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6336ada928ad5bdd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.160+0000 7f14a2ffd700 1 -- 172.21.15.49:0/1844876085 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494001670 con 0x7f14a40cbe40 2023-07-18T03:40:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.160+0000 7f14a2ffd700 1 -- 172.21.15.49:0/1844876085 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c004030 con 0x7f14a40cbe40 2023-07-18T03:40:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.160+0000 7f14a2ffd700 1 -- 172.21.15.49:0/1844876085 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0043a0 con 0x7f14a40cbe40 2023-07-18T03:40:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.161+0000 7f14a9bff700 1 -- 172.21.15.49:0/1844876085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14a40cbe40 msgr2=0x7f14a40cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:37.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.161+0000 7f14a9bff700 1 --2- 172.21.15.49:0/1844876085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14a40cbe40 0x7f14a40cda40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f148c009840 tx=0x7f148c009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:37.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.161+0000 7f14a9bff700 1 -- 172.21.15.49:0/1844876085 shutdown_connections 2023-07-18T03:40:37.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.161+0000 7f14a9bff700 1 --2- 172.21.15.49:0/1844876085 >> v2:172.21.15.107:3300/0 conn(0x7f14a40cdf80 0x7f14a406ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.161+0000 7f14a9bff700 1 --2- 172.21.15.49:0/1844876085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14a40cbe40 0x7f14a40cda40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.161+0000 7f14a9bff700 1 -- 172.21.15.49:0/1844876085 >> 172.21.15.49:0/1844876085 conn(0x7f14a40b05d0 msgr2=0x7f14a40b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:37.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.161+0000 7f14a9bff700 1 -- 172.21.15.49:0/1844876085 shutdown_connections 2023-07-18T03:40:37.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.161+0000 7f14a9bff700 1 -- 172.21.15.49:0/1844876085 wait complete. 2023-07-18T03:40:37.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.163+0000 7f14a9bff700 1 Processor -- start 2023-07-18T03:40:37.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.163+0000 7f14a9bff700 1 -- start start 2023-07-18T03:40:37.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f14a9bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a40cbe40 0x7f14a40c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:37.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f14a9bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14a40cdf80 0x7f14a414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:37.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f14a9bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14a4158bb0 0x7f14a40b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:37.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f14a9bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a414aec0 con 0x7f14a40cdf80 2023-07-18T03:40:37.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f14a9bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a414b8f0 con 0x7f14a40cbe40 2023-07-18T03:40:37.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f14a9bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a414bb20 con 0x7f14a4158bb0 2023-07-18T03:40:37.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f149bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14a40cdf80 0x7f14a414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:37.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f149bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14a40cdf80 0x7f14a414b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33068/0 (socket says 172.21.15.49:33068) 2023-07-18T03:40:37.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f149bfff700 1 -- 172.21.15.49:0/129457382 learned_addr learned my addr 172.21.15.49:0/129457382 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:37.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f14a37fe700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a40cbe40 0x7f14a40c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:37.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f14a3fff700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14a4158bb0 0x7f14a40b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:37.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f149bfff700 1 -- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14a4158bb0 msgr2=0x7f14a40b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:37.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f149bfff700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14a4158bb0 0x7f14a40b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f149bfff700 1 -- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a40cbe40 msgr2=0x7f14a40c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:37.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f149bfff700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a40cbe40 0x7f14a40c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f149bfff700 1 -- 172.21.15.49:0/129457382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c0094a0 con 0x7f14a40cdf80 2023-07-18T03:40:37.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f14a3fff700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14a4158bb0 0x7f14a40b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:37.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.164+0000 7f14a37fe700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a40cbe40 0x7f14a40c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:37.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.165+0000 7f149bfff700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14a40cdf80 0x7f14a414b3b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f148c0090b0 tx=0x7f148c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:37.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.165+0000 7f14a0ff9700 1 -- 172.21.15.49:0/129457382 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c003df0 con 0x7f14a40cdf80 2023-07-18T03:40:37.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.165+0000 7f14a0ff9700 1 -- 172.21.15.49:0/129457382 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c01f070 con 0x7f14a40cdf80 2023-07-18T03:40:37.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.165+0000 7f14a9bff700 1 -- 172.21.15.49:0/129457382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a414be00 con 0x7f14a40cdf80 2023-07-18T03:40:37.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.165+0000 7f14a0ff9700 1 -- 172.21.15.49:0/129457382 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c011870 con 0x7f14a40cdf80 2023-07-18T03:40:37.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.165+0000 7f14a9bff700 1 -- 172.21.15.49:0/129457382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a40b25f0 con 0x7f14a40cdf80 2023-07-18T03:40:37.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.166+0000 7f14a9bff700 1 -- 172.21.15.49:0/129457382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a4074350 con 0x7f14a40cdf80 2023-07-18T03:40:37.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.167+0000 7f14a0ff9700 1 -- 172.21.15.49:0/129457382 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f148c021030 con 0x7f14a40cdf80 2023-07-18T03:40:37.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.168+0000 7f14a0ff9700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f148406ad70 0x7f148406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:37.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.168+0000 7f14a0ff9700 1 -- 172.21.15.49:0/129457382 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f148c08a5f0 con 0x7f14a40cdf80 2023-07-18T03:40:37.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.168+0000 7f14a37fe700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f148406ad70 0x7f148406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:37.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.169+0000 7f14a37fe700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f148406ad70 0x7f148406d220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f14a40b2190 tx=0x7f1494009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:37.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.173+0000 7f14a0ff9700 1 -- 172.21.15.49:0/129457382 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f148c055990 con 0x7f14a40cdf80 2023-07-18T03:40:37.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.382+0000 7f14a9bff700 1 -- 172.21.15.49:0/129457382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f14a40b4a40 con 0x7f14a40cdf80 2023-07-18T03:40:37.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.383+0000 7f14a0ff9700 1 -- 172.21.15.49:0/129457382 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f148c058fb0 con 0x7f14a40cdf80 2023-07-18T03:40:37.377 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:40:37.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.385+0000 7f14a9bff700 1 -- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f148406ad70 msgr2=0x7f148406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:37.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.385+0000 7f14a9bff700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f148406ad70 0x7f148406d220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f14a40b2190 tx=0x7f1494009040 comp rx=0 tx=0).stop 2023-07-18T03:40:37.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.385+0000 7f14a9bff700 1 -- 172.21.15.49:0/129457382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14a40cdf80 msgr2=0x7f14a414b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:37.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.385+0000 7f14a9bff700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14a40cdf80 0x7f14a414b3b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f148c0090b0 tx=0x7f148c004750 comp rx=0 tx=0).stop 2023-07-18T03:40:37.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.385+0000 7f14a9bff700 1 -- 172.21.15.49:0/129457382 shutdown_connections 2023-07-18T03:40:37.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.385+0000 7f14a9bff700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14a4158bb0 0x7f14a40b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.385+0000 7f14a9bff700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a40cbe40 0x7f14a40c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.385+0000 7f14a9bff700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f148406ad70 0x7f148406d220 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.385+0000 7f14a9bff700 1 --2- 172.21.15.49:0/129457382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14a40cdf80 0x7f14a414b3b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.386+0000 7f14a9bff700 1 -- 172.21.15.49:0/129457382 >> 172.21.15.49:0/129457382 conn(0x7f14a40b05d0 msgr2=0x7f14a40b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:37.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.386+0000 7f14a9bff700 1 -- 172.21.15.49:0/129457382 shutdown_connections 2023-07-18T03:40:37.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.386+0000 7f14a9bff700 1 -- 172.21.15.49:0/129457382 wait complete. 2023-07-18T03:40:37.398 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:40:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.611+0000 7f3e023ff700 1 Processor -- start 2023-07-18T03:40:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.611+0000 7f3e023ff700 1 -- start start 2023-07-18T03:40:37.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.612+0000 7f3e023ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3dfc0b8660 0x7f3dfc0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:37.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.612+0000 7f3e023ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dfc0cd970 0x7f3dfc0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:37.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.612+0000 7f3e023ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dfc0d02a0 con 0x7f3dfc0b4da0 2023-07-18T03:40:37.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.612+0000 7f3e023ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dfc0b51c0 con 0x7f3dfc0b8660 2023-07-18T03:40:37.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.612+0000 7f3e023ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc0cfce0 con 0x7f3dfc0cd970 2023-07-18T03:40:37.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.613+0000 7f3dfbfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3dfc0b8660 0x7f3dfc0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.613+0000 7f3e0099c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3dfc0b4da0 0x7f3dfc1601f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35566/0 (socket says 172.21.15.49:35566) 2023-07-18T03:40:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.613+0000 7f3e0099c700 1 -- 172.21.15.49:0/502871743 learned_addr learned my addr 172.21.15.49:0/502871743 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.613+0000 7f3dfb7fe700 1 --2- 172.21.15.49:0/502871743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dfc0cd970 0x7f3dfc0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.613+0000 7f3dfaffd700 1 -- 172.21.15.49:0/502871743 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769702358 0 0) 0x7f3dfc0d02a0 con 0x7f3dfc0b4da0 2023-07-18T03:40:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.613+0000 7f3dfaffd700 1 -- 172.21.15.49:0/502871743 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3de40035b0 con 0x7f3dfc0b4da0 2023-07-18T03:40:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.613+0000 7f3dfbfff700 1 -- 172.21.15.49:0/502871743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dfc0cd970 msgr2=0x7f3dfc0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.613+0000 7f3dfbfff700 1 --2- 172.21.15.49:0/502871743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dfc0cd970 0x7f3dfc0b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.613+0000 7f3dfbfff700 1 -- 172.21.15.49:0/502871743 >> v1:172.21.15.49:6789/0 conn(0x7f3dfc0b4da0 legacy=0x7f3dfc1601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.613+0000 7f3dfbfff700 1 -- 172.21.15.49:0/502871743 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc0cffc0 con 0x7f3dfc0b8660 2023-07-18T03:40:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.614+0000 7f3dfbfff700 1 --2- 172.21.15.49:0/502871743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dfc0b8660 0x7f3dfc0cd430 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009840 tx=0x7f3dec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c18f4e26488160a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.614+0000 7f3dfaffd700 1 -- 172.21.15.49:0/502871743 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec011830 con 0x7f3dfc0b8660 2023-07-18T03:40:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.614+0000 7f3dfaffd700 1 -- 172.21.15.49:0/502871743 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dec011990 con 0x7f3dfc0b8660 2023-07-18T03:40:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.614+0000 7f3dfaffd700 1 -- 172.21.15.49:0/502871743 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec011d20 con 0x7f3dfc0b8660 2023-07-18T03:40:37.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.614+0000 7f3e023ff700 1 -- 172.21.15.49:0/502871743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dfc0b8660 msgr2=0x7f3dfc0cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.614+0000 7f3e023ff700 1 --2- 172.21.15.49:0/502871743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dfc0b8660 0x7f3dfc0cd430 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009840 tx=0x7f3dec009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.614+0000 7f3e023ff700 1 -- 172.21.15.49:0/502871743 shutdown_connections 2023-07-18T03:40:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.614+0000 7f3e023ff700 1 --2- 172.21.15.49:0/502871743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dfc0b8660 0x7f3dfc0cd430 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.614+0000 7f3e023ff700 1 --2- 172.21.15.49:0/502871743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dfc0cd970 0x7f3dfc0b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.614+0000 7f3e023ff700 1 -- 172.21.15.49:0/502871743 >> 172.21.15.49:0/502871743 conn(0x7f3dfc066860 msgr2=0x7f3dfc06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.615+0000 7f3e023ff700 1 -- 172.21.15.49:0/502871743 shutdown_connections 2023-07-18T03:40:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.615+0000 7f3e023ff700 1 -- 172.21.15.49:0/502871743 wait complete. 2023-07-18T03:40:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.617+0000 7f3e023ff700 1 Processor -- start 2023-07-18T03:40:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.617+0000 7f3e023ff700 1 -- start start 2023-07-18T03:40:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.618+0000 7f3e023ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dfc0b4da0 0x7f3dfc0b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.618+0000 7f3e023ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dfc0b8660 0x7f3dfc15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3e023ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dfc0cd970 0x7f3dfc14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3e023ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc1f28b0 con 0x7f3dfc0b8660 2023-07-18T03:40:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3e023ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc15d660 con 0x7f3dfc0b4da0 2023-07-18T03:40:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3e023ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc15d940 con 0x7f3dfc0cd970 2023-07-18T03:40:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3dfb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dfc0b8660 0x7f3dfc15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3dfb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dfc0b8660 0x7f3dfc15cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33098/0 (socket says 172.21.15.49:33098) 2023-07-18T03:40:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3dfb7fe700 1 -- 172.21.15.49:0/1319126913 learned_addr learned my addr 172.21.15.49:0/1319126913 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3dfbfff700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dfc0b4da0 0x7f3dfc0b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3e0099c700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dfc0cd970 0x7f3dfc14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3dfb7fe700 1 -- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dfc0cd970 msgr2=0x7f3dfc14dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3dfb7fe700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dfc0cd970 0x7f3dfc14dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3dfb7fe700 1 -- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dfc0b4da0 msgr2=0x7f3dfc0b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3dfb7fe700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dfc0b4da0 0x7f3dfc0b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3dfb7fe700 1 -- 172.21.15.49:0/1319126913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec0094a0 con 0x7f3dfc0b8660 2023-07-18T03:40:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.619+0000 7f3e0099c700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dfc0cd970 0x7f3dfc14dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:37.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.620+0000 7f3dfb7fe700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dfc0b8660 0x7f3dfc15cbc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3df000e980 tx=0x7f3df000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:37.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.620+0000 7f3df8ff9700 1 -- 172.21.15.49:0/1319126913 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df0009710 con 0x7f3dfc0b8660 2023-07-18T03:40:37.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.620+0000 7f3df8ff9700 1 -- 172.21.15.49:0/1319126913 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df000eed0 con 0x7f3dfc0b8660 2023-07-18T03:40:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.620+0000 7f3df8ff9700 1 -- 172.21.15.49:0/1319126913 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df000ca90 con 0x7f3dfc0b8660 2023-07-18T03:40:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.620+0000 7f3e023ff700 1 -- 172.21.15.49:0/1319126913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dfc15dc20 con 0x7f3dfc0b8660 2023-07-18T03:40:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.620+0000 7f3e023ff700 1 -- 172.21.15.49:0/1319126913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dfc0c9180 con 0x7f3dfc0b8660 2023-07-18T03:40:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.621+0000 7f3e023ff700 1 -- 172.21.15.49:0/1319126913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dfc0b5590 con 0x7f3dfc0b8660 2023-07-18T03:40:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.622+0000 7f3df8ff9700 1 -- 172.21.15.49:0/1319126913 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f3df000cbf0 con 0x7f3dfc0b8660 2023-07-18T03:40:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.623+0000 7f3df8ff9700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3de806ad70 0x7f3de806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.623+0000 7f3df8ff9700 1 -- 172.21.15.49:0/1319126913 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3df00891e0 con 0x7f3dfc0b8660 2023-07-18T03:40:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.624+0000 7f3dfbfff700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3de806ad70 0x7f3de806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.624+0000 7f3dfbfff700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3de806ad70 0x7f3de806d220 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3dec000c00 tx=0x7f3dec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.628+0000 7f3df8ff9700 1 -- 172.21.15.49:0/1319126913 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3dfc0b5590 con 0x7f3dfc0b8660 2023-07-18T03:40:37.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.833+0000 7f3e023ff700 1 -- 172.21.15.49:0/1319126913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3dfc0b5590 con 0x7f3dfc0b8660 2023-07-18T03:40:37.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.833+0000 7f3df8ff9700 1 -- 172.21.15.49:0/1319126913 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3dfc0b5590 con 0x7f3dfc0b8660 2023-07-18T03:40:37.828 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:40:37.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.836+0000 7f3e023ff700 1 -- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3de806ad70 msgr2=0x7f3de806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:37.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.836+0000 7f3e023ff700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3de806ad70 0x7f3de806d220 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3dec000c00 tx=0x7f3dec008030 comp rx=0 tx=0).stop 2023-07-18T03:40:37.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.836+0000 7f3e023ff700 1 -- 172.21.15.49:0/1319126913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dfc0b8660 msgr2=0x7f3dfc15cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:37.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.836+0000 7f3e023ff700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dfc0b8660 0x7f3dfc15cbc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3df000e980 tx=0x7f3df000ec90 comp rx=0 tx=0).stop 2023-07-18T03:40:37.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.836+0000 7f3e023ff700 1 -- 172.21.15.49:0/1319126913 shutdown_connections 2023-07-18T03:40:37.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.836+0000 7f3e023ff700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dfc0cd970 0x7f3dfc14dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.836+0000 7f3e023ff700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dfc0b4da0 0x7f3dfc0b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.836+0000 7f3e023ff700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3de806ad70 0x7f3de806d220 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.836+0000 7f3e023ff700 1 --2- 172.21.15.49:0/1319126913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3dfc0b8660 0x7f3dfc15cbc0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:37.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.836+0000 7f3e023ff700 1 -- 172.21.15.49:0/1319126913 >> 172.21.15.49:0/1319126913 conn(0x7f3dfc066860 msgr2=0x7f3dfc0cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:37.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.836+0000 7f3e023ff700 1 -- 172.21.15.49:0/1319126913 shutdown_connections 2023-07-18T03:40:37.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:37.836+0000 7f3e023ff700 1 -- 172.21.15.49:0/1319126913 wait complete. 2023-07-18T03:40:37.848 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-18T03:40:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.059+0000 7f08273ff700 1 Processor -- start 2023-07-18T03:40:38.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.060+0000 7f08273ff700 1 -- start start 2023-07-18T03:40:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.061+0000 7f08273ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f082015a1d0 0x7f08200ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.061+0000 7f08273ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08200cdbe0 con 0x7f082015a5a0 2023-07-18T03:40:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.061+0000 7f08273ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08200cc4c0 con 0x7f08200b0b90 2023-07-18T03:40:38.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.061+0000 7f08273ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08200cd420 con 0x7f082015a1d0 2023-07-18T03:40:38.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.061+0000 7f082499a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f082015a5a0 0x7f08200b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35580/0 (socket says 172.21.15.49:35580) 2023-07-18T03:40:38.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.061+0000 7f082499a700 1 -- 172.21.15.49:0/1146034193 learned_addr learned my addr 172.21.15.49:0/1146034193 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:38.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.061+0000 7f082519b700 1 --2- 172.21.15.49:0/1146034193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f082015a1d0 0x7f08200ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:38.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.062+0000 7f0817fff700 1 -- 172.21.15.49:0/1146034193 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049932282 0 0) 0x7f08200cdbe0 con 0x7f082015a5a0 2023-07-18T03:40:38.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.062+0000 7f0817fff700 1 -- 172.21.15.49:0/1146034193 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f080c0035b0 con 0x7f082015a5a0 2023-07-18T03:40:38.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.062+0000 7f0817fff700 1 -- 172.21.15.49:0/1146034193 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81204945 0 0) 0x7f08200cc4c0 con 0x7f08200b0b90 2023-07-18T03:40:38.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.062+0000 7f0817fff700 1 -- 172.21.15.49:0/1146034193 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08200cdbe0 con 0x7f08200b0b90 2023-07-18T03:40:38.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.062+0000 7f082519b700 1 -- 172.21.15.49:0/1146034193 >> v1:172.21.15.107:6789/0 conn(0x7f08200b0b90 legacy=0x7f082006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:38.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.062+0000 7f082519b700 1 -- 172.21.15.49:0/1146034193 >> v1:172.21.15.49:6789/0 conn(0x7f082015a5a0 legacy=0x7f08200b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:38.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.062+0000 7f082519b700 1 -- 172.21.15.49:0/1146034193 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08200cc7a0 con 0x7f082015a1d0 2023-07-18T03:40:38.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.062+0000 7f082519b700 1 --2- 172.21.15.49:0/1146034193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f082015a1d0 0x7f08200ad9d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f081c009870 tx=0x7f081c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a17a0afc20ff64c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:38.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.062+0000 7f0817fff700 1 -- 172.21.15.49:0/1146034193 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c011830 con 0x7f082015a1d0 2023-07-18T03:40:38.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.062+0000 7f0817fff700 1 -- 172.21.15.49:0/1146034193 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081c011990 con 0x7f082015a1d0 2023-07-18T03:40:38.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.062+0000 7f0817fff700 1 -- 172.21.15.49:0/1146034193 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c011d20 con 0x7f082015a1d0 2023-07-18T03:40:38.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.063+0000 7f08273ff700 1 -- 172.21.15.49:0/1146034193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f082015a1d0 msgr2=0x7f08200ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:38.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.063+0000 7f08273ff700 1 --2- 172.21.15.49:0/1146034193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f082015a1d0 0x7f08200ad9d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f081c009870 tx=0x7f081c009b80 comp rx=0 tx=0).stop 2023-07-18T03:40:38.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.063+0000 7f08273ff700 1 -- 172.21.15.49:0/1146034193 shutdown_connections 2023-07-18T03:40:38.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.063+0000 7f08273ff700 1 --2- 172.21.15.49:0/1146034193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f082015a1d0 0x7f08200ad9d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.063+0000 7f08273ff700 1 -- 172.21.15.49:0/1146034193 >> 172.21.15.49:0/1146034193 conn(0x7f08200ce980 msgr2=0x7f08200ced50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:38.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.063+0000 7f08273ff700 1 -- 172.21.15.49:0/1146034193 shutdown_connections 2023-07-18T03:40:38.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.063+0000 7f08273ff700 1 -- 172.21.15.49:0/1146034193 wait complete. 2023-07-18T03:40:38.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.065+0000 7f08273ff700 1 Processor -- start 2023-07-18T03:40:38.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.065+0000 7f08273ff700 1 -- start start 2023-07-18T03:40:38.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.066+0000 7f08273ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08200b0b90 0x7f082014ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:38.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.066+0000 7f08273ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f082015a1d0 0x7f08200b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:38.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.066+0000 7f08273ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f082015a5a0 0x7f0820154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:38.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.066+0000 7f08273ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082015d930 con 0x7f08200b0b90 2023-07-18T03:40:38.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.066+0000 7f08273ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082015cdb0 con 0x7f082015a1d0 2023-07-18T03:40:38.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.066+0000 7f08273ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082015d090 con 0x7f082015a5a0 2023-07-18T03:40:38.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.066+0000 7f082519b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08200b0b90 0x7f082014ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:38.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.066+0000 7f082519b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08200b0b90 0x7f082014ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33104/0 (socket says 172.21.15.49:33104) 2023-07-18T03:40:38.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.066+0000 7f082519b700 1 -- 172.21.15.49:0/771059674 learned_addr learned my addr 172.21.15.49:0/771059674 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:38.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f082499a700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f082015a1d0 0x7f08200b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:38.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f082599c700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f082015a5a0 0x7f0820154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:38.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f082519b700 1 -- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f082015a5a0 msgr2=0x7f0820154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:38.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f082519b700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f082015a5a0 0x7f0820154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f082519b700 1 -- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f082015a1d0 msgr2=0x7f08200b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:38.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f082519b700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f082015a1d0 0x7f08200b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f082519b700 1 -- 172.21.15.49:0/771059674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c0094a0 con 0x7f08200b0b90 2023-07-18T03:40:38.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f082499a700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f082015a1d0 0x7f08200b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:38.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f082519b700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08200b0b90 0x7f082014ade0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f081c0090b0 tx=0x7f081c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:38.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f0815ffb700 1 -- 172.21.15.49:0/771059674 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c01a860 con 0x7f08200b0b90 2023-07-18T03:40:38.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f0815ffb700 1 -- 172.21.15.49:0/771059674 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081c01a9c0 con 0x7f08200b0b90 2023-07-18T03:40:38.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f0815ffb700 1 -- 172.21.15.49:0/771059674 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c01ad30 con 0x7f08200b0b90 2023-07-18T03:40:38.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.067+0000 7f08273ff700 1 -- 172.21.15.49:0/771059674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082015d370 con 0x7f08200b0b90 2023-07-18T03:40:38.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.068+0000 7f08273ff700 1 -- 172.21.15.49:0/771059674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082014b320 con 0x7f08200b0b90 2023-07-18T03:40:38.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.069+0000 7f08273ff700 1 -- 172.21.15.49:0/771059674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082004e210 con 0x7f08200b0b90 2023-07-18T03:40:38.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.069+0000 7f0815ffb700 1 -- 172.21.15.49:0/771059674 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f081c022070 con 0x7f08200b0b90 2023-07-18T03:40:38.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.071+0000 7f0815ffb700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f080406ad70 0x7f080406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:38.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.071+0000 7f0815ffb700 1 -- 172.21.15.49:0/771059674 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f081c089ee0 con 0x7f08200b0b90 2023-07-18T03:40:38.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.072+0000 7f082499a700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f080406ad70 0x7f080406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:38.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.072+0000 7f082499a700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f080406ad70 0x7f080406d220 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f082015cce0 tx=0x7f0810009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:38.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.075+0000 7f0815ffb700 1 -- 172.21.15.49:0/771059674 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f081c055450 con 0x7f08200b0b90 2023-07-18T03:40:38.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.287+0000 7f08273ff700 1 -- 172.21.15.49:0/771059674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f082006d630 con 0x7f08200b0b90 2023-07-18T03:40:38.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.287+0000 7f0815ffb700 1 -- 172.21.15.49:0/771059674 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) v1 ==== 84+0+126 (secure 0 0 0) 0x7f081c029020 con 0x7f08200b0b90 2023-07-18T03:40:38.282 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:38.282 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2023-07-18T03:40:38.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:38.282 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2023-07-18T03:40:38.282 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": "rbd" 2023-07-18T03:40:38.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-07-18T03:40:38.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:38.283 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 2, 2023-07-18T03:40:38.283 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2023-07-18T03:40:38.283 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:38.283 INFO:teuthology.orchestra.run.smithi049.stdout:] 2023-07-18T03:40:38.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.290+0000 7f08273ff700 1 -- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f080406ad70 msgr2=0x7f080406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:38.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.290+0000 7f08273ff700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f080406ad70 0x7f080406d220 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f082015cce0 tx=0x7f0810009040 comp rx=0 tx=0).stop 2023-07-18T03:40:38.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.290+0000 7f08273ff700 1 -- 172.21.15.49:0/771059674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08200b0b90 msgr2=0x7f082014ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:38.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.290+0000 7f08273ff700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08200b0b90 0x7f082014ade0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f081c0090b0 tx=0x7f081c004750 comp rx=0 tx=0).stop 2023-07-18T03:40:38.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.290+0000 7f08273ff700 1 -- 172.21.15.49:0/771059674 shutdown_connections 2023-07-18T03:40:38.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.290+0000 7f08273ff700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f082015a5a0 0x7f0820154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.290+0000 7f08273ff700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f080406ad70 0x7f080406d220 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.290+0000 7f08273ff700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08200b0b90 0x7f082014ade0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.290+0000 7f08273ff700 1 --2- 172.21.15.49:0/771059674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f082015a1d0 0x7f08200b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.290+0000 7f08273ff700 1 -- 172.21.15.49:0/771059674 >> 172.21.15.49:0/771059674 conn(0x7f08200ce980 msgr2=0x7f08200afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:38.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.290+0000 7f08273ff700 1 -- 172.21.15.49:0/771059674 shutdown_connections 2023-07-18T03:40:38.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.290+0000 7f08273ff700 1 -- 172.21.15.49:0/771059674 wait complete. 2023-07-18T03:40:38.302 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-18T03:40:38.303 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-18T03:40:38.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.517+0000 7f0275dff700 1 Processor -- start 2023-07-18T03:40:38.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.517+0000 7f0275dff700 1 -- start start 2023-07-18T03:40:38.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f0275dff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f027015c7b0 0x7f02700d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:38.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f0275dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b1e50 0x7f02700d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:38.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f0275dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f02700d0f70 0x7f0270160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:38.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f0275dff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02700d3de0 con 0x7f027015c7b0 2023-07-18T03:40:38.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f0275dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02700d13f0 con 0x7f02700d0f70 2023-07-18T03:40:38.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f0275dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02700d0190 con 0x7f02700b1e50 2023-07-18T03:40:38.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f026f7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f027015c7b0 0x7f02700d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:38.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f026f7fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f027015c7b0 0x7f02700d28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33120/0 (socket says 172.21.15.49:33120) 2023-07-18T03:40:38.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f026f7fe700 1 -- 172.21.15.49:0/2574572198 learned_addr learned my addr 172.21.15.49:0/2574572198 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:38.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f026effd700 1 --2- 172.21.15.49:0/2574572198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b1e50 0x7f02700d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:38.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f026ffff700 1 --2- 172.21.15.49:0/2574572198 >> v2:172.21.15.107:3300/0 conn(0x7f02700d0f70 0x7f0270160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:38.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f026f7fe700 1 -- 172.21.15.49:0/2574572198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b1e50 msgr2=0x7f02700d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:38.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f026f7fe700 1 --2- 172.21.15.49:0/2574572198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b1e50 0x7f02700d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f026f7fe700 1 -- 172.21.15.49:0/2574572198 >> v2:172.21.15.107:3300/0 conn(0x7f02700d0f70 msgr2=0x7f0270160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:38.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f026f7fe700 1 --2- 172.21.15.49:0/2574572198 >> v2:172.21.15.107:3300/0 conn(0x7f02700d0f70 0x7f0270160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f026f7fe700 1 -- 172.21.15.49:0/2574572198 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02700d0470 con 0x7f027015c7b0 2023-07-18T03:40:38.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f026effd700 1 --2- 172.21.15.49:0/2574572198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b1e50 0x7f02700d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:38.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.519+0000 7f026ffff700 1 --2- 172.21.15.49:0/2574572198 >> v2:172.21.15.107:3300/0 conn(0x7f02700d0f70 0x7f0270160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:38.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.520+0000 7f026f7fe700 1 --2- 172.21.15.49:0/2574572198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f027015c7b0 0x7f02700d28a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f027008d670 tx=0x7f0260009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915d5c07cb1712f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:38.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.520+0000 7f026e7fc700 1 -- 172.21.15.49:0/2574572198 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026000b870 con 0x7f027015c7b0 2023-07-18T03:40:38.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.520+0000 7f026e7fc700 1 -- 172.21.15.49:0/2574572198 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026000b9d0 con 0x7f027015c7b0 2023-07-18T03:40:38.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.520+0000 7f026e7fc700 1 -- 172.21.15.49:0/2574572198 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026000bd60 con 0x7f027015c7b0 2023-07-18T03:40:38.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.520+0000 7f0275dff700 1 -- 172.21.15.49:0/2574572198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f027015c7b0 msgr2=0x7f02700d28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:38.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.520+0000 7f0275dff700 1 --2- 172.21.15.49:0/2574572198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f027015c7b0 0x7f02700d28a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f027008d670 tx=0x7f0260009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.521+0000 7f0275dff700 1 -- 172.21.15.49:0/2574572198 shutdown_connections 2023-07-18T03:40:38.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.521+0000 7f0275dff700 1 --2- 172.21.15.49:0/2574572198 >> v2:172.21.15.107:3300/0 conn(0x7f02700d0f70 0x7f0270160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.521+0000 7f0275dff700 1 --2- 172.21.15.49:0/2574572198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f027015c7b0 0x7f02700d28a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.521+0000 7f0275dff700 1 --2- 172.21.15.49:0/2574572198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b1e50 0x7f02700d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.521+0000 7f0275dff700 1 -- 172.21.15.49:0/2574572198 >> 172.21.15.49:0/2574572198 conn(0x7f027009c1b0 msgr2=0x7f02700d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:38.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.521+0000 7f0275dff700 1 -- 172.21.15.49:0/2574572198 shutdown_connections 2023-07-18T03:40:38.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.521+0000 7f0275dff700 1 -- 172.21.15.49:0/2574572198 wait complete. 2023-07-18T03:40:38.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.523+0000 7f0275dff700 1 Processor -- start 2023-07-18T03:40:38.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.523+0000 7f0275dff700 1 -- start start 2023-07-18T03:40:38.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f0275dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b1e50 0x7f02700b8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:38.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f0275dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02700d0f70 0x7f027015e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:38.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f0275dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027015c7b0 0x7f027014e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:38.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f0275dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027015dc00 con 0x7f02700d0f70 2023-07-18T03:40:38.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f0275dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027015edd0 con 0x7f027015c7b0 2023-07-18T03:40:38.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f0275dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027015bbb0 con 0x7f02700b1e50 2023-07-18T03:40:38.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f026effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02700d0f70 0x7f027015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:38.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f026effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02700d0f70 0x7f027015e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33136/0 (socket says 172.21.15.49:33136) 2023-07-18T03:40:38.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f026effd700 1 -- 172.21.15.49:0/1540316326 learned_addr learned my addr 172.21.15.49:0/1540316326 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f026f7fe700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b1e50 0x7f02700b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f026ffff700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027015c7b0 0x7f027014e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f026effd700 1 -- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b1e50 msgr2=0x7f02700b8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f026effd700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b1e50 0x7f02700b8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f026effd700 1 -- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027015c7b0 msgr2=0x7f027014e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:38.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f026effd700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027015c7b0 0x7f027014e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.524+0000 7f026effd700 1 -- 172.21.15.49:0/1540316326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0260009550 con 0x7f02700d0f70 2023-07-18T03:40:38.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.525+0000 7f026effd700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02700d0f70 0x7f027015e020 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f026400e810 tx=0x7f026400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:38.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.525+0000 7f0257fff700 1 -- 172.21.15.49:0/1540316326 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026400c020 con 0x7f02700d0f70 2023-07-18T03:40:38.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.525+0000 7f0257fff700 1 -- 172.21.15.49:0/1540316326 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0264004250 con 0x7f02700d0f70 2023-07-18T03:40:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.525+0000 7f0275dff700 1 -- 172.21.15.49:0/1540316326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027015f070 con 0x7f02700d0f70 2023-07-18T03:40:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.525+0000 7f0257fff700 1 -- 172.21.15.49:0/1540316326 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02640036a0 con 0x7f02700d0f70 2023-07-18T03:40:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.525+0000 7f0275dff700 1 -- 172.21.15.49:0/1540316326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027014ea10 con 0x7f02700d0f70 2023-07-18T03:40:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.526+0000 7f0275dff700 1 -- 172.21.15.49:0/1540316326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f027015dd40 con 0x7f02700d0f70 2023-07-18T03:40:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.527+0000 7f0257fff700 1 -- 172.21.15.49:0/1540316326 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f0264003800 con 0x7f02700d0f70 2023-07-18T03:40:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.528+0000 7f0257fff700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f025c06ad70 0x7f025c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.528+0000 7f0257fff700 1 -- 172.21.15.49:0/1540316326 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f02640890d0 con 0x7f02700d0f70 2023-07-18T03:40:38.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.529+0000 7f026f7fe700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f025c06ad70 0x7f025c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:38.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.529+0000 7f026f7fe700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f025c06ad70 0x7f025c06d220 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0260000c00 tx=0x7f0260008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:38.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.532+0000 7f0257fff700 1 -- 172.21.15.49:0/1540316326 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f027015dd40 con 0x7f02700d0f70 2023-07-18T03:40:38.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.740+0000 7f0275dff700 1 -- 172.21.15.49:0/1540316326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f027015dd40 con 0x7f02700d0f70 2023-07-18T03:40:38.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.925+0000 7f0257fff700 1 -- 172.21.15.49:0/1540316326 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v46) v1 ==== 138+0+0 (secure 0 0 0) 0x7f027015dd40 con 0x7f02700d0f70 2023-07-18T03:40:38.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.930+0000 7f0275dff700 1 -- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f025c06ad70 msgr2=0x7f025c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:38.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.930+0000 7f0275dff700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f025c06ad70 0x7f025c06d220 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0260000c00 tx=0x7f0260008030 comp rx=0 tx=0).stop 2023-07-18T03:40:38.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.930+0000 7f0275dff700 1 -- 172.21.15.49:0/1540316326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02700d0f70 msgr2=0x7f027015e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:38.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.930+0000 7f0275dff700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02700d0f70 0x7f027015e020 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f026400e810 tx=0x7f026400ebd0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.931+0000 7f0275dff700 1 -- 172.21.15.49:0/1540316326 shutdown_connections 2023-07-18T03:40:38.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.931+0000 7f0275dff700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027015c7b0 0x7f027014e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.931+0000 7f0275dff700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b1e50 0x7f02700b8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.931+0000 7f0275dff700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f025c06ad70 0x7f025c06d220 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.931+0000 7f0275dff700 1 --2- 172.21.15.49:0/1540316326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02700d0f70 0x7f027015e020 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:38.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.931+0000 7f0275dff700 1 -- 172.21.15.49:0/1540316326 >> 172.21.15.49:0/1540316326 conn(0x7f027009c1b0 msgr2=0x7f02700afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:38.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.931+0000 7f0275dff700 1 -- 172.21.15.49:0/1540316326 shutdown_connections 2023-07-18T03:40:38.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:38.931+0000 7f0275dff700 1 -- 172.21.15.49:0/1540316326 wait complete. 2023-07-18T03:40:38.928 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2023-07-18T03:40:38.946 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-18T03:40:39.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.163+0000 7f1323dff700 1 Processor -- start 2023-07-18T03:40:39.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.163+0000 7f1323dff700 1 -- start start 2023-07-18T03:40:39.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.164+0000 7f1323dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f131c0b8a60 0x7f131c0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:39.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.164+0000 7f1323dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f131c0b4530 0x7f131c0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:39.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.164+0000 7f1323dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131c0b8e30 con 0x7f131c0d1050 2023-07-18T03:40:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.164+0000 7f1323dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131c0d14d0 con 0x7f131c0b4530 2023-07-18T03:40:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.164+0000 7f1323dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0b1e30 con 0x7f131c0b8a60 2023-07-18T03:40:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.165+0000 7f132239c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f131c0d1050 0x7f131c0b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50422/0 (socket says 172.21.15.49:50422) 2023-07-18T03:40:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.165+0000 7f132239c700 1 -- 172.21.15.49:0/1843156495 learned_addr learned my addr 172.21.15.49:0/1843156495 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.165+0000 7f1321b9b700 1 --2- 172.21.15.49:0/1843156495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f131c0b8a60 0x7f131c0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:39.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.165+0000 7f132139a700 1 --2- 172.21.15.49:0/1843156495 >> v2:172.21.15.107:3300/0 conn(0x7f131c0b4530 0x7f131c0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.165+0000 7f1320b99700 1 -- 172.21.15.49:0/1843156495 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820432782 0 0) 0x7f131c0b8e30 con 0x7f131c0d1050 2023-07-18T03:40:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.165+0000 7f1320b99700 1 -- 172.21.15.49:0/1843156495 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13080035b0 con 0x7f131c0d1050 2023-07-18T03:40:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.165+0000 7f1321b9b700 1 -- 172.21.15.49:0/1843156495 >> v2:172.21.15.107:3300/0 conn(0x7f131c0b4530 msgr2=0x7f131c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.165+0000 7f1321b9b700 1 --2- 172.21.15.49:0/1843156495 >> v2:172.21.15.107:3300/0 conn(0x7f131c0b4530 0x7f131c0d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:39.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.165+0000 7f1321b9b700 1 -- 172.21.15.49:0/1843156495 >> v1:172.21.15.49:6789/0 conn(0x7f131c0d1050 legacy=0x7f131c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:39.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.165+0000 7f1321b9b700 1 -- 172.21.15.49:0/1843156495 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c0afaa0 con 0x7f131c0b8a60 2023-07-18T03:40:39.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.166+0000 7f1321b9b700 1 --2- 172.21.15.49:0/1843156495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f131c0b8a60 0x7f131c0b3ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1318009840 tx=0x7f1318009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad2f074da05783bc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:39.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.166+0000 7f1320b99700 1 -- 172.21.15.49:0/1843156495 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318011830 con 0x7f131c0b8a60 2023-07-18T03:40:39.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.166+0000 7f1320b99700 1 -- 172.21.15.49:0/1843156495 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1318011990 con 0x7f131c0b8a60 2023-07-18T03:40:39.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.166+0000 7f1320b99700 1 -- 172.21.15.49:0/1843156495 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318011d20 con 0x7f131c0b8a60 2023-07-18T03:40:39.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.166+0000 7f1323dff700 1 -- 172.21.15.49:0/1843156495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f131c0b8a60 msgr2=0x7f131c0b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:39.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.166+0000 7f1323dff700 1 --2- 172.21.15.49:0/1843156495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f131c0b8a60 0x7f131c0b3ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1318009840 tx=0x7f1318009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:39.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.166+0000 7f1323dff700 1 -- 172.21.15.49:0/1843156495 shutdown_connections 2023-07-18T03:40:39.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.166+0000 7f1323dff700 1 --2- 172.21.15.49:0/1843156495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f131c0b8a60 0x7f131c0b3ff0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:39.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.166+0000 7f1323dff700 1 --2- 172.21.15.49:0/1843156495 >> v2:172.21.15.107:3300/0 conn(0x7f131c0b4530 0x7f131c0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:39.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.166+0000 7f1323dff700 1 -- 172.21.15.49:0/1843156495 >> 172.21.15.49:0/1843156495 conn(0x7f131c09c1b0 msgr2=0x7f131c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:39.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.167+0000 7f1323dff700 1 -- 172.21.15.49:0/1843156495 shutdown_connections 2023-07-18T03:40:39.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.167+0000 7f1323dff700 1 -- 172.21.15.49:0/1843156495 wait complete. 2023-07-18T03:40:39.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.168+0000 7f1323dff700 1 Processor -- start 2023-07-18T03:40:39.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.168+0000 7f1323dff700 1 -- start start 2023-07-18T03:40:39.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.169+0000 7f1323dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f131c0b4530 0x7f131c1f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:39.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.169+0000 7f1323dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f131c0b8a60 0x7f131c1f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:39.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.169+0000 7f1323dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f131c0d1050 0x7f131c0cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:39.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.169+0000 7f1323dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c15d140 con 0x7f131c0b4530 2023-07-18T03:40:39.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.169+0000 7f1323dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c1f28a0 con 0x7f131c0b8a60 2023-07-18T03:40:39.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.169+0000 7f1323dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c1f2b80 con 0x7f131c0d1050 2023-07-18T03:40:39.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f1321b9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f131c0b4530 0x7f131c1f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:39.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f1321b9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f131c0b4530 0x7f131c1f63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35446/0 (socket says 172.21.15.49:35446) 2023-07-18T03:40:39.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f1321b9b700 1 -- 172.21.15.49:0/1356725015 learned_addr learned my addr 172.21.15.49:0/1356725015 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:39.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f132239c700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f131c0d1050 0x7f131c0cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:39.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f132139a700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f131c0b8a60 0x7f131c1f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f1321b9b700 1 -- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f131c0d1050 msgr2=0x7f131c0cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f1321b9b700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f131c0d1050 0x7f131c0cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f1321b9b700 1 -- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f131c0b8a60 msgr2=0x7f131c1f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f1321b9b700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f131c0b8a60 0x7f131c1f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f1321b9b700 1 -- 172.21.15.49:0/1356725015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13180094a0 con 0x7f131c0b4530 2023-07-18T03:40:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f132239c700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f131c0d1050 0x7f131c0cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:39.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f132139a700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f131c0b8a60 0x7f131c1f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.170+0000 7f1321b9b700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f131c0b4530 0x7f131c1f63e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f13180090b0 tx=0x7f1318004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.171+0000 7f13127fc700 1 -- 172.21.15.49:0/1356725015 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13180036a0 con 0x7f131c0b4530 2023-07-18T03:40:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.171+0000 7f13127fc700 1 -- 172.21.15.49:0/1356725015 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131801f070 con 0x7f131c0b4530 2023-07-18T03:40:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.171+0000 7f13127fc700 1 -- 172.21.15.49:0/1356725015 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318003a00 con 0x7f131c0b4530 2023-07-18T03:40:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.171+0000 7f1323dff700 1 -- 172.21.15.49:0/1356725015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c1f2e60 con 0x7f131c0b4530 2023-07-18T03:40:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.171+0000 7f1323dff700 1 -- 172.21.15.49:0/1356725015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131c1f3140 con 0x7f131c0b4530 2023-07-18T03:40:39.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.172+0000 7f1323dff700 1 -- 172.21.15.49:0/1356725015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131c074350 con 0x7f131c0b4530 2023-07-18T03:40:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.173+0000 7f13127fc700 1 -- 172.21.15.49:0/1356725015 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f1318003b60 con 0x7f131c0b4530 2023-07-18T03:40:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.174+0000 7f13127fc700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f130006ad70 0x7f130006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.174+0000 7f13127fc700 1 -- 172.21.15.49:0/1356725015 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f1318021070 con 0x7f131c0b4530 2023-07-18T03:40:39.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.175+0000 7f132139a700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f130006ad70 0x7f130006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:39.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.175+0000 7f132139a700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f130006ad70 0x7f130006d220 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f131c1efe40 tx=0x7f130c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:39.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.178+0000 7f13127fc700 1 -- 172.21.15.49:0/1356725015 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1318055f20 con 0x7f131c0b4530 2023-07-18T03:40:39.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.386+0000 7f1323dff700 1 -- 172.21.15.49:0/1356725015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f131c0b2000 con 0x7f131c0b4530 2023-07-18T03:40:39.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.928+0000 7f13127fc700 1 -- 172.21.15.49:0/1356725015 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v47) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1318059540 con 0x7f131c0b4530 2023-07-18T03:40:39.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.937+0000 7f1323dff700 1 -- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f130006ad70 msgr2=0x7f130006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:39.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.937+0000 7f1323dff700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f130006ad70 0x7f130006d220 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f131c1efe40 tx=0x7f130c008040 comp rx=0 tx=0).stop 2023-07-18T03:40:39.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.937+0000 7f1323dff700 1 -- 172.21.15.49:0/1356725015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f131c0b4530 msgr2=0x7f131c1f63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:39.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.937+0000 7f1323dff700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f131c0b4530 0x7f131c1f63e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f13180090b0 tx=0x7f1318004750 comp rx=0 tx=0).stop 2023-07-18T03:40:39.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.940+0000 7f1323dff700 1 -- 172.21.15.49:0/1356725015 shutdown_connections 2023-07-18T03:40:39.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.940+0000 7f1323dff700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f131c0d1050 0x7f131c0cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:39.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.940+0000 7f1323dff700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f130006ad70 0x7f130006d220 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:39.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.940+0000 7f1323dff700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f131c0b4530 0x7f131c1f63e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:39.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.940+0000 7f1323dff700 1 --2- 172.21.15.49:0/1356725015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f131c0b8a60 0x7f131c1f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:39.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.940+0000 7f1323dff700 1 -- 172.21.15.49:0/1356725015 >> 172.21.15.49:0/1356725015 conn(0x7f131c09c1b0 msgr2=0x7f131c0b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:39.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.941+0000 7f1323dff700 1 -- 172.21.15.49:0/1356725015 shutdown_connections 2023-07-18T03:40:39.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:39.941+0000 7f1323dff700 1 -- 172.21.15.49:0/1356725015 wait complete. 2023-07-18T03:40:39.945 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2023-07-18T03:40:39.963 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-18T03:40:40.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.178+0000 7ff73c3ff700 1 Processor -- start 2023-07-18T03:40:40.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.178+0000 7ff73c3ff700 1 -- start start 2023-07-18T03:40:40.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.179+0000 7ff73c3ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff7340d0b10 0x7ff7340d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:40.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.179+0000 7ff73c3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff7340b0e20 0x7ff7340b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:40.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.180+0000 7ff73c3ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7340b8e30 con 0x7ff7340b8a60 2023-07-18T03:40:40.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.180+0000 7ff73c3ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7340d14d0 con 0x7ff7340d0b10 2023-07-18T03:40:40.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.180+0000 7ff73c3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7340afaa0 con 0x7ff7340b0e20 2023-07-18T03:40:40.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.180+0000 7ff73999a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff7340d0b10 0x7ff7340d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:40.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.180+0000 7ff73a19b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff7340b8a60 0x7ff7340b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50426/0 (socket says 172.21.15.49:50426) 2023-07-18T03:40:40.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.180+0000 7ff73a19b700 1 -- 172.21.15.49:0/2983215232 learned_addr learned my addr 172.21.15.49:0/2983215232 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:40.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.180+0000 7ff73a99c700 1 --2- 172.21.15.49:0/2983215232 >> v2:172.21.15.107:3300/0 conn(0x7ff7340b0e20 0x7ff7340b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:40.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.180+0000 7ff73999a700 1 -- 172.21.15.49:0/2983215232 >> v2:172.21.15.107:3300/0 conn(0x7ff7340b0e20 msgr2=0x7ff7340b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:40.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.180+0000 7ff73999a700 1 --2- 172.21.15.49:0/2983215232 >> v2:172.21.15.107:3300/0 conn(0x7ff7340b0e20 0x7ff7340b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:40.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.180+0000 7ff73999a700 1 -- 172.21.15.49:0/2983215232 >> v1:172.21.15.49:6789/0 conn(0x7ff7340b8a60 legacy=0x7ff7340b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:40.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.180+0000 7ff73999a700 1 -- 172.21.15.49:0/2983215232 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7340afd80 con 0x7ff7340d0b10 2023-07-18T03:40:40.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.180+0000 7ff739199700 1 -- 172.21.15.49:0/2983215232 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461345949 0 0) 0x7ff7340b8e30 con 0x7ff7340b8a60 2023-07-18T03:40:40.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.181+0000 7ff73999a700 1 --2- 172.21.15.49:0/2983215232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7340d0b10 0x7ff7340d0f90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff7340674e0 tx=0x7ff724009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41aa01b17a039bc3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:40.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.181+0000 7ff739199700 1 -- 172.21.15.49:0/2983215232 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724011830 con 0x7ff7340d0b10 2023-07-18T03:40:40.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.181+0000 7ff739199700 1 -- 172.21.15.49:0/2983215232 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff724011990 con 0x7ff7340d0b10 2023-07-18T03:40:40.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.181+0000 7ff739199700 1 -- 172.21.15.49:0/2983215232 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724011d20 con 0x7ff7340d0b10 2023-07-18T03:40:40.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.181+0000 7ff73c3ff700 1 -- 172.21.15.49:0/2983215232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7340d0b10 msgr2=0x7ff7340d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:40.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.181+0000 7ff73c3ff700 1 --2- 172.21.15.49:0/2983215232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7340d0b10 0x7ff7340d0f90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff7340674e0 tx=0x7ff724009b20 comp rx=0 tx=0).stop 2023-07-18T03:40:40.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.182+0000 7ff73c3ff700 1 -- 172.21.15.49:0/2983215232 shutdown_connections 2023-07-18T03:40:40.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.182+0000 7ff73c3ff700 1 --2- 172.21.15.49:0/2983215232 >> v2:172.21.15.107:3300/0 conn(0x7ff7340b0e20 0x7ff7340b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:40.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.182+0000 7ff73c3ff700 1 --2- 172.21.15.49:0/2983215232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7340d0b10 0x7ff7340d0f90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:40.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.182+0000 7ff73c3ff700 1 -- 172.21.15.49:0/2983215232 >> 172.21.15.49:0/2983215232 conn(0x7ff73409c1b0 msgr2=0x7ff7340d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:40.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.182+0000 7ff73c3ff700 1 -- 172.21.15.49:0/2983215232 shutdown_connections 2023-07-18T03:40:40.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.182+0000 7ff73c3ff700 1 -- 172.21.15.49:0/2983215232 wait complete. 2023-07-18T03:40:40.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.183+0000 7ff73c3ff700 1 Processor -- start 2023-07-18T03:40:40.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.184+0000 7ff73c3ff700 1 -- start start 2023-07-18T03:40:40.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.184+0000 7ff73c3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7340b0e20 0x7ff7341526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:40.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.184+0000 7ff73c3ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7340b8a60 0x7ff7340cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:40.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73c3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7340d0b10 0x7ff7340cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:40.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73c3ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340d0650 con 0x7ff7340b8a60 2023-07-18T03:40:40.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73c3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340d0090 con 0x7ff7340b0e20 2023-07-18T03:40:40.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73c3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340d0370 con 0x7ff7340d0b10 2023-07-18T03:40:40.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73999a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7340b8a60 0x7ff7340cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:40.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73999a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7340b8a60 0x7ff7340cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35462/0 (socket says 172.21.15.49:35462) 2023-07-18T03:40:40.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73999a700 1 -- 172.21.15.49:0/993867551 learned_addr learned my addr 172.21.15.49:0/993867551 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:40.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73a19b700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7340b0e20 0x7ff7341526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:40.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73a99c700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7340d0b10 0x7ff7340cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:40.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73999a700 1 -- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7340d0b10 msgr2=0x7ff7340cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:40.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73999a700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7340d0b10 0x7ff7340cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:40.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73999a700 1 -- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7340b0e20 msgr2=0x7ff7341526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:40.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73999a700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7340b0e20 0x7ff7341526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:40.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73999a700 1 -- 172.21.15.49:0/993867551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240094a0 con 0x7ff7340b8a60 2023-07-18T03:40:40.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.185+0000 7ff73999a700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7340b8a60 0x7ff7340cf6f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff724000c00 tx=0x7ff724004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:40.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.186+0000 7ff72affd700 1 -- 172.21.15.49:0/993867551 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7240036a0 con 0x7ff7340b8a60 2023-07-18T03:40:40.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.186+0000 7ff72affd700 1 -- 172.21.15.49:0/993867551 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72401e070 con 0x7ff7340b8a60 2023-07-18T03:40:40.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.186+0000 7ff73c3ff700 1 -- 172.21.15.49:0/993867551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73415d1d0 con 0x7ff7340b8a60 2023-07-18T03:40:40.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.186+0000 7ff72affd700 1 -- 172.21.15.49:0/993867551 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724003a60 con 0x7ff7340b8a60 2023-07-18T03:40:40.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.186+0000 7ff73c3ff700 1 -- 172.21.15.49:0/993867551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7340ca010 con 0x7ff7340b8a60 2023-07-18T03:40:40.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.187+0000 7ff73c3ff700 1 -- 172.21.15.49:0/993867551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73415d030 con 0x7ff7340b8a60 2023-07-18T03:40:40.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.188+0000 7ff72affd700 1 -- 172.21.15.49:0/993867551 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7ff724003bc0 con 0x7ff7340b8a60 2023-07-18T03:40:40.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.189+0000 7ff72affd700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff71806ad70 0x7ff71806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:40.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.189+0000 7ff72affd700 1 -- 172.21.15.49:0/993867551 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5962+0+0 (secure 0 0 0) 0x7ff72408a5a0 con 0x7ff7340b8a60 2023-07-18T03:40:40.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.190+0000 7ff73a19b700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff71806ad70 0x7ff71806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:40.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.190+0000 7ff73a19b700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff71806ad70 0x7ff71806d220 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff730000f60 tx=0x7ff730007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:40.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.194+0000 7ff72affd700 1 -- 172.21.15.49:0/993867551 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff73415d030 con 0x7ff7340b8a60 2023-07-18T03:40:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.450+0000 7ff73c3ff700 1 -- 172.21.15.49:0/993867551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff73415d030 con 0x7ff7340b8a60 2023-07-18T03:40:40.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.944+0000 7ff72affd700 1 -- 172.21.15.49:0/993867551 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v24) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff73415d030 con 0x7ff7340b8a60 2023-07-18T03:40:40.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.958+0000 7ff73c3ff700 1 -- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff71806ad70 msgr2=0x7ff71806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.958+0000 7ff73c3ff700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff71806ad70 0x7ff71806d220 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff730000f60 tx=0x7ff730007be0 comp rx=0 tx=0).stop 2023-07-18T03:40:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.958+0000 7ff73c3ff700 1 -- 172.21.15.49:0/993867551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7340b8a60 msgr2=0x7ff7340cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:40.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.958+0000 7ff73c3ff700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7340b8a60 0x7ff7340cf6f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff724000c00 tx=0x7ff724004750 comp rx=0 tx=0).stop 2023-07-18T03:40:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.962+0000 7ff73c3ff700 1 -- 172.21.15.49:0/993867551 shutdown_connections 2023-07-18T03:40:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.962+0000 7ff73c3ff700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7340d0b10 0x7ff7340cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.962+0000 7ff73c3ff700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7340b0e20 0x7ff7341526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:40.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.962+0000 7ff73c3ff700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff71806ad70 0x7ff71806d220 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:40.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.962+0000 7ff73c3ff700 1 --2- 172.21.15.49:0/993867551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7340b8a60 0x7ff7340cf6f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:40.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.962+0000 7ff73c3ff700 1 -- 172.21.15.49:0/993867551 >> 172.21.15.49:0/993867551 conn(0x7ff73409c1b0 msgr2=0x7ff7340b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:40.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.972+0000 7ff73c3ff700 1 -- 172.21.15.49:0/993867551 shutdown_connections 2023-07-18T03:40:40.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:40.975+0000 7ff73c3ff700 1 -- 172.21.15.49:0/993867551 wait complete. 2023-07-18T03:40:40.975 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 7 and data pool 8 2023-07-18T03:40:40.994 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:41.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.206+0000 7f7cc33ff700 1 Processor -- start 2023-07-18T03:40:41.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.206+0000 7f7cc33ff700 1 -- start start 2023-07-18T03:40:41.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.207+0000 7f7cc33ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cbc15e890 0x7f7cbc0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:41.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.207+0000 7f7cc33ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7cbc15afd0 0x7f7cbc0b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:41.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.207+0000 7f7cc33ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cbc0ce110 con 0x7f7cbc0b3780 2023-07-18T03:40:41.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.207+0000 7f7cc33ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cbc15b450 con 0x7f7cbc15afd0 2023-07-18T03:40:41.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.207+0000 7f7cc33ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc0cf380 con 0x7f7cbc15e890 2023-07-18T03:40:41.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.207+0000 7f7cc099a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7cbc0b3780 0x7f7cbc15a8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50428/0 (socket says 172.21.15.49:50428) 2023-07-18T03:40:41.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.207+0000 7f7cc099a700 1 -- 172.21.15.49:0/1765612485 learned_addr learned my addr 172.21.15.49:0/1765612485 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:41.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.208+0000 7f7cc199c700 1 --2- 172.21.15.49:0/1765612485 >> v2:172.21.15.49:3300/0 conn(0x7f7cbc15afd0 0x7f7cbc0b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:41.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.208+0000 7f7cc119b700 1 --2- 172.21.15.49:0/1765612485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cbc15e890 0x7f7cbc0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:41.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.208+0000 7f7cb3fff700 1 -- 172.21.15.49:0/1765612485 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57009068 0 0) 0x7f7cbc0ce110 con 0x7f7cbc0b3780 2023-07-18T03:40:41.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.208+0000 7f7cc199c700 1 -- 172.21.15.49:0/1765612485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cbc15e890 msgr2=0x7f7cbc0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:41.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.208+0000 7f7cc199c700 1 --2- 172.21.15.49:0/1765612485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cbc15e890 0x7f7cbc0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.208+0000 7f7cc199c700 1 -- 172.21.15.49:0/1765612485 >> v1:172.21.15.49:6789/0 conn(0x7f7cbc0b3780 legacy=0x7f7cbc15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:41.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.208+0000 7f7cc199c700 1 -- 172.21.15.49:0/1765612485 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc15b660 con 0x7f7cbc15afd0 2023-07-18T03:40:41.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.208+0000 7f7cc199c700 1 --2- 172.21.15.49:0/1765612485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cbc15afd0 0x7f7cbc0b3bf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009d90 tx=0x7f7cb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856a2b52e4de11bd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:41.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.209+0000 7f7cb3fff700 1 -- 172.21.15.49:0/1765612485 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400e070 con 0x7f7cbc15afd0 2023-07-18T03:40:41.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.209+0000 7f7cb3fff700 1 -- 172.21.15.49:0/1765612485 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb400bb20 con 0x7f7cbc15afd0 2023-07-18T03:40:41.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.209+0000 7f7cb3fff700 1 -- 172.21.15.49:0/1765612485 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4012430 con 0x7f7cbc15afd0 2023-07-18T03:40:41.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.209+0000 7f7cc33ff700 1 -- 172.21.15.49:0/1765612485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cbc15afd0 msgr2=0x7f7cbc0b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:41.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.209+0000 7f7cc33ff700 1 --2- 172.21.15.49:0/1765612485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cbc15afd0 0x7f7cbc0b3bf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009d90 tx=0x7f7cb400b330 comp rx=0 tx=0).stop 2023-07-18T03:40:41.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.209+0000 7f7cc33ff700 1 -- 172.21.15.49:0/1765612485 shutdown_connections 2023-07-18T03:40:41.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.209+0000 7f7cc33ff700 1 --2- 172.21.15.49:0/1765612485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cbc15afd0 0x7f7cbc0b3bf0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.209+0000 7f7cc33ff700 1 --2- 172.21.15.49:0/1765612485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cbc15e890 0x7f7cbc0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.209+0000 7f7cc33ff700 1 -- 172.21.15.49:0/1765612485 >> 172.21.15.49:0/1765612485 conn(0x7f7cbc069f30 msgr2=0x7f7cbc06a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:41.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.210+0000 7f7cc33ff700 1 -- 172.21.15.49:0/1765612485 shutdown_connections 2023-07-18T03:40:41.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.210+0000 7f7cc33ff700 1 -- 172.21.15.49:0/1765612485 wait complete. 2023-07-18T03:40:41.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.211+0000 7f7cc33ff700 1 Processor -- start 2023-07-18T03:40:41.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.212+0000 7f7cc33ff700 1 -- start start 2023-07-18T03:40:41.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc33ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cbc0b3780 0x7f7cbc0c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:41.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc33ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cbc15afd0 0x7f7cbc1e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:41.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc33ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cbc15e890 0x7f7cbc152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:41.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc33ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc153060 con 0x7f7cbc0b3780 2023-07-18T03:40:41.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc33ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc0c4bd0 con 0x7f7cbc15afd0 2023-07-18T03:40:41.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc33ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc153570 con 0x7f7cbc15e890 2023-07-18T03:40:41.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc119b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cbc0b3780 0x7f7cbc0c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:41.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc119b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cbc0b3780 0x7f7cbc0c39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35486/0 (socket says 172.21.15.49:35486) 2023-07-18T03:40:41.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc119b700 1 -- 172.21.15.49:0/94371279 learned_addr learned my addr 172.21.15.49:0/94371279 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:41.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc099a700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cbc15afd0 0x7f7cbc1e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:41.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc199c700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cbc15e890 0x7f7cbc152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:41.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc119b700 1 -- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cbc15e890 msgr2=0x7f7cbc152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:41.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc119b700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cbc15e890 0x7f7cbc152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc119b700 1 -- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cbc15afd0 msgr2=0x7f7cbc1e84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:41.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc119b700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cbc15afd0 0x7f7cbc1e84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc119b700 1 -- 172.21.15.49:0/94371279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb4009a70 con 0x7f7cbc0b3780 2023-07-18T03:40:41.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.213+0000 7f7cc099a700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cbc15afd0 0x7f7cbc1e84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:41.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.214+0000 7f7cc119b700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cbc0b3780 0x7f7cbc0c39e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800e9e0 tx=0x7f7cb800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:41.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.214+0000 7f7cb1ffb700 1 -- 172.21.15.49:0/94371279 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8009710 con 0x7f7cbc0b3780 2023-07-18T03:40:41.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.214+0000 7f7cb1ffb700 1 -- 172.21.15.49:0/94371279 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb800c850 con 0x7f7cbc0b3780 2023-07-18T03:40:41.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.214+0000 7f7cb1ffb700 1 -- 172.21.15.49:0/94371279 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb800cbc0 con 0x7f7cbc0b3780 2023-07-18T03:40:41.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.214+0000 7f7cc33ff700 1 -- 172.21.15.49:0/94371279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cbc0c3fc0 con 0x7f7cbc0b3780 2023-07-18T03:40:41.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.214+0000 7f7cc33ff700 1 -- 172.21.15.49:0/94371279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cbc0c4580 con 0x7f7cbc0b3780 2023-07-18T03:40:41.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.215+0000 7f7cc33ff700 1 -- 172.21.15.49:0/94371279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cbc04e280 con 0x7f7cbc0b3780 2023-07-18T03:40:41.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.216+0000 7f7cb1ffb700 1 -- 172.21.15.49:0/94371279 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f7cb800cd20 con 0x7f7cbc0b3780 2023-07-18T03:40:41.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.217+0000 7f7cb1ffb700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7ca006ad70 0x7f7ca006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:41.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.217+0000 7f7cb1ffb700 1 -- 172.21.15.49:0/94371279 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7cb8089610 con 0x7f7cbc0b3780 2023-07-18T03:40:41.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.218+0000 7f7cc099a700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7ca006ad70 0x7f7ca006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:41.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.218+0000 7f7cc099a700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7ca006ad70 0x7f7ca006d220 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7cac007fd0 tx=0x7f7cac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.222+0000 7f7cb1ffb700 1 -- 172.21.15.49:0/94371279 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7cb80580f0 con 0x7f7cbc0b3780 2023-07-18T03:40:41.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.431+0000 7f7cc33ff700 1 -- 172.21.15.49:0/94371279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cbc0b14d0 con 0x7f7cbc0b3780 2023-07-18T03:40:41.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.432+0000 7f7cb1ffb700 1 -- 172.21.15.49:0/94371279 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+18270 (secure 0 0 0) 0x7f7cb8057c80 con 0x7f7cbc0b3780 2023-07-18T03:40:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:41.427 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":48,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:40:40.451353+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:40:38.792808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-18T03:40:39.438590+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:41.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.434+0000 7f7cc33ff700 1 -- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7ca006ad70 msgr2=0x7f7ca006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:41.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.434+0000 7f7cc33ff700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7ca006ad70 0x7f7ca006d220 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7cac007fd0 tx=0x7f7cac009040 comp rx=0 tx=0).stop 2023-07-18T03:40:41.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.434+0000 7f7cc33ff700 1 -- 172.21.15.49:0/94371279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cbc0b3780 msgr2=0x7f7cbc0c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:41.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.434+0000 7f7cc33ff700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cbc0b3780 0x7f7cbc0c39e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800e9e0 tx=0x7f7cb800eda0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.435+0000 7f7cc33ff700 1 -- 172.21.15.49:0/94371279 shutdown_connections 2023-07-18T03:40:41.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.435+0000 7f7cc33ff700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cbc15e890 0x7f7cbc152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.435+0000 7f7cc33ff700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f7ca006ad70 0x7f7ca006d220 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.435+0000 7f7cc33ff700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cbc0b3780 0x7f7cbc0c39e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.435+0000 7f7cc33ff700 1 --2- 172.21.15.49:0/94371279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cbc15afd0 0x7f7cbc1e84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.435+0000 7f7cc33ff700 1 -- 172.21.15.49:0/94371279 >> 172.21.15.49:0/94371279 conn(0x7f7cbc069f30 msgr2=0x7f7cbc0b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:41.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.435+0000 7f7cc33ff700 1 -- 172.21.15.49:0/94371279 shutdown_connections 2023-07-18T03:40:41.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.435+0000 7f7cc33ff700 1 -- 172.21.15.49:0/94371279 wait complete. 2023-07-18T03:40:41.447 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:41.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.657+0000 7f774a9ff700 1 Processor -- start 2023-07-18T03:40:41.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.657+0000 7f774a9ff700 1 -- start start 2023-07-18T03:40:41.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.659+0000 7f774a9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f77440b87d0 0x7f7744160150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:41.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.659+0000 7f774a9ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f774406ae40 0x7f77440d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:41.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.659+0000 7f774a9ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77440cd4e0 con 0x7f77440cfc60 2023-07-18T03:40:41.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.659+0000 7f774a9ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77440cdcf0 con 0x7f774406ae40 2023-07-18T03:40:41.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.659+0000 7f774a9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77440d00d0 con 0x7f77440b87d0 2023-07-18T03:40:41.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.659+0000 7f7748f9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f774406ae40 0x7f77440d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:41.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.659+0000 7f77437fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f77440cfc60 0x7f7744160690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50442/0 (socket says 172.21.15.49:50442) 2023-07-18T03:40:41.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.659+0000 7f77437fe700 1 -- 172.21.15.49:0/76492232 learned_addr learned my addr 172.21.15.49:0/76492232 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:41.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.659+0000 7f7743fff700 1 --2- 172.21.15.49:0/76492232 >> v2:172.21.15.107:3300/0 conn(0x7f77440b87d0 0x7f7744160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:41.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.660+0000 7f7748f9c700 1 -- 172.21.15.49:0/76492232 >> v2:172.21.15.107:3300/0 conn(0x7f77440b87d0 msgr2=0x7f7744160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:41.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.660+0000 7f7748f9c700 1 --2- 172.21.15.49:0/76492232 >> v2:172.21.15.107:3300/0 conn(0x7f77440b87d0 0x7f7744160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.660+0000 7f7748f9c700 1 -- 172.21.15.49:0/76492232 >> v1:172.21.15.49:6789/0 conn(0x7f77440cfc60 legacy=0x7f7744160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:41.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.660+0000 7f7742ffd700 1 -- 172.21.15.49:0/76492232 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793594486 0 0) 0x7f77440cd4e0 con 0x7f77440cfc60 2023-07-18T03:40:41.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.660+0000 7f7748f9c700 1 -- 172.21.15.49:0/76492232 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77440cd730 con 0x7f774406ae40 2023-07-18T03:40:41.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.660+0000 7f7748f9c700 1 --2- 172.21.15.49:0/76492232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f774406ae40 0x7f77440d0e20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f77440b5a50 tx=0x7f773c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784996d86d7ab84 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:41.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.660+0000 7f7742ffd700 1 -- 172.21.15.49:0/76492232 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c00e070 con 0x7f774406ae40 2023-07-18T03:40:41.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.660+0000 7f7742ffd700 1 -- 172.21.15.49:0/76492232 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773c00bb20 con 0x7f774406ae40 2023-07-18T03:40:41.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.660+0000 7f7742ffd700 1 -- 172.21.15.49:0/76492232 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c012430 con 0x7f774406ae40 2023-07-18T03:40:41.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.661+0000 7f774a9ff700 1 -- 172.21.15.49:0/76492232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f774406ae40 msgr2=0x7f77440d0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:41.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.661+0000 7f774a9ff700 1 --2- 172.21.15.49:0/76492232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f774406ae40 0x7f77440d0e20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f77440b5a50 tx=0x7f773c00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:41.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.661+0000 7f774a9ff700 1 -- 172.21.15.49:0/76492232 shutdown_connections 2023-07-18T03:40:41.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.661+0000 7f774a9ff700 1 --2- 172.21.15.49:0/76492232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f774406ae40 0x7f77440d0e20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.661+0000 7f774a9ff700 1 --2- 172.21.15.49:0/76492232 >> v2:172.21.15.107:3300/0 conn(0x7f77440b87d0 0x7f7744160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.661+0000 7f774a9ff700 1 -- 172.21.15.49:0/76492232 >> 172.21.15.49:0/76492232 conn(0x7f774406c140 msgr2=0x7f774406c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.661+0000 7f774a9ff700 1 -- 172.21.15.49:0/76492232 shutdown_connections 2023-07-18T03:40:41.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.661+0000 7f774a9ff700 1 -- 172.21.15.49:0/76492232 wait complete. 2023-07-18T03:40:41.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.663+0000 7f774a9ff700 1 Processor -- start 2023-07-18T03:40:41.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.663+0000 7f774a9ff700 1 -- start start 2023-07-18T03:40:41.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.664+0000 7f774a9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77440b87d0 0x7f7744157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:41.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.664+0000 7f774a9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77440cfc60 0x7f77440c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:41.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.664+0000 7f774a9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77440c79d0 0x7f77441f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:41.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.664+0000 7f774a9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77440b2230 con 0x7f77440cfc60 2023-07-18T03:40:41.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.664+0000 7f774a9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77440b27d0 con 0x7f77440b87d0 2023-07-18T03:40:41.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.664+0000 7f774a9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77440b3350 con 0x7f77440c79d0 2023-07-18T03:40:41.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.664+0000 7f77437fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77440cfc60 0x7f77440c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:41.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.664+0000 7f77437fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77440cfc60 0x7f77440c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35504/0 (socket says 172.21.15.49:35504) 2023-07-18T03:40:41.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.664+0000 7f77437fe700 1 -- 172.21.15.49:0/2302414509 learned_addr learned my addr 172.21.15.49:0/2302414509 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:41.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.664+0000 7f7748f9c700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77440c79d0 0x7f77441f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:41.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.664+0000 7f7743fff700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77440b87d0 0x7f7744157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:41.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f77437fe700 1 -- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77440c79d0 msgr2=0x7f77441f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:41.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f77437fe700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77440c79d0 0x7f77441f2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f77437fe700 1 -- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77440b87d0 msgr2=0x7f7744157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:41.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f77437fe700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77440b87d0 0x7f7744157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f77437fe700 1 -- 172.21.15.49:0/2302414509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c009a70 con 0x7f77440cfc60 2023-07-18T03:40:41.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f7743fff700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77440b87d0 0x7f7744157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:41.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f7748f9c700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77440c79d0 0x7f77441f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:41.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f77437fe700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77440cfc60 0x7f77440c7490 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f773800c580 tx=0x7f773800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:41.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f7740ff9700 1 -- 172.21.15.49:0/2302414509 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738004c80 con 0x7f77440cfc60 2023-07-18T03:40:41.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f7740ff9700 1 -- 172.21.15.49:0/2302414509 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7738004e00 con 0x7f77440cfc60 2023-07-18T03:40:41.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f774a9ff700 1 -- 172.21.15.49:0/2302414509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77440b2a20 con 0x7f77440cfc60 2023-07-18T03:40:41.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f7740ff9700 1 -- 172.21.15.49:0/2302414509 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738007310 con 0x7f77440cfc60 2023-07-18T03:40:41.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.665+0000 7f774a9ff700 1 -- 172.21.15.49:0/2302414509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77440b2fe0 con 0x7f77440cfc60 2023-07-18T03:40:41.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.666+0000 7f774a9ff700 1 -- 172.21.15.49:0/2302414509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7744066f00 con 0x7f77440cfc60 2023-07-18T03:40:41.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.667+0000 7f7740ff9700 1 -- 172.21.15.49:0/2302414509 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f7738007470 con 0x7f77440cfc60 2023-07-18T03:40:41.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.668+0000 7f7740ff9700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f773006ad70 0x7f773006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:41.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.668+0000 7f7740ff9700 1 -- 172.21.15.49:0/2302414509 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7738089230 con 0x7f77440cfc60 2023-07-18T03:40:41.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.669+0000 7f7743fff700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f773006ad70 0x7f773006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:41.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.670+0000 7f7743fff700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f773006ad70 0x7f773006d220 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7734009c30 tx=0x7f7734008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:41.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.673+0000 7f7740ff9700 1 -- 172.21.15.49:0/2302414509 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f77380541f0 con 0x7f77440cfc60 2023-07-18T03:40:41.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.881+0000 7f774a9ff700 1 -- 172.21.15.49:0/2302414509 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f774404e280 con 0x7f77440cfc60 2023-07-18T03:40:41.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.883+0000 7f7740ff9700 1 -- 172.21.15.49:0/2302414509 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+18270 (secure 0 0 0) 0x7f7738057810 con 0x7f77440cfc60 2023-07-18T03:40:41.877 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:41.878 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":48,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:40:40.451353+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:40:38.792808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-18T03:40:39.438590+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.885+0000 7f774a9ff700 1 -- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f773006ad70 msgr2=0x7f773006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.885+0000 7f774a9ff700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f773006ad70 0x7f773006d220 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7734009c30 tx=0x7f7734008040 comp rx=0 tx=0).stop 2023-07-18T03:40:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.885+0000 7f774a9ff700 1 -- 172.21.15.49:0/2302414509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77440cfc60 msgr2=0x7f77440c7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.885+0000 7f774a9ff700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77440cfc60 0x7f77440c7490 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f773800c580 tx=0x7f773800c940 comp rx=0 tx=0).stop 2023-07-18T03:40:41.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.885+0000 7f774a9ff700 1 -- 172.21.15.49:0/2302414509 shutdown_connections 2023-07-18T03:40:41.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.885+0000 7f774a9ff700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77440c79d0 0x7f77441f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.885+0000 7f774a9ff700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77440b87d0 0x7f7744157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.885+0000 7f774a9ff700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f773006ad70 0x7f773006d220 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.885+0000 7f774a9ff700 1 --2- 172.21.15.49:0/2302414509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77440cfc60 0x7f77440c7490 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:41.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.885+0000 7f774a9ff700 1 -- 172.21.15.49:0/2302414509 >> 172.21.15.49:0/2302414509 conn(0x7f774406c140 msgr2=0x7f774415c140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:41.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.885+0000 7f774a9ff700 1 -- 172.21.15.49:0/2302414509 shutdown_connections 2023-07-18T03:40:41.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:41.885+0000 7f774a9ff700 1 -- 172.21.15.49:0/2302414509 wait complete. 2023-07-18T03:40:41.898 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-18T03:40:42.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.115+0000 7f43aa7ff700 1 Processor -- start 2023-07-18T03:40:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.115+0000 7f43aa7ff700 1 -- start start 2023-07-18T03:40:42.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.117+0000 7f43aa7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a40d0b10 0x7f43a40d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:42.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.117+0000 7f43aa7ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f43a40b0e20 0x7f43a40b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:42.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.117+0000 7f43aa7ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a40d14a0 con 0x7f43a40b8a60 2023-07-18T03:40:42.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.117+0000 7f43aa7ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a40b8e30 con 0x7f43a40b0e20 2023-07-18T03:40:42.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.117+0000 7f43aa7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40afaa0 con 0x7f43a40d0b10 2023-07-18T03:40:42.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.117+0000 7f43a3fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f43a40b8a60 0x7f43a40b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50454/0 (socket says 172.21.15.49:50454) 2023-07-18T03:40:42.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.117+0000 7f43a3fff700 1 -- 172.21.15.49:0/832305276 learned_addr learned my addr 172.21.15.49:0/832305276 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:42.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.117+0000 7f43a8d9c700 1 --2- 172.21.15.49:0/832305276 >> v2:172.21.15.49:3300/0 conn(0x7f43a40b0e20 0x7f43a40b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:42.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.117+0000 7f43a37fe700 1 --2- 172.21.15.49:0/832305276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a40d0b10 0x7f43a40d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:42.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.118+0000 7f43a2ffd700 1 -- 172.21.15.49:0/832305276 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585284840 0 0) 0x7f4394001110 con 0x7f43a40b8a60 2023-07-18T03:40:42.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.118+0000 7f43a8d9c700 1 -- 172.21.15.49:0/832305276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a40d0b10 msgr2=0x7f43a40d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:42.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.118+0000 7f43a8d9c700 1 --2- 172.21.15.49:0/832305276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a40d0b10 0x7f43a40d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:42.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.118+0000 7f43a8d9c700 1 -- 172.21.15.49:0/832305276 >> v1:172.21.15.49:6789/0 conn(0x7f43a40b8a60 legacy=0x7f43a40b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:42.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.118+0000 7f43a8d9c700 1 -- 172.21.15.49:0/832305276 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a40afd80 con 0x7f43a40b0e20 2023-07-18T03:40:42.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.118+0000 7f43a8d9c700 1 --2- 172.21.15.49:0/832305276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a40b0e20 0x7f43a40b12a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f439c009d90 tx=0x7f439c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedda271b6346dfd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:42.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.118+0000 7f43a2ffd700 1 -- 172.21.15.49:0/832305276 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c00e070 con 0x7f43a40b0e20 2023-07-18T03:40:42.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.118+0000 7f43a2ffd700 1 -- 172.21.15.49:0/832305276 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c00bb20 con 0x7f43a40b0e20 2023-07-18T03:40:42.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.118+0000 7f43a2ffd700 1 -- 172.21.15.49:0/832305276 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c012430 con 0x7f43a40b0e20 2023-07-18T03:40:42.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.119+0000 7f43aa7ff700 1 -- 172.21.15.49:0/832305276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a40b0e20 msgr2=0x7f43a40b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:42.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.119+0000 7f43aa7ff700 1 --2- 172.21.15.49:0/832305276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a40b0e20 0x7f43a40b12a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f439c009d90 tx=0x7f439c00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:42.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.119+0000 7f43aa7ff700 1 -- 172.21.15.49:0/832305276 shutdown_connections 2023-07-18T03:40:42.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.119+0000 7f43aa7ff700 1 --2- 172.21.15.49:0/832305276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a40b0e20 0x7f43a40b12a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:42.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.119+0000 7f43aa7ff700 1 --2- 172.21.15.49:0/832305276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a40d0b10 0x7f43a40d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:42.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.119+0000 7f43aa7ff700 1 -- 172.21.15.49:0/832305276 >> 172.21.15.49:0/832305276 conn(0x7f43a409c1b0 msgr2=0x7f43a40d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:42.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.119+0000 7f43aa7ff700 1 -- 172.21.15.49:0/832305276 shutdown_connections 2023-07-18T03:40:42.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.119+0000 7f43aa7ff700 1 -- 172.21.15.49:0/832305276 wait complete. 2023-07-18T03:40:42.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.121+0000 7f43aa7ff700 1 Processor -- start 2023-07-18T03:40:42.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.121+0000 7f43aa7ff700 1 -- start start 2023-07-18T03:40:42.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.122+0000 7f43aa7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a40b0e20 0x7f43a41f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:42.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.122+0000 7f43aa7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43a40b8a60 0x7f43a415f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:42.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.122+0000 7f43aa7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a40d0b10 0x7f43a40ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:42.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.122+0000 7f43aa7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a415fde0 con 0x7f43a40b0e20 2023-07-18T03:40:42.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.122+0000 7f43aa7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40cbca0 con 0x7f43a40b8a60 2023-07-18T03:40:42.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.122+0000 7f43aa7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40cbf80 con 0x7f43a40d0b10 2023-07-18T03:40:42.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.122+0000 7f43a3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a40b0e20 0x7f43a41f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:42.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.122+0000 7f43a3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a40b0e20 0x7f43a41f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35524/0 (socket says 172.21.15.49:35524) 2023-07-18T03:40:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.122+0000 7f43a3fff700 1 -- 172.21.15.49:0/101981571 learned_addr learned my addr 172.21.15.49:0/101981571 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.122+0000 7f43a37fe700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43a40b8a60 0x7f43a415f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.122+0000 7f43a8d9c700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a40d0b10 0x7f43a40ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.123+0000 7f43a3fff700 1 -- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a40d0b10 msgr2=0x7f43a40ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.123+0000 7f43a3fff700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a40d0b10 0x7f43a40ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.123+0000 7f43a3fff700 1 -- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43a40b8a60 msgr2=0x7f43a415f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.123+0000 7f43a3fff700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43a40b8a60 0x7f43a415f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:42.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.123+0000 7f43a3fff700 1 -- 172.21.15.49:0/101981571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c009a70 con 0x7f43a40b0e20 2023-07-18T03:40:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.123+0000 7f43a37fe700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43a40b8a60 0x7f43a415f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.123+0000 7f43a3fff700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a40b0e20 0x7f43a41f2ab0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f439400c730 tx=0x7f439400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.123+0000 7f43a0ff9700 1 -- 172.21.15.49:0/101981571 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394004c80 con 0x7f43a40b0e20 2023-07-18T03:40:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.123+0000 7f43a0ff9700 1 -- 172.21.15.49:0/101981571 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4394004e00 con 0x7f43a40b0e20 2023-07-18T03:40:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.123+0000 7f43aa7ff700 1 -- 172.21.15.49:0/101981571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a40cc260 con 0x7f43a40b0e20 2023-07-18T03:40:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.123+0000 7f43a0ff9700 1 -- 172.21.15.49:0/101981571 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394007310 con 0x7f43a40b0e20 2023-07-18T03:40:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.123+0000 7f43aa7ff700 1 -- 172.21.15.49:0/101981571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a40cac70 con 0x7f43a40b0e20 2023-07-18T03:40:42.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.124+0000 7f43aa7ff700 1 -- 172.21.15.49:0/101981571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a4074350 con 0x7f43a40b0e20 2023-07-18T03:40:42.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.125+0000 7f43a0ff9700 1 -- 172.21.15.49:0/101981571 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f4394007470 con 0x7f43a40b0e20 2023-07-18T03:40:42.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.126+0000 7f43a0ff9700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f439006ad70 0x7f439006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:42.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.126+0000 7f43a0ff9700 1 -- 172.21.15.49:0/101981571 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f43940891b0 con 0x7f43a40b0e20 2023-07-18T03:40:42.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.127+0000 7f43a37fe700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f439006ad70 0x7f439006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:42.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.127+0000 7f43a37fe700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f439006ad70 0x7f439006d220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f43a40c9e20 tx=0x7f4398008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:42.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.131+0000 7f43a0ff9700 1 -- 172.21.15.49:0/101981571 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4394054170 con 0x7f43a40b0e20 2023-07-18T03:40:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.397+0000 7f43aa7ff700 1 -- 172.21.15.49:0/101981571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f43a40d14a0 con 0x7f43a40b0e20 2023-07-18T03:40:42.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.955+0000 7f43a0ff9700 1 -- 172.21.15.49:0/101981571 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v27) v1 ==== 118+0+0 (secure 0 0 0) 0x7f43a40d14a0 con 0x7f43a40b0e20 2023-07-18T03:40:42.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.958+0000 7f43aa7ff700 1 -- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f439006ad70 msgr2=0x7f439006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:42.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.958+0000 7f43aa7ff700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f439006ad70 0x7f439006d220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f43a40c9e20 tx=0x7f4398008040 comp rx=0 tx=0).stop 2023-07-18T03:40:42.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.958+0000 7f43aa7ff700 1 -- 172.21.15.49:0/101981571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a40b0e20 msgr2=0x7f43a41f2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:42.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.958+0000 7f43aa7ff700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a40b0e20 0x7f43a41f2ab0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f439400c730 tx=0x7f439400caf0 comp rx=0 tx=0).stop 2023-07-18T03:40:42.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.958+0000 7f43aa7ff700 1 -- 172.21.15.49:0/101981571 shutdown_connections 2023-07-18T03:40:42.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.958+0000 7f43aa7ff700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a40d0b10 0x7f43a40ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:42.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.958+0000 7f43aa7ff700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f439006ad70 0x7f439006d220 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:42.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.958+0000 7f43aa7ff700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43a40b0e20 0x7f43a41f2ab0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:42.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.958+0000 7f43aa7ff700 1 --2- 172.21.15.49:0/101981571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43a40b8a60 0x7f43a415f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:42.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.958+0000 7f43aa7ff700 1 -- 172.21.15.49:0/101981571 >> 172.21.15.49:0/101981571 conn(0x7f43a409c1b0 msgr2=0x7f43a40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:42.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.958+0000 7f43aa7ff700 1 -- 172.21.15.49:0/101981571 shutdown_connections 2023-07-18T03:40:42.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:42.958+0000 7f43aa7ff700 1 -- 172.21.15.49:0/101981571 wait complete. 2023-07-18T03:40:42.972 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-18T03:40:42.973 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-18T03:40:43.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.189+0000 7f638bbff700 1 Processor -- start 2023-07-18T03:40:43.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.189+0000 7f638bbff700 1 -- start start 2023-07-18T03:40:43.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.190+0000 7f638bbff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f63840b8a60 0x7f63840b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:43.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.190+0000 7f638bbff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f63840b4530 0x7f63840d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:43.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.190+0000 7f638bbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63840d1050 0x7f63840b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:43.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.190+0000 7f638bbff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63840d14d0 con 0x7f63840b4530 2023-07-18T03:40:43.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.190+0000 7f638bbff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63840b8e30 con 0x7f63840b8a60 2023-07-18T03:40:43.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.190+0000 7f638bbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63840afaa0 con 0x7f63840d1050 2023-07-18T03:40:43.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.190+0000 7f638919a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f63840b4530 0x7f63840d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:43.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.190+0000 7f638919a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f63840b4530 0x7f63840d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35540/0 (socket says 172.21.15.49:35540) 2023-07-18T03:40:43.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.190+0000 7f638919a700 1 -- 172.21.15.49:0/2277799301 learned_addr learned my addr 172.21.15.49:0/2277799301 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:43.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f638999b700 1 --2- 172.21.15.49:0/2277799301 >> v2:172.21.15.107:3300/0 conn(0x7f63840b8a60 0x7f63840b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:43.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f638a19c700 1 --2- 172.21.15.49:0/2277799301 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63840d1050 0x7f63840b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:43.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f638919a700 1 -- 172.21.15.49:0/2277799301 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63840d1050 msgr2=0x7f63840b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:43.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f638919a700 1 --2- 172.21.15.49:0/2277799301 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63840d1050 0x7f63840b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:43.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f638919a700 1 -- 172.21.15.49:0/2277799301 >> v2:172.21.15.107:3300/0 conn(0x7f63840b8a60 msgr2=0x7f63840b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:43.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f638919a700 1 --2- 172.21.15.49:0/2277799301 >> v2:172.21.15.107:3300/0 conn(0x7f63840b8a60 0x7f63840b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f638919a700 1 -- 172.21.15.49:0/2277799301 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63840afd80 con 0x7f63840b4530 2023-07-18T03:40:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f638999b700 1 --2- 172.21.15.49:0/2277799301 >> v2:172.21.15.107:3300/0 conn(0x7f63840b8a60 0x7f63840b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f638a19c700 1 --2- 172.21.15.49:0/2277799301 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63840d1050 0x7f63840b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:43.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f638919a700 1 --2- 172.21.15.49:0/2277799301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63840b4530 0x7f63840d0b10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f63840b5b00 tx=0x7f6374009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2593bff1785613 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f6388999700 1 -- 172.21.15.49:0/2277799301 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637400b870 con 0x7f63840b4530 2023-07-18T03:40:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f6388999700 1 -- 172.21.15.49:0/2277799301 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637400b9d0 con 0x7f63840b4530 2023-07-18T03:40:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.191+0000 7f6388999700 1 -- 172.21.15.49:0/2277799301 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637400bd60 con 0x7f63840b4530 2023-07-18T03:40:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.192+0000 7f638bbff700 1 -- 172.21.15.49:0/2277799301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63840b4530 msgr2=0x7f63840d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.192+0000 7f638bbff700 1 --2- 172.21.15.49:0/2277799301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63840b4530 0x7f63840d0b10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f63840b5b00 tx=0x7f6374009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.192+0000 7f638bbff700 1 -- 172.21.15.49:0/2277799301 shutdown_connections 2023-07-18T03:40:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.192+0000 7f638bbff700 1 --2- 172.21.15.49:0/2277799301 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63840d1050 0x7f63840b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.192+0000 7f638bbff700 1 --2- 172.21.15.49:0/2277799301 >> v2:172.21.15.107:3300/0 conn(0x7f63840b8a60 0x7f63840b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.192+0000 7f638bbff700 1 --2- 172.21.15.49:0/2277799301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63840b4530 0x7f63840d0b10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.192+0000 7f638bbff700 1 -- 172.21.15.49:0/2277799301 >> 172.21.15.49:0/2277799301 conn(0x7f638409c1b0 msgr2=0x7f63840d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.192+0000 7f638bbff700 1 -- 172.21.15.49:0/2277799301 shutdown_connections 2023-07-18T03:40:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.192+0000 7f638bbff700 1 -- 172.21.15.49:0/2277799301 wait complete. 2023-07-18T03:40:43.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.194+0000 7f638bbff700 1 Processor -- start 2023-07-18T03:40:43.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.194+0000 7f638bbff700 1 -- start start 2023-07-18T03:40:43.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.195+0000 7f638bbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63840b4530 0x7f63840d0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:43.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.195+0000 7f638bbff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63840b8a60 0x7f63840cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:43.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.195+0000 7f638bbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63840d1050 0x7f63841564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:43.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.195+0000 7f638bbff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63840cf070 con 0x7f63840b8a60 2023-07-18T03:40:43.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.195+0000 7f638bbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63840ce7d0 con 0x7f63840b4530 2023-07-18T03:40:43.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.195+0000 7f638bbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63840ceab0 con 0x7f63840d1050 2023-07-18T03:40:43.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.195+0000 7f638919a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63840b8a60 0x7f63840cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:43.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.195+0000 7f638919a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63840b8a60 0x7f63840cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35542/0 (socket says 172.21.15.49:35542) 2023-07-18T03:40:43.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.195+0000 7f638919a700 1 -- 172.21.15.49:0/2108037690 learned_addr learned my addr 172.21.15.49:0/2108037690 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:43.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.195+0000 7f638999b700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63840b4530 0x7f63840d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:43.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.195+0000 7f638a19c700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63840d1050 0x7f63841564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:43.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.196+0000 7f638919a700 1 -- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63840d1050 msgr2=0x7f63841564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:43.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.196+0000 7f638919a700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63840d1050 0x7f63841564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:43.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.196+0000 7f638919a700 1 -- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63840b4530 msgr2=0x7f63840d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:43.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.196+0000 7f638919a700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63840b4530 0x7f63840d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:43.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.196+0000 7f638919a700 1 -- 172.21.15.49:0/2108037690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6374009550 con 0x7f63840b8a60 2023-07-18T03:40:43.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.196+0000 7f638999b700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63840b4530 0x7f63840d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:43.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.196+0000 7f638919a700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63840b8a60 0x7f63840cdf80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f63840681d0 tx=0x7f6374004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:43.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.196+0000 7f637a7fc700 1 -- 172.21.15.49:0/2108037690 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374004310 con 0x7f63840b8a60 2023-07-18T03:40:43.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.196+0000 7f637a7fc700 1 -- 172.21.15.49:0/2108037690 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637401f070 con 0x7f63840b8a60 2023-07-18T03:40:43.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.196+0000 7f637a7fc700 1 -- 172.21.15.49:0/2108037690 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63740036a0 con 0x7f63840b8a60 2023-07-18T03:40:43.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.196+0000 7f638bbff700 1 -- 172.21.15.49:0/2108037690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63840ced90 con 0x7f63840b8a60 2023-07-18T03:40:43.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.196+0000 7f638bbff700 1 -- 172.21.15.49:0/2108037690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63840ce4c0 con 0x7f63840b8a60 2023-07-18T03:40:43.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.197+0000 7f638bbff700 1 -- 172.21.15.49:0/2108037690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6384074350 con 0x7f63840b8a60 2023-07-18T03:40:43.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.198+0000 7f637a7fc700 1 -- 172.21.15.49:0/2108037690 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f6374021030 con 0x7f63840b8a60 2023-07-18T03:40:43.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.199+0000 7f637a7fc700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f636806ad70 0x7f636806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:43.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.199+0000 7f637a7fc700 1 -- 172.21.15.49:0/2108037690 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f637408aad0 con 0x7f63840b8a60 2023-07-18T03:40:43.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.200+0000 7f638999b700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f636806ad70 0x7f636806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:43.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.200+0000 7f638999b700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f636806ad70 0x7f636806d220 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6380006fd0 tx=0x7f6380008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:43.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.204+0000 7f637a7fc700 1 -- 172.21.15.49:0/2108037690 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6374055c40 con 0x7f63840b8a60 2023-07-18T03:40:43.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.457+0000 7f638bbff700 1 -- 172.21.15.49:0/2108037690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f63840b08f0 con 0x7f63840b8a60 2023-07-18T03:40:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.961+0000 7f637a7fc700 1 -- 172.21.15.49:0/2108037690 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6374059260 con 0x7f63840b8a60 2023-07-18T03:40:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.964+0000 7f638bbff700 1 -- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f636806ad70 msgr2=0x7f636806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.964+0000 7f638bbff700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f636806ad70 0x7f636806d220 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6380006fd0 tx=0x7f6380008040 comp rx=0 tx=0).stop 2023-07-18T03:40:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.964+0000 7f638bbff700 1 -- 172.21.15.49:0/2108037690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63840b8a60 msgr2=0x7f63840cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.964+0000 7f638bbff700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63840b8a60 0x7f63840cdf80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f63840681d0 tx=0x7f6374004750 comp rx=0 tx=0).stop 2023-07-18T03:40:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.964+0000 7f638bbff700 1 -- 172.21.15.49:0/2108037690 shutdown_connections 2023-07-18T03:40:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.964+0000 7f638bbff700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63840d1050 0x7f63841564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.964+0000 7f638bbff700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63840b4530 0x7f63840d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.964+0000 7f638bbff700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f636806ad70 0x7f636806d220 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.964+0000 7f638bbff700 1 --2- 172.21.15.49:0/2108037690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f63840b8a60 0x7f63840cdf80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.964+0000 7f638bbff700 1 -- 172.21.15.49:0/2108037690 >> 172.21.15.49:0/2108037690 conn(0x7f638409c1b0 msgr2=0x7f63840b7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.964+0000 7f638bbff700 1 -- 172.21.15.49:0/2108037690 shutdown_connections 2023-07-18T03:40:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:43.964+0000 7f638bbff700 1 -- 172.21.15.49:0/2108037690 wait complete. 2023-07-18T03:40:43.978 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-18T03:40:43.978 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:40:44.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.191+0000 7f618ebff700 1 Processor -- start 2023-07-18T03:40:44.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.191+0000 7f618ebff700 1 -- start start 2023-07-18T03:40:44.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.192+0000 7f618ebff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f61880b4510 0x7f61880b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.192+0000 7f618ebff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f61880d0b10 0x7f61880d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.192+0000 7f618ebff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61880b7f50 con 0x7f61880b5d90 2023-07-18T03:40:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.192+0000 7f618ebff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61880b48e0 con 0x7f61880d0b10 2023-07-18T03:40:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.192+0000 7f618ebff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61880d14d0 con 0x7f61880b4510 2023-07-18T03:40:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.192+0000 7f618d19c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f61880d0b10 0x7f61880d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.192+0000 7f618d19c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f61880d0b10 0x7f61880d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35554/0 (socket says 172.21.15.49:35554) 2023-07-18T03:40:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.193+0000 7f618d19c700 1 -- 172.21.15.49:0/1120746082 learned_addr learned my addr 172.21.15.49:0/1120746082 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.193+0000 7f618c99b700 1 --2- 172.21.15.49:0/1120746082 >> v2:172.21.15.107:3300/0 conn(0x7f61880b4510 0x7f61880b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.193+0000 7f618d19c700 1 -- 172.21.15.49:0/1120746082 >> v2:172.21.15.107:3300/0 conn(0x7f61880b4510 msgr2=0x7f61880b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.193+0000 7f618d19c700 1 --2- 172.21.15.49:0/1120746082 >> v2:172.21.15.107:3300/0 conn(0x7f61880b4510 0x7f61880b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.193+0000 7f618d19c700 1 -- 172.21.15.49:0/1120746082 >> v1:172.21.15.107:6789/0 conn(0x7f61880b5d90 legacy=0x7f61880cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.193+0000 7f618d19c700 1 -- 172.21.15.49:0/1120746082 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61880cdca0 con 0x7f61880d0b10 2023-07-18T03:40:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.193+0000 7f618d19c700 1 --2- 172.21.15.49:0/1120746082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61880d0b10 0x7f61880d0f90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f618808bd80 tx=0x7f618000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcdc36088758dad server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.193+0000 7f61877fe700 1 -- 172.21.15.49:0/1120746082 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000e070 con 0x7f61880d0b10 2023-07-18T03:40:44.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.194+0000 7f61877fe700 1 -- 172.21.15.49:0/1120746082 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618000b9c0 con 0x7f61880d0b10 2023-07-18T03:40:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.194+0000 7f61877fe700 1 -- 172.21.15.49:0/1120746082 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000bd70 con 0x7f61880d0b10 2023-07-18T03:40:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.194+0000 7f618ebff700 1 -- 172.21.15.49:0/1120746082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61880d0b10 msgr2=0x7f61880d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.194+0000 7f618ebff700 1 --2- 172.21.15.49:0/1120746082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61880d0b10 0x7f61880d0f90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f618808bd80 tx=0x7f618000b330 comp rx=0 tx=0).stop 2023-07-18T03:40:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.194+0000 7f618ebff700 1 -- 172.21.15.49:0/1120746082 shutdown_connections 2023-07-18T03:40:44.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.194+0000 7f618ebff700 1 --2- 172.21.15.49:0/1120746082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61880d0b10 0x7f61880d0f90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.194+0000 7f618ebff700 1 --2- 172.21.15.49:0/1120746082 >> v2:172.21.15.107:3300/0 conn(0x7f61880b4510 0x7f61880b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.194+0000 7f618ebff700 1 -- 172.21.15.49:0/1120746082 >> 172.21.15.49:0/1120746082 conn(0x7f618809c1d0 msgr2=0x7f61880d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.194+0000 7f618ebff700 1 -- 172.21.15.49:0/1120746082 shutdown_connections 2023-07-18T03:40:44.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.194+0000 7f618ebff700 1 -- 172.21.15.49:0/1120746082 wait complete. 2023-07-18T03:40:44.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.196+0000 7f618ebff700 1 Processor -- start 2023-07-18T03:40:44.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.196+0000 7f618ebff700 1 -- start start 2023-07-18T03:40:44.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.197+0000 7f618ebff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61880b4510 0x7f61880c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.197+0000 7f618ebff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61880b5d90 0x7f6188154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.197+0000 7f618ebff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61880d0b10 0x7f6188150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.197+0000 7f618ebff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61881550f0 con 0x7f61880b5d90 2023-07-18T03:40:44.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.197+0000 7f618ebff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6188151ab0 con 0x7f61880d0b10 2023-07-18T03:40:44.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.197+0000 7f618ebff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61880c9330 con 0x7f61880b4510 2023-07-18T03:40:44.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.197+0000 7f6187fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61880b5d90 0x7f6188154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.198+0000 7f6187fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61880b5d90 0x7f6188154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35566/0 (socket says 172.21.15.49:35566) 2023-07-18T03:40:44.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.198+0000 7f6187fff700 1 -- 172.21.15.49:0/1310833732 learned_addr learned my addr 172.21.15.49:0/1310833732 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:44.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.198+0000 7f618d19c700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61880d0b10 0x7f6188150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.198+0000 7f618c99b700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61880b4510 0x7f61880c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.198+0000 7f6187fff700 1 -- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61880b4510 msgr2=0x7f61880c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.198+0000 7f6187fff700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61880b4510 0x7f61880c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.198+0000 7f6187fff700 1 -- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61880d0b10 msgr2=0x7f6188150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.198+0000 7f6187fff700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61880d0b10 0x7f6188150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.198+0000 7f6187fff700 1 -- 172.21.15.49:0/1310833732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6180009b30 con 0x7f61880b5d90 2023-07-18T03:40:44.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.198+0000 7f6187fff700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61880b5d90 0x7f6188154bb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f617400c450 tx=0x7f617400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:44.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.198+0000 7f61857fa700 1 -- 172.21.15.49:0/1310833732 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61740052d0 con 0x7f61880b5d90 2023-07-18T03:40:44.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.199+0000 7f61857fa700 1 -- 172.21.15.49:0/1310833732 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6174005430 con 0x7f61880b5d90 2023-07-18T03:40:44.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.199+0000 7f618ebff700 1 -- 172.21.15.49:0/1310833732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61880c9610 con 0x7f61880b5d90 2023-07-18T03:40:44.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.199+0000 7f61857fa700 1 -- 172.21.15.49:0/1310833732 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61740038c0 con 0x7f61880b5d90 2023-07-18T03:40:44.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.199+0000 7f618ebff700 1 -- 172.21.15.49:0/1310833732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61880c9bd0 con 0x7f61880b5d90 2023-07-18T03:40:44.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.200+0000 7f618ebff700 1 -- 172.21.15.49:0/1310833732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61880c9dc0 con 0x7f61880b5d90 2023-07-18T03:40:44.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.201+0000 7f61857fa700 1 -- 172.21.15.49:0/1310833732 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f6174003a20 con 0x7f61880b5d90 2023-07-18T03:40:44.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.203+0000 7f61857fa700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f616c06ad70 0x7f616c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.203+0000 7f61857fa700 1 -- 172.21.15.49:0/1310833732 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6174088820 con 0x7f61880b5d90 2023-07-18T03:40:44.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.203+0000 7f618c99b700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f616c06ad70 0x7f616c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.204+0000 7f618c99b700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f616c06ad70 0x7f616c06d220 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f617c006fd0 tx=0x7f617c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:44.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.207+0000 7f61857fa700 1 -- 172.21.15.49:0/1310833732 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6174053870 con 0x7f61880b5d90 2023-07-18T03:40:44.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.458+0000 7f618ebff700 1 -- 172.21.15.49:0/1310833732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f618804e280 con 0x7f61880b5d90 2023-07-18T03:40:44.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.459+0000 7f61857fa700 1 -- 172.21.15.49:0/1310833732 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f617400ce60 con 0x7f61880b5d90 2023-07-18T03:40:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:44.453 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/777524677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":777524677},{"type":"v1","addr":"172.21.15.107:6837","nonce":777524677}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/2382452558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2382452558},{"type":"v1","addr":"172.21.15.49:6834","nonce":2382452558}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/2843143219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2843143219},{"type":"v1","addr":"172.21.15.107:6836","nonce":2843143219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:40:40.451705+0000","modified":"2023-07-18T03:40:43.957663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5479":{"gid":5479,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1453078087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":1453078087},{"type":"v1","addr":"172.21.15.49:6835","nonce":1453078087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-18T03:40:44.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.461+0000 7f618ebff700 1 -- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f616c06ad70 msgr2=0x7f616c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.461+0000 7f618ebff700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f616c06ad70 0x7f616c06d220 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f617c006fd0 tx=0x7f617c008040 comp rx=0 tx=0).stop 2023-07-18T03:40:44.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.461+0000 7f618ebff700 1 -- 172.21.15.49:0/1310833732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61880b5d90 msgr2=0x7f6188154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.461+0000 7f618ebff700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61880b5d90 0x7f6188154bb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f617400c450 tx=0x7f617400c760 comp rx=0 tx=0).stop 2023-07-18T03:40:44.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.461+0000 7f618ebff700 1 -- 172.21.15.49:0/1310833732 shutdown_connections 2023-07-18T03:40:44.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.461+0000 7f618ebff700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61880d0b10 0x7f6188150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.461+0000 7f618ebff700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61880b4510 0x7f61880c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.461+0000 7f618ebff700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f616c06ad70 0x7f616c06d220 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.461+0000 7f618ebff700 1 --2- 172.21.15.49:0/1310833732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61880b5d90 0x7f6188154bb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.461+0000 7f618ebff700 1 -- 172.21.15.49:0/1310833732 >> 172.21.15.49:0/1310833732 conn(0x7f618809c1d0 msgr2=0x7f61880cf180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:44.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.461+0000 7f618ebff700 1 -- 172.21.15.49:0/1310833732 shutdown_connections 2023-07-18T03:40:44.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.461+0000 7f618ebff700 1 -- 172.21.15.49:0/1310833732 wait complete. 2023-07-18T03:40:44.457 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2023-07-18T03:40:44.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:44.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.684+0000 7f05e83ff700 1 Processor -- start 2023-07-18T03:40:44.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.684+0000 7f05e83ff700 1 -- start start 2023-07-18T03:40:44.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.685+0000 7f05e83ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05e00d1220 0x7f05e00b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.685+0000 7f05e83ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f05e00b1e00 0x7f05e00afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.685+0000 7f05e83ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e00d2ce0 con 0x7f05e015c950 2023-07-18T03:40:44.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.685+0000 7f05e83ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e00b3350 con 0x7f05e00b1e00 2023-07-18T03:40:44.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.685+0000 7f05e83ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00b3630 con 0x7f05e00d1220 2023-07-18T03:40:44.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.686+0000 7f05e619b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f05e015c950 0x7f05e00d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:35350/0 (socket says 172.21.15.49:35350) 2023-07-18T03:40:44.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.686+0000 7f05e619b700 1 -- 172.21.15.49:0/1401027246 learned_addr learned my addr 172.21.15.49:0/1401027246 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:44.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.686+0000 7f05e599a700 1 --2- 172.21.15.49:0/1401027246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05e00d1220 0x7f05e00b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.686+0000 7f05e699c700 1 --2- 172.21.15.49:0/1401027246 >> v2:172.21.15.107:3300/0 conn(0x7f05e00b1e00 0x7f05e00afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.686+0000 7f05e599a700 1 -- 172.21.15.49:0/1401027246 >> v2:172.21.15.107:3300/0 conn(0x7f05e00b1e00 msgr2=0x7f05e00afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:44.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.686+0000 7f05e599a700 1 --2- 172.21.15.49:0/1401027246 >> v2:172.21.15.107:3300/0 conn(0x7f05e00b1e00 0x7f05e00afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.686+0000 7f05e5199700 1 -- 172.21.15.49:0/1401027246 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883788310 0 0) 0x7f05e00d2ce0 con 0x7f05e015c950 2023-07-18T03:40:44.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.686+0000 7f05e599a700 1 -- 172.21.15.49:0/1401027246 >> v1:172.21.15.107:6789/0 conn(0x7f05e015c950 legacy=0x7f05e00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.686+0000 7f05e599a700 1 -- 172.21.15.49:0/1401027246 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00b3910 con 0x7f05e00d1220 2023-07-18T03:40:44.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.686+0000 7f05e699c700 1 --2- 172.21.15.49:0/1401027246 >> v2:172.21.15.107:3300/0 conn(0x7f05e00b1e00 0x7f05e00afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:44.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.687+0000 7f05e599a700 1 --2- 172.21.15.49:0/1401027246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05e00d1220 0x7f05e00b43f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f05d0009840 tx=0x7f05d0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21778f47dae1a6cb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:44.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.687+0000 7f05e5199700 1 -- 172.21.15.49:0/1401027246 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0011830 con 0x7f05e00d1220 2023-07-18T03:40:44.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.687+0000 7f05e5199700 1 -- 172.21.15.49:0/1401027246 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d0011990 con 0x7f05e00d1220 2023-07-18T03:40:44.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.687+0000 7f05e5199700 1 -- 172.21.15.49:0/1401027246 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0011d20 con 0x7f05e00d1220 2023-07-18T03:40:44.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.688+0000 7f05e83ff700 1 -- 172.21.15.49:0/1401027246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05e00d1220 msgr2=0x7f05e00b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.688+0000 7f05e83ff700 1 --2- 172.21.15.49:0/1401027246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05e00d1220 0x7f05e00b43f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f05d0009840 tx=0x7f05d0009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:44.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.688+0000 7f05e83ff700 1 -- 172.21.15.49:0/1401027246 shutdown_connections 2023-07-18T03:40:44.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.688+0000 7f05e83ff700 1 --2- 172.21.15.49:0/1401027246 >> v2:172.21.15.107:3300/0 conn(0x7f05e00b1e00 0x7f05e00afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.688+0000 7f05e83ff700 1 --2- 172.21.15.49:0/1401027246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05e00d1220 0x7f05e00b43f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.688+0000 7f05e83ff700 1 -- 172.21.15.49:0/1401027246 >> 172.21.15.49:0/1401027246 conn(0x7f05e009c1d0 msgr2=0x7f05e00d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:44.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.688+0000 7f05e83ff700 1 -- 172.21.15.49:0/1401027246 shutdown_connections 2023-07-18T03:40:44.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.688+0000 7f05e83ff700 1 -- 172.21.15.49:0/1401027246 wait complete. 2023-07-18T03:40:44.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.690+0000 7f05e83ff700 1 Processor -- start 2023-07-18T03:40:44.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.690+0000 7f05e83ff700 1 -- start start 2023-07-18T03:40:44.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.691+0000 7f05e83ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05e00b1e00 0x7f05e00cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.691+0000 7f05e83ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05e00d1220 0x7f05e0151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.691+0000 7f05e83ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05e015c950 0x7f05e01522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.691+0000 7f05e83ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00c6ae0 con 0x7f05e00d1220 2023-07-18T03:40:44.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.691+0000 7f05e83ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00ce260 con 0x7f05e00b1e00 2023-07-18T03:40:44.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.691+0000 7f05e83ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00ce540 con 0x7f05e015c950 2023-07-18T03:40:44.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.691+0000 7f05e599a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05e00d1220 0x7f05e0151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.691+0000 7f05e619b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05e00b1e00 0x7f05e00cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.691+0000 7f05e599a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05e00d1220 0x7f05e0151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35568/0 (socket says 172.21.15.49:35568) 2023-07-18T03:40:44.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.691+0000 7f05e599a700 1 -- 172.21.15.49:0/2756875608 learned_addr learned my addr 172.21.15.49:0/2756875608 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:44.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.691+0000 7f05e699c700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05e015c950 0x7f05e01522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.692+0000 7f05e599a700 1 -- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05e015c950 msgr2=0x7f05e01522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.692+0000 7f05e599a700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05e015c950 0x7f05e01522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.692+0000 7f05e599a700 1 -- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05e00b1e00 msgr2=0x7f05e00cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.692+0000 7f05e599a700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05e00b1e00 0x7f05e00cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.692+0000 7f05e599a700 1 -- 172.21.15.49:0/2756875608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d8009a70 con 0x7f05e00d1220 2023-07-18T03:40:44.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.692+0000 7f05e599a700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05e00d1220 0x7f05e0151da0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f05d0009fd0 tx=0x7f05d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:44.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.692+0000 7f05d6ffd700 1 -- 172.21.15.49:0/2756875608 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d00045d0 con 0x7f05e00d1220 2023-07-18T03:40:44.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.692+0000 7f05d6ffd700 1 -- 172.21.15.49:0/2756875608 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d001f070 con 0x7f05e00d1220 2023-07-18T03:40:44.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.692+0000 7f05d6ffd700 1 -- 172.21.15.49:0/2756875608 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d00038a0 con 0x7f05e00d1220 2023-07-18T03:40:44.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.692+0000 7f05e83ff700 1 -- 172.21.15.49:0/2756875608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d00094a0 con 0x7f05e00d1220 2023-07-18T03:40:44.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.693+0000 7f05e83ff700 1 -- 172.21.15.49:0/2756875608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e00ceb00 con 0x7f05e00d1220 2023-07-18T03:40:44.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.694+0000 7f05e83ff700 1 -- 172.21.15.49:0/2756875608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e00c6c20 con 0x7f05e00d1220 2023-07-18T03:40:44.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.694+0000 7f05d6ffd700 1 -- 172.21.15.49:0/2756875608 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f05d0003a00 con 0x7f05e00d1220 2023-07-18T03:40:44.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.695+0000 7f05d6ffd700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f05c406ad70 0x7f05c406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.695+0000 7f05d6ffd700 1 -- 172.21.15.49:0/2756875608 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f05d0021070 con 0x7f05e00d1220 2023-07-18T03:40:44.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.696+0000 7f05e619b700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f05c406ad70 0x7f05c406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.697+0000 7f05e619b700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f05c406ad70 0x7f05c406d220 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f05dc000fb0 tx=0x7f05dc007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:44.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.700+0000 7f05d6ffd700 1 -- 172.21.15.49:0/2756875608 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f05e00c6c20 con 0x7f05e00d1220 2023-07-18T03:40:44.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.908+0000 7f05e83ff700 1 -- 172.21.15.49:0/2756875608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05e00c6c20 con 0x7f05e00d1220 2023-07-18T03:40:44.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.909+0000 7f05d6ffd700 1 -- 172.21.15.49:0/2756875608 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18257 (secure 0 0 0) 0x7f05e00c6c20 con 0x7f05e00d1220 2023-07-18T03:40:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:44.904 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":49,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:40:41.947761+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:40:38.792808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-18T03:40:39.438590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:44.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.912+0000 7f05e83ff700 1 -- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f05c406ad70 msgr2=0x7f05c406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.912+0000 7f05e83ff700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f05c406ad70 0x7f05c406d220 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f05dc000fb0 tx=0x7f05dc007ec0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.912+0000 7f05e83ff700 1 -- 172.21.15.49:0/2756875608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05e00d1220 msgr2=0x7f05e0151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.912+0000 7f05e83ff700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05e00d1220 0x7f05e0151da0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f05d0009fd0 tx=0x7f05d0004750 comp rx=0 tx=0).stop 2023-07-18T03:40:44.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.912+0000 7f05e83ff700 1 -- 172.21.15.49:0/2756875608 shutdown_connections 2023-07-18T03:40:44.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.912+0000 7f05e83ff700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05e015c950 0x7f05e01522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.912+0000 7f05e83ff700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05e00b1e00 0x7f05e00cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.912+0000 7f05e83ff700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f05c406ad70 0x7f05c406d220 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.912+0000 7f05e83ff700 1 --2- 172.21.15.49:0/2756875608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05e00d1220 0x7f05e0151da0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.912+0000 7f05e83ff700 1 -- 172.21.15.49:0/2756875608 >> 172.21.15.49:0/2756875608 conn(0x7f05e009c1d0 msgr2=0x7f05e00d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:44.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.912+0000 7f05e83ff700 1 -- 172.21.15.49:0/2756875608 shutdown_connections 2023-07-18T03:40:44.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:44.912+0000 7f05e83ff700 1 -- 172.21.15.49:0/2756875608 wait complete. 2023-07-18T03:40:44.925 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-18T03:40:44.926 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:40:45.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.138+0000 7f2e689ff700 1 Processor -- start 2023-07-18T03:40:45.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.138+0000 7f2e689ff700 1 -- start start 2023-07-18T03:40:45.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.139+0000 7f2e689ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e640b4510 0x7f2e640b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:45.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.139+0000 7f2e689ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2e640d0b10 0x7f2e640d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:45.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.139+0000 7f2e689ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e640b7f50 con 0x7f2e640b5d90 2023-07-18T03:40:45.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.139+0000 7f2e689ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e640b48e0 con 0x7f2e640d0b10 2023-07-18T03:40:45.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.139+0000 7f2e689ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640d14d0 con 0x7f2e640b4510 2023-07-18T03:40:45.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.139+0000 7f2e5bfff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2e640b5d90 0x7f2e640cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50460/0 (socket says 172.21.15.49:50460) 2023-07-18T03:40:45.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.139+0000 7f2e5bfff700 1 -- 172.21.15.49:0/1634091414 learned_addr learned my addr 172.21.15.49:0/1634091414 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:45.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.139+0000 7f2e62d9d700 1 --2- 172.21.15.49:0/1634091414 >> v2:172.21.15.49:3300/0 conn(0x7f2e640d0b10 0x7f2e640d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:45.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.139+0000 7f2e6259c700 1 --2- 172.21.15.49:0/1634091414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e640b4510 0x7f2e640b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:45.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.140+0000 7f2e61d9b700 1 -- 172.21.15.49:0/1634091414 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250921698 0 0) 0x7f2e640b7f50 con 0x7f2e640b5d90 2023-07-18T03:40:45.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.140+0000 7f2e61d9b700 1 -- 172.21.15.49:0/1634091414 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e500035b0 con 0x7f2e640b5d90 2023-07-18T03:40:45.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.140+0000 7f2e62d9d700 1 -- 172.21.15.49:0/1634091414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e640b4510 msgr2=0x7f2e640b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:45.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.140+0000 7f2e62d9d700 1 --2- 172.21.15.49:0/1634091414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e640b4510 0x7f2e640b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.140+0000 7f2e62d9d700 1 -- 172.21.15.49:0/1634091414 >> v1:172.21.15.49:6789/0 conn(0x7f2e640b5d90 legacy=0x7f2e640cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:45.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.140+0000 7f2e62d9d700 1 -- 172.21.15.49:0/1634091414 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e640cdca0 con 0x7f2e640d0b10 2023-07-18T03:40:45.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.140+0000 7f2e62d9d700 1 --2- 172.21.15.49:0/1634091414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e640d0b10 0x7f2e640d0f90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009d90 tx=0x7f2e5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acebc1faa06c0079 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.140+0000 7f2e61d9b700 1 -- 172.21.15.49:0/1634091414 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c00e070 con 0x7f2e640d0b10 2023-07-18T03:40:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.140+0000 7f2e61d9b700 1 -- 172.21.15.49:0/1634091414 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c00bb20 con 0x7f2e640d0b10 2023-07-18T03:40:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.140+0000 7f2e61d9b700 1 -- 172.21.15.49:0/1634091414 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c012430 con 0x7f2e640d0b10 2023-07-18T03:40:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.141+0000 7f2e689ff700 1 -- 172.21.15.49:0/1634091414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e640d0b10 msgr2=0x7f2e640d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.141+0000 7f2e689ff700 1 --2- 172.21.15.49:0/1634091414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e640d0b10 0x7f2e640d0f90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009d90 tx=0x7f2e5c00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.141+0000 7f2e689ff700 1 -- 172.21.15.49:0/1634091414 shutdown_connections 2023-07-18T03:40:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.141+0000 7f2e689ff700 1 --2- 172.21.15.49:0/1634091414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e640d0b10 0x7f2e640d0f90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.141+0000 7f2e689ff700 1 --2- 172.21.15.49:0/1634091414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e640b4510 0x7f2e640b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.141+0000 7f2e689ff700 1 -- 172.21.15.49:0/1634091414 >> 172.21.15.49:0/1634091414 conn(0x7f2e6409c1d0 msgr2=0x7f2e640d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.141+0000 7f2e689ff700 1 -- 172.21.15.49:0/1634091414 shutdown_connections 2023-07-18T03:40:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.141+0000 7f2e689ff700 1 -- 172.21.15.49:0/1634091414 wait complete. 2023-07-18T03:40:45.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.143+0000 7f2e689ff700 1 Processor -- start 2023-07-18T03:40:45.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.143+0000 7f2e689ff700 1 -- start start 2023-07-18T03:40:45.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.144+0000 7f2e689ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e640b4510 0x7f2e64154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:45.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.144+0000 7f2e689ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e640b5d90 0x7f2e64153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:45.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.144+0000 7f2e689ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e640d0b10 0x7f2e641e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:45.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.144+0000 7f2e689ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64155740 con 0x7f2e640b4510 2023-07-18T03:40:45.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.144+0000 7f2e689ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64154bc0 con 0x7f2e640b5d90 2023-07-18T03:40:45.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.144+0000 7f2e689ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64154ea0 con 0x7f2e640d0b10 2023-07-18T03:40:45.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.144+0000 7f2e6259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e640b4510 0x7f2e64154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:45.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.145+0000 7f2e6259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e640b4510 0x7f2e64154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35584/0 (socket says 172.21.15.49:35584) 2023-07-18T03:40:45.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.145+0000 7f2e6259c700 1 -- 172.21.15.49:0/3898872975 learned_addr learned my addr 172.21.15.49:0/3898872975 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:45.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.145+0000 7f2e5bfff700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e640b5d90 0x7f2e64153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:45.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.145+0000 7f2e62d9d700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e640d0b10 0x7f2e641e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:45.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.145+0000 7f2e6259c700 1 -- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e640d0b10 msgr2=0x7f2e641e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:45.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.145+0000 7f2e6259c700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e640d0b10 0x7f2e641e5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.145+0000 7f2e6259c700 1 -- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e640b5d90 msgr2=0x7f2e64153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:45.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.145+0000 7f2e6259c700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e640b5d90 0x7f2e64153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.145+0000 7f2e6259c700 1 -- 172.21.15.49:0/3898872975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c009a70 con 0x7f2e640b4510 2023-07-18T03:40:45.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.145+0000 7f2e6259c700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e640b4510 0x7f2e64154370 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400e900 tx=0x7f2e5400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:45.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.146+0000 7f2e5affd700 1 -- 172.21.15.49:0/3898872975 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54013680 con 0x7f2e640b4510 2023-07-18T03:40:45.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.146+0000 7f2e5affd700 1 -- 172.21.15.49:0/3898872975 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e540137e0 con 0x7f2e640b4510 2023-07-18T03:40:45.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.146+0000 7f2e5affd700 1 -- 172.21.15.49:0/3898872975 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54013b50 con 0x7f2e640b4510 2023-07-18T03:40:45.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.146+0000 7f2e689ff700 1 -- 172.21.15.49:0/3898872975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e64155180 con 0x7f2e640b4510 2023-07-18T03:40:45.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.146+0000 7f2e689ff700 1 -- 172.21.15.49:0/3898872975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e641548b0 con 0x7f2e640b4510 2023-07-18T03:40:45.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.147+0000 7f2e689ff700 1 -- 172.21.15.49:0/3898872975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6404e280 con 0x7f2e640b4510 2023-07-18T03:40:45.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.148+0000 7f2e5affd700 1 -- 172.21.15.49:0/3898872975 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f2e54013cb0 con 0x7f2e640b4510 2023-07-18T03:40:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.149+0000 7f2e5affd700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2e4406ad70 0x7f2e4406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.149+0000 7f2e5affd700 1 -- 172.21.15.49:0/3898872975 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e540897d0 con 0x7f2e640b4510 2023-07-18T03:40:45.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.149+0000 7f2e5bfff700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2e4406ad70 0x7f2e4406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.150+0000 7f2e5bfff700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2e4406ad70 0x7f2e4406d220 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2e640c6680 tx=0x7f2e4c007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:45.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.153+0000 7f2e5affd700 1 -- 172.21.15.49:0/3898872975 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2e54054820 con 0x7f2e640b4510 2023-07-18T03:40:45.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.361+0000 7f2e689ff700 1 -- 172.21.15.49:0/3898872975 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e640b7620 con 0x7f2e4406ad70 2023-07-18T03:40:45.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.373+0000 7f2e5affd700 1 -- 172.21.15.49:0/3898872975 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311762 (secure 0 0 0) 0x7f2e4c0058e0 con 0x7f2e4406ad70 2023-07-18T03:40:45.371 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:45.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.377+0000 7f2e689ff700 1 -- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2e4406ad70 msgr2=0x7f2e4406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:45.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.377+0000 7f2e689ff700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2e4406ad70 0x7f2e4406d220 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2e640c6680 tx=0x7f2e4c007dc0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.377+0000 7f2e689ff700 1 -- 172.21.15.49:0/3898872975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e640b4510 msgr2=0x7f2e64154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:45.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.377+0000 7f2e689ff700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e640b4510 0x7f2e64154370 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400e900 tx=0x7f2e5400ec10 comp rx=0 tx=0).stop 2023-07-18T03:40:45.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.377+0000 7f2e689ff700 1 -- 172.21.15.49:0/3898872975 shutdown_connections 2023-07-18T03:40:45.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.377+0000 7f2e689ff700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e640d0b10 0x7f2e641e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.377+0000 7f2e689ff700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2e4406ad70 0x7f2e4406d220 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.377+0000 7f2e689ff700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e640b4510 0x7f2e64154370 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.377+0000 7f2e689ff700 1 --2- 172.21.15.49:0/3898872975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e640b5d90 0x7f2e64153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.377+0000 7f2e689ff700 1 -- 172.21.15.49:0/3898872975 >> 172.21.15.49:0/3898872975 conn(0x7f2e6409c1d0 msgr2=0x7f2e640cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:45.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.377+0000 7f2e689ff700 1 -- 172.21.15.49:0/3898872975 shutdown_connections 2023-07-18T03:40:45.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.377+0000 7f2e689ff700 1 -- 172.21.15.49:0/3898872975 wait complete. 2023-07-18T03:40:45.378 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:40:45.390 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2023-07-18T03:40:44.127550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237636,"kb_used_data":5636,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754737084,"statfs":{"total":773094113280,"available":772850774016,"internally_reserved":0,"allocated":5771264,"data_stored":3140325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.203371"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955090+0000","last_change":"2023-07-18T03:40:40.961987+0000","last_active":"2023-07-18T03:40:41.955090+0000","last_peered":"2023-07-18T03:40:41.955090+0000","last_clean":"2023-07-18T03:40:41.955090+0000","last_became_active":"2023-07-18T03:40:40.961699+0000","last_became_peered":"2023-07-18T03:40:40.961699+0000","last_unstale":"2023-07-18T03:40:41.955090+0000","last_undegraded":"2023-07-18T03:40:41.955090+0000","last_fullsized":"2023-07-18T03:40:41.955090+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:12:05.109987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175566+0000","last_change":"2023-07-18T03:40:40.962694+0000","last_active":"2023-07-18T03:40:42.175566+0000","last_peered":"2023-07-18T03:40:42.175566+0000","last_clean":"2023-07-18T03:40:42.175566+0000","last_became_active":"2023-07-18T03:40:40.962429+0000","last_became_peered":"2023-07-18T03:40:40.962429+0000","last_unstale":"2023-07-18T03:40:42.175566+0000","last_undegraded":"2023-07-18T03:40:42.175566+0000","last_fullsized":"2023-07-18T03:40:42.175566+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:39:12.540003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175659+0000","last_change":"2023-07-18T03:40:40.973776+0000","last_active":"2023-07-18T03:40:42.175659+0000","last_peered":"2023-07-18T03:40:42.175659+0000","last_clean":"2023-07-18T03:40:42.175659+0000","last_became_active":"2023-07-18T03:40:40.973451+0000","last_became_peered":"2023-07-18T03:40:40.973451+0000","last_unstale":"2023-07-18T03:40:42.175659+0000","last_undegraded":"2023-07-18T03:40:42.175659+0000","last_fullsized":"2023-07-18T03:40:42.175659+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:20:53.697417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175815+0000","last_change":"2023-07-18T03:40:40.966335+0000","last_active":"2023-07-18T03:40:42.175815+0000","last_peered":"2023-07-18T03:40:42.175815+0000","last_clean":"2023-07-18T03:40:42.175815+0000","last_became_active":"2023-07-18T03:40:40.966041+0000","last_became_peered":"2023-07-18T03:40:40.966041+0000","last_unstale":"2023-07-18T03:40:42.175815+0000","last_undegraded":"2023-07-18T03:40:42.175815+0000","last_fullsized":"2023-07-18T03:40:42.175815+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:05.139386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958766+0000","last_change":"2023-07-18T03:40:40.949371+0000","last_active":"2023-07-18T03:40:41.958766+0000","last_peered":"2023-07-18T03:40:41.958766+0000","last_clean":"2023-07-18T03:40:41.958766+0000","last_became_active":"2023-07-18T03:40:40.948989+0000","last_became_peered":"2023-07-18T03:40:40.948989+0000","last_unstale":"2023-07-18T03:40:41.958766+0000","last_undegraded":"2023-07-18T03:40:41.958766+0000","last_fullsized":"2023-07-18T03:40:41.958766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:16:23.107948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955132+0000","last_change":"2023-07-18T03:40:40.957623+0000","last_active":"2023-07-18T03:40:41.955132+0000","last_peered":"2023-07-18T03:40:41.955132+0000","last_clean":"2023-07-18T03:40:41.955132+0000","last_became_active":"2023-07-18T03:40:40.957363+0000","last_became_peered":"2023-07-18T03:40:40.957363+0000","last_unstale":"2023-07-18T03:40:41.955132+0000","last_undegraded":"2023-07-18T03:40:41.955132+0000","last_fullsized":"2023-07-18T03:40:41.955132+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:23:27.311259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175474+0000","last_change":"2023-07-18T03:40:40.966144+0000","last_active":"2023-07-18T03:40:42.175474+0000","last_peered":"2023-07-18T03:40:42.175474+0000","last_clean":"2023-07-18T03:40:42.175474+0000","last_became_active":"2023-07-18T03:40:40.965812+0000","last_became_peered":"2023-07-18T03:40:40.965812+0000","last_unstale":"2023-07-18T03:40:42.175474+0000","last_undegraded":"2023-07-18T03:40:42.175474+0000","last_fullsized":"2023-07-18T03:40:42.175474+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:25:54.877763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.978588+0000","last_change":"2023-07-18T03:40:40.971844+0000","last_active":"2023-07-18T03:40:40.978588+0000","last_peered":"2023-07-18T03:40:40.978588+0000","last_clean":"2023-07-18T03:40:40.978588+0000","last_became_active":"2023-07-18T03:40:40.971231+0000","last_became_peered":"2023-07-18T03:40:40.971231+0000","last_unstale":"2023-07-18T03:40:40.978588+0000","last_undegraded":"2023-07-18T03:40:40.978588+0000","last_fullsized":"2023-07-18T03:40:40.978588+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:53:38.313583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.982771+0000","last_change":"2023-07-18T03:40:40.977579+0000","last_active":"2023-07-18T03:40:40.982771+0000","last_peered":"2023-07-18T03:40:40.982771+0000","last_clean":"2023-07-18T03:40:40.982771+0000","last_became_active":"2023-07-18T03:40:40.977307+0000","last_became_peered":"2023-07-18T03:40:40.977307+0000","last_unstale":"2023-07-18T03:40:40.982771+0000","last_undegraded":"2023-07-18T03:40:40.982771+0000","last_fullsized":"2023-07-18T03:40:40.982771+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:30:32.044631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955561+0000","last_change":"2023-07-18T03:40:40.952693+0000","last_active":"2023-07-18T03:40:41.955561+0000","last_peered":"2023-07-18T03:40:41.955561+0000","last_clean":"2023-07-18T03:40:41.955561+0000","last_became_active":"2023-07-18T03:40:40.952384+0000","last_became_peered":"2023-07-18T03:40:40.952384+0000","last_unstale":"2023-07-18T03:40:41.955561+0000","last_undegraded":"2023-07-18T03:40:41.955561+0000","last_fullsized":"2023-07-18T03:40:41.955561+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:38:48.622343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.956398+0000","last_change":"2023-07-18T03:40:40.962995+0000","last_active":"2023-07-18T03:40:41.956398+0000","last_peered":"2023-07-18T03:40:41.956398+0000","last_clean":"2023-07-18T03:40:41.956398+0000","last_became_active":"2023-07-18T03:40:40.962702+0000","last_became_peered":"2023-07-18T03:40:40.962702+0000","last_unstale":"2023-07-18T03:40:41.956398+0000","last_undegraded":"2023-07-18T03:40:41.956398+0000","last_fullsized":"2023-07-18T03:40:41.956398+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:03:23.647349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.981591+0000","last_change":"2023-07-18T03:40:40.974913+0000","last_active":"2023-07-18T03:40:40.981591+0000","last_peered":"2023-07-18T03:40:40.981591+0000","last_clean":"2023-07-18T03:40:40.981591+0000","last_became_active":"2023-07-18T03:40:40.974618+0000","last_became_peered":"2023-07-18T03:40:40.974618+0000","last_unstale":"2023-07-18T03:40:40.981591+0000","last_undegraded":"2023-07-18T03:40:40.981591+0000","last_fullsized":"2023-07-18T03:40:40.981591+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:45:04.720217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.954950+0000","last_change":"2023-07-18T03:40:40.952736+0000","last_active":"2023-07-18T03:40:41.954950+0000","last_peered":"2023-07-18T03:40:41.954950+0000","last_clean":"2023-07-18T03:40:41.954950+0000","last_became_active":"2023-07-18T03:40:40.952398+0000","last_became_peered":"2023-07-18T03:40:40.952398+0000","last_unstale":"2023-07-18T03:40:41.954950+0000","last_undegraded":"2023-07-18T03:40:41.954950+0000","last_fullsized":"2023-07-18T03:40:41.954950+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:22:06.699892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175176+0000","last_change":"2023-07-18T03:40:40.973447+0000","last_active":"2023-07-18T03:40:42.175176+0000","last_peered":"2023-07-18T03:40:42.175176+0000","last_clean":"2023-07-18T03:40:42.175176+0000","last_became_active":"2023-07-18T03:40:40.973162+0000","last_became_peered":"2023-07-18T03:40:40.973162+0000","last_unstale":"2023-07-18T03:40:42.175176+0000","last_undegraded":"2023-07-18T03:40:42.175176+0000","last_fullsized":"2023-07-18T03:40:42.175176+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:30:25.988049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175662+0000","last_change":"2023-07-18T03:40:40.957906+0000","last_active":"2023-07-18T03:40:42.175662+0000","last_peered":"2023-07-18T03:40:42.175662+0000","last_clean":"2023-07-18T03:40:42.175662+0000","last_became_active":"2023-07-18T03:40:40.957647+0000","last_became_peered":"2023-07-18T03:40:40.957647+0000","last_unstale":"2023-07-18T03:40:42.175662+0000","last_undegraded":"2023-07-18T03:40:42.175662+0000","last_fullsized":"2023-07-18T03:40:42.175662+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:08:06.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.954809+0000","last_change":"2023-07-18T03:40:40.954836+0000","last_active":"2023-07-18T03:40:41.954809+0000","last_peered":"2023-07-18T03:40:41.954809+0000","last_clean":"2023-07-18T03:40:41.954809+0000","last_became_active":"2023-07-18T03:40:40.954559+0000","last_became_peered":"2023-07-18T03:40:40.954559+0000","last_unstale":"2023-07-18T03:40:41.954809+0000","last_undegraded":"2023-07-18T03:40:41.954809+0000","last_fullsized":"2023-07-18T03:40:41.954809+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:43:12.774955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175821+0000","last_change":"2023-07-18T03:40:40.970811+0000","last_active":"2023-07-18T03:40:42.175821+0000","last_peered":"2023-07-18T03:40:42.175821+0000","last_clean":"2023-07-18T03:40:42.175821+0000","last_became_active":"2023-07-18T03:40:40.969518+0000","last_became_peered":"2023-07-18T03:40:40.969518+0000","last_unstale":"2023-07-18T03:40:42.175821+0000","last_undegraded":"2023-07-18T03:40:42.175821+0000","last_fullsized":"2023-07-18T03:40:42.175821+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:16.011690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175900+0000","last_change":"2023-07-18T03:40:40.962259+0000","last_active":"2023-07-18T03:40:42.175900+0000","last_peered":"2023-07-18T03:40:42.175900+0000","last_clean":"2023-07-18T03:40:42.175900+0000","last_became_active":"2023-07-18T03:40:40.961970+0000","last_became_peered":"2023-07-18T03:40:40.961970+0000","last_unstale":"2023-07-18T03:40:42.175900+0000","last_undegraded":"2023-07-18T03:40:42.175900+0000","last_fullsized":"2023-07-18T03:40:42.175900+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:52:23.122594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175955+0000","last_change":"2023-07-18T03:40:40.971561+0000","last_active":"2023-07-18T03:40:42.175955+0000","last_peered":"2023-07-18T03:40:42.175955+0000","last_clean":"2023-07-18T03:40:42.175955+0000","last_became_active":"2023-07-18T03:40:40.969715+0000","last_became_peered":"2023-07-18T03:40:40.969715+0000","last_unstale":"2023-07-18T03:40:42.175955+0000","last_undegraded":"2023-07-18T03:40:42.175955+0000","last_fullsized":"2023-07-18T03:40:42.175955+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:38:26.570999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175777+0000","last_change":"2023-07-18T03:40:40.960847+0000","last_active":"2023-07-18T03:40:42.175777+0000","last_peered":"2023-07-18T03:40:42.175777+0000","last_clean":"2023-07-18T03:40:42.175777+0000","last_became_active":"2023-07-18T03:40:40.960592+0000","last_became_peered":"2023-07-18T03:40:40.960592+0000","last_unstale":"2023-07-18T03:40:42.175777+0000","last_undegraded":"2023-07-18T03:40:42.175777+0000","last_fullsized":"2023-07-18T03:40:42.175777+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:22:06.699892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175853+0000","last_change":"2023-07-18T03:40:40.958959+0000","last_active":"2023-07-18T03:40:42.175853+0000","last_peered":"2023-07-18T03:40:42.175853+0000","last_clean":"2023-07-18T03:40:42.175853+0000","last_became_active":"2023-07-18T03:40:40.958659+0000","last_became_peered":"2023-07-18T03:40:40.958659+0000","last_unstale":"2023-07-18T03:40:42.175853+0000","last_undegraded":"2023-07-18T03:40:42.175853+0000","last_fullsized":"2023-07-18T03:40:42.175853+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:20:30.773061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.977967+0000","last_change":"2023-07-18T03:40:40.970419+0000","last_active":"2023-07-18T03:40:40.977967+0000","last_peered":"2023-07-18T03:40:40.977967+0000","last_clean":"2023-07-18T03:40:40.977967+0000","last_became_active":"2023-07-18T03:40:40.970138+0000","last_became_peered":"2023-07-18T03:40:40.970138+0000","last_unstale":"2023-07-18T03:40:40.977967+0000","last_undegraded":"2023-07-18T03:40:40.977967+0000","last_fullsized":"2023-07-18T03:40:40.977967+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:21.834512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.956022+0000","last_change":"2023-07-18T03:40:40.961886+0000","last_active":"2023-07-18T03:40:41.956022+0000","last_peered":"2023-07-18T03:40:41.956022+0000","last_clean":"2023-07-18T03:40:41.956022+0000","last_became_active":"2023-07-18T03:40:40.961593+0000","last_became_peered":"2023-07-18T03:40:40.961593+0000","last_unstale":"2023-07-18T03:40:41.956022+0000","last_undegraded":"2023-07-18T03:40:41.956022+0000","last_fullsized":"2023-07-18T03:40:41.956022+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:03:49.133039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175046+0000","last_change":"2023-07-18T03:40:40.969148+0000","last_active":"2023-07-18T03:40:42.175046+0000","last_peered":"2023-07-18T03:40:42.175046+0000","last_clean":"2023-07-18T03:40:42.175046+0000","last_became_active":"2023-07-18T03:40:40.968878+0000","last_became_peered":"2023-07-18T03:40:40.968878+0000","last_unstale":"2023-07-18T03:40:42.175046+0000","last_undegraded":"2023-07-18T03:40:42.175046+0000","last_fullsized":"2023-07-18T03:40:42.175046+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:17:22.645893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175774+0000","last_change":"2023-07-18T03:40:40.980878+0000","last_active":"2023-07-18T03:40:42.175774+0000","last_peered":"2023-07-18T03:40:42.175774+0000","last_clean":"2023-07-18T03:40:42.175774+0000","last_became_active":"2023-07-18T03:40:40.980407+0000","last_became_peered":"2023-07-18T03:40:40.980407+0000","last_unstale":"2023-07-18T03:40:42.175774+0000","last_undegraded":"2023-07-18T03:40:42.175774+0000","last_fullsized":"2023-07-18T03:40:42.175774+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:45:07.583653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.981293+0000","last_change":"2023-07-18T03:40:40.974638+0000","last_active":"2023-07-18T03:40:40.981293+0000","last_peered":"2023-07-18T03:40:40.981293+0000","last_clean":"2023-07-18T03:40:40.981293+0000","last_became_active":"2023-07-18T03:40:40.974268+0000","last_became_peered":"2023-07-18T03:40:40.974268+0000","last_unstale":"2023-07-18T03:40:40.981293+0000","last_undegraded":"2023-07-18T03:40:40.981293+0000","last_fullsized":"2023-07-18T03:40:40.981293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:03:17.053200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955774+0000","last_change":"2023-07-18T03:40:40.973345+0000","last_active":"2023-07-18T03:40:41.955774+0000","last_peered":"2023-07-18T03:40:41.955774+0000","last_clean":"2023-07-18T03:40:41.955774+0000","last_became_active":"2023-07-18T03:40:40.973095+0000","last_became_peered":"2023-07-18T03:40:40.973095+0000","last_unstale":"2023-07-18T03:40:41.955774+0000","last_undegraded":"2023-07-18T03:40:41.955774+0000","last_fullsized":"2023-07-18T03:40:41.955774+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:23:20.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175956+0000","last_change":"2023-07-18T03:40:40.968398+0000","last_active":"2023-07-18T03:40:42.175956+0000","last_peered":"2023-07-18T03:40:42.175956+0000","last_clean":"2023-07-18T03:40:42.175956+0000","last_became_active":"2023-07-18T03:40:40.968140+0000","last_became_peered":"2023-07-18T03:40:40.968140+0000","last_unstale":"2023-07-18T03:40:42.175956+0000","last_undegraded":"2023-07-18T03:40:42.175956+0000","last_fullsized":"2023-07-18T03:40:42.175956+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:46:38.287766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175569+0000","last_change":"2023-07-18T03:40:40.964061+0000","last_active":"2023-07-18T03:40:42.175569+0000","last_peered":"2023-07-18T03:40:42.175569+0000","last_clean":"2023-07-18T03:40:42.175569+0000","last_became_active":"2023-07-18T03:40:40.963746+0000","last_became_peered":"2023-07-18T03:40:40.963746+0000","last_unstale":"2023-07-18T03:40:42.175569+0000","last_undegraded":"2023-07-18T03:40:42.175569+0000","last_fullsized":"2023-07-18T03:40:42.175569+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:00:25.826683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176160+0000","last_change":"2023-07-18T03:40:40.967820+0000","last_active":"2023-07-18T03:40:42.176160+0000","last_peered":"2023-07-18T03:40:42.176160+0000","last_clean":"2023-07-18T03:40:42.176160+0000","last_became_active":"2023-07-18T03:40:40.967515+0000","last_became_peered":"2023-07-18T03:40:40.967515+0000","last_unstale":"2023-07-18T03:40:42.176160+0000","last_undegraded":"2023-07-18T03:40:42.176160+0000","last_fullsized":"2023-07-18T03:40:42.176160+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:19.679560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175341+0000","last_change":"2023-07-18T03:40:40.973531+0000","last_active":"2023-07-18T03:40:42.175341+0000","last_peered":"2023-07-18T03:40:42.175341+0000","last_clean":"2023-07-18T03:40:42.175341+0000","last_became_active":"2023-07-18T03:40:40.973212+0000","last_became_peered":"2023-07-18T03:40:40.973212+0000","last_unstale":"2023-07-18T03:40:42.175341+0000","last_undegraded":"2023-07-18T03:40:42.175341+0000","last_fullsized":"2023-07-18T03:40:42.175341+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:39.170107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175173+0000","last_change":"2023-07-18T03:40:40.970303+0000","last_active":"2023-07-18T03:40:42.175173+0000","last_peered":"2023-07-18T03:40:42.175173+0000","last_clean":"2023-07-18T03:40:42.175173+0000","last_became_active":"2023-07-18T03:40:40.970041+0000","last_became_peered":"2023-07-18T03:40:40.970041+0000","last_unstale":"2023-07-18T03:40:42.175173+0000","last_undegraded":"2023-07-18T03:40:42.175173+0000","last_fullsized":"2023-07-18T03:40:42.175173+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:16:07.234893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175221+0000","last_change":"2023-07-18T03:40:40.962657+0000","last_active":"2023-07-18T03:40:42.175221+0000","last_peered":"2023-07-18T03:40:42.175221+0000","last_clean":"2023-07-18T03:40:42.175221+0000","last_became_active":"2023-07-18T03:40:40.962363+0000","last_became_peered":"2023-07-18T03:40:40.962363+0000","last_unstale":"2023-07-18T03:40:42.175221+0000","last_undegraded":"2023-07-18T03:40:42.175221+0000","last_fullsized":"2023-07-18T03:40:42.175221+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:16:10.691858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176164+0000","last_change":"2023-07-18T03:40:39.945226+0000","last_active":"2023-07-18T03:40:42.176164+0000","last_peered":"2023-07-18T03:40:42.176164+0000","last_clean":"2023-07-18T03:40:42.176164+0000","last_became_active":"2023-07-18T03:40:39.944923+0000","last_became_peered":"2023-07-18T03:40:39.944923+0000","last_unstale":"2023-07-18T03:40:42.176164+0000","last_undegraded":"2023-07-18T03:40:42.176164+0000","last_fullsized":"2023-07-18T03:40:42.176164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:28:29.317447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176528+0000","last_change":"2023-07-18T03:40:40.965946+0000","last_active":"2023-07-18T03:40:42.176528+0000","last_peered":"2023-07-18T03:40:42.176528+0000","last_clean":"2023-07-18T03:40:42.176528+0000","last_became_active":"2023-07-18T03:40:40.965616+0000","last_became_peered":"2023-07-18T03:40:40.965616+0000","last_unstale":"2023-07-18T03:40:42.176528+0000","last_undegraded":"2023-07-18T03:40:42.176528+0000","last_fullsized":"2023-07-18T03:40:42.176528+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:20:47.935526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955256+0000","last_change":"2023-07-18T03:40:39.950318+0000","last_active":"2023-07-18T03:40:41.955256+0000","last_peered":"2023-07-18T03:40:41.955256+0000","last_clean":"2023-07-18T03:40:41.955256+0000","last_became_active":"2023-07-18T03:40:39.950056+0000","last_became_peered":"2023-07-18T03:40:39.950056+0000","last_unstale":"2023-07-18T03:40:41.955256+0000","last_undegraded":"2023-07-18T03:40:41.955256+0000","last_fullsized":"2023-07-18T03:40:41.955256+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:32:17.247797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175126+0000","last_change":"2023-07-18T03:40:40.958875+0000","last_active":"2023-07-18T03:40:42.175126+0000","last_peered":"2023-07-18T03:40:42.175126+0000","last_clean":"2023-07-18T03:40:42.175126+0000","last_became_active":"2023-07-18T03:40:40.958569+0000","last_became_peered":"2023-07-18T03:40:40.958569+0000","last_unstale":"2023-07-18T03:40:42.175126+0000","last_undegraded":"2023-07-18T03:40:42.175126+0000","last_fullsized":"2023-07-18T03:40:42.175126+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:18:40.983271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955893+0000","last_change":"2023-07-18T03:40:39.943162+0000","last_active":"2023-07-18T03:40:41.955893+0000","last_peered":"2023-07-18T03:40:41.955893+0000","last_clean":"2023-07-18T03:40:41.955893+0000","last_became_active":"2023-07-18T03:40:39.942873+0000","last_became_peered":"2023-07-18T03:40:39.942873+0000","last_unstale":"2023-07-18T03:40:41.955893+0000","last_undegraded":"2023-07-18T03:40:41.955893+0000","last_fullsized":"2023-07-18T03:40:41.955893+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:11:46.013861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.981199+0000","last_change":"2023-07-18T03:40:40.976222+0000","last_active":"2023-07-18T03:40:40.981199+0000","last_peered":"2023-07-18T03:40:40.981199+0000","last_clean":"2023-07-18T03:40:40.981199+0000","last_became_active":"2023-07-18T03:40:40.975964+0000","last_became_peered":"2023-07-18T03:40:40.975964+0000","last_unstale":"2023-07-18T03:40:40.981199+0000","last_undegraded":"2023-07-18T03:40:40.981199+0000","last_fullsized":"2023-07-18T03:40:40.981199+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:29:56.985236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176637+0000","last_change":"2023-07-18T03:40:39.952921+0000","last_active":"2023-07-18T03:40:42.176637+0000","last_peered":"2023-07-18T03:40:42.176637+0000","last_clean":"2023-07-18T03:40:42.176637+0000","last_became_active":"2023-07-18T03:40:39.952687+0000","last_became_peered":"2023-07-18T03:40:39.952687+0000","last_unstale":"2023-07-18T03:40:42.176637+0000","last_undegraded":"2023-07-18T03:40:42.176637+0000","last_fullsized":"2023-07-18T03:40:42.176637+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:04:51.980579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958827+0000","last_change":"2023-07-18T03:40:40.973478+0000","last_active":"2023-07-18T03:40:41.958827+0000","last_peered":"2023-07-18T03:40:41.958827+0000","last_clean":"2023-07-18T03:40:41.958827+0000","last_became_active":"2023-07-18T03:40:40.973208+0000","last_became_peered":"2023-07-18T03:40:40.973208+0000","last_unstale":"2023-07-18T03:40:41.958827+0000","last_undegraded":"2023-07-18T03:40:41.958827+0000","last_fullsized":"2023-07-18T03:40:41.958827+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:14:33.202960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175779+0000","last_change":"2023-07-18T03:40:39.954484+0000","last_active":"2023-07-18T03:40:42.175779+0000","last_peered":"2023-07-18T03:40:42.175779+0000","last_clean":"2023-07-18T03:40:42.175779+0000","last_became_active":"2023-07-18T03:40:39.954155+0000","last_became_peered":"2023-07-18T03:40:39.954155+0000","last_unstale":"2023-07-18T03:40:42.175779+0000","last_undegraded":"2023-07-18T03:40:42.175779+0000","last_fullsized":"2023-07-18T03:40:42.175779+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:48:06.694651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176597+0000","last_change":"2023-07-18T03:40:40.966808+0000","last_active":"2023-07-18T03:40:42.176597+0000","last_peered":"2023-07-18T03:40:42.176597+0000","last_clean":"2023-07-18T03:40:42.176597+0000","last_became_active":"2023-07-18T03:40:40.966529+0000","last_became_peered":"2023-07-18T03:40:40.966529+0000","last_unstale":"2023-07-18T03:40:42.176597+0000","last_undegraded":"2023-07-18T03:40:42.176597+0000","last_fullsized":"2023-07-18T03:40:42.176597+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:02:01.209598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958529+0000","last_change":"2023-07-18T03:40:39.945758+0000","last_active":"2023-07-18T03:40:41.958529+0000","last_peered":"2023-07-18T03:40:41.958529+0000","last_clean":"2023-07-18T03:40:41.958529+0000","last_became_active":"2023-07-18T03:40:39.945472+0000","last_became_peered":"2023-07-18T03:40:39.945472+0000","last_unstale":"2023-07-18T03:40:41.958529+0000","last_undegraded":"2023-07-18T03:40:41.958529+0000","last_fullsized":"2023-07-18T03:40:41.958529+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:20.448131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.981311+0000","last_change":"2023-07-18T03:40:40.974986+0000","last_active":"2023-07-18T03:40:40.981311+0000","last_peered":"2023-07-18T03:40:40.981311+0000","last_clean":"2023-07-18T03:40:40.981311+0000","last_became_active":"2023-07-18T03:40:40.974134+0000","last_became_peered":"2023-07-18T03:40:40.974134+0000","last_unstale":"2023-07-18T03:40:40.981311+0000","last_undegraded":"2023-07-18T03:40:40.981311+0000","last_fullsized":"2023-07-18T03:40:40.981311+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:05:22.268526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175314+0000","last_change":"2023-07-18T03:40:39.945758+0000","last_active":"2023-07-18T03:40:42.175314+0000","last_peered":"2023-07-18T03:40:42.175314+0000","last_clean":"2023-07-18T03:40:42.175314+0000","last_became_active":"2023-07-18T03:40:39.945260+0000","last_became_peered":"2023-07-18T03:40:39.945260+0000","last_unstale":"2023-07-18T03:40:42.175314+0000","last_undegraded":"2023-07-18T03:40:42.175314+0000","last_fullsized":"2023-07-18T03:40:42.175314+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:27:52.130590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175770+0000","last_change":"2023-07-18T03:40:40.973616+0000","last_active":"2023-07-18T03:40:42.175770+0000","last_peered":"2023-07-18T03:40:42.175770+0000","last_clean":"2023-07-18T03:40:42.175770+0000","last_became_active":"2023-07-18T03:40:40.973351+0000","last_became_peered":"2023-07-18T03:40:40.973351+0000","last_unstale":"2023-07-18T03:40:42.175770+0000","last_undegraded":"2023-07-18T03:40:42.175770+0000","last_fullsized":"2023-07-18T03:40:42.175770+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:47:58.326201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175113+0000","last_change":"2023-07-18T03:40:39.953906+0000","last_active":"2023-07-18T03:40:42.175113+0000","last_peered":"2023-07-18T03:40:42.175113+0000","last_clean":"2023-07-18T03:40:42.175113+0000","last_became_active":"2023-07-18T03:40:39.953591+0000","last_became_peered":"2023-07-18T03:40:39.953591+0000","last_unstale":"2023-07-18T03:40:42.175113+0000","last_undegraded":"2023-07-18T03:40:42.175113+0000","last_fullsized":"2023-07-18T03:40:42.175113+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:53:21.854085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958391+0000","last_change":"2023-07-18T03:40:40.965005+0000","last_active":"2023-07-18T03:40:41.958391+0000","last_peered":"2023-07-18T03:40:41.958391+0000","last_clean":"2023-07-18T03:40:41.958391+0000","last_became_active":"2023-07-18T03:40:40.964741+0000","last_became_peered":"2023-07-18T03:40:40.964741+0000","last_unstale":"2023-07-18T03:40:41.958391+0000","last_undegraded":"2023-07-18T03:40:41.958391+0000","last_fullsized":"2023-07-18T03:40:41.958391+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:09:36.480771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175841+0000","last_change":"2023-07-18T03:40:39.943379+0000","last_active":"2023-07-18T03:40:42.175841+0000","last_peered":"2023-07-18T03:40:42.175841+0000","last_clean":"2023-07-18T03:40:42.175841+0000","last_became_active":"2023-07-18T03:40:39.943077+0000","last_became_peered":"2023-07-18T03:40:39.943077+0000","last_unstale":"2023-07-18T03:40:42.175841+0000","last_undegraded":"2023-07-18T03:40:42.175841+0000","last_fullsized":"2023-07-18T03:40:42.175841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:10:23.919030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176263+0000","last_change":"2023-07-18T03:40:40.961277+0000","last_active":"2023-07-18T03:40:42.176263+0000","last_peered":"2023-07-18T03:40:42.176263+0000","last_clean":"2023-07-18T03:40:42.176263+0000","last_became_active":"2023-07-18T03:40:40.960859+0000","last_became_peered":"2023-07-18T03:40:40.960859+0000","last_unstale":"2023-07-18T03:40:42.176263+0000","last_undegraded":"2023-07-18T03:40:42.176263+0000","last_fullsized":"2023-07-18T03:40:42.176263+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:33:51.672942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958745+0000","last_change":"2023-07-18T03:40:39.956587+0000","last_active":"2023-07-18T03:40:41.958745+0000","last_peered":"2023-07-18T03:40:41.958745+0000","last_clean":"2023-07-18T03:40:41.958745+0000","last_became_active":"2023-07-18T03:40:39.956218+0000","last_became_peered":"2023-07-18T03:40:39.956218+0000","last_unstale":"2023-07-18T03:40:41.958745+0000","last_undegraded":"2023-07-18T03:40:41.958745+0000","last_fullsized":"2023-07-18T03:40:41.958745+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:43:27.074767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175313+0000","last_change":"2023-07-18T03:40:40.964462+0000","last_active":"2023-07-18T03:40:42.175313+0000","last_peered":"2023-07-18T03:40:42.175313+0000","last_clean":"2023-07-18T03:40:42.175313+0000","last_became_active":"2023-07-18T03:40:40.964234+0000","last_became_peered":"2023-07-18T03:40:40.964234+0000","last_unstale":"2023-07-18T03:40:42.175313+0000","last_undegraded":"2023-07-18T03:40:42.175313+0000","last_fullsized":"2023-07-18T03:40:42.175313+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:07:06.495446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175994+0000","last_change":"2023-07-18T03:40:39.944342+0000","last_active":"2023-07-18T03:40:42.175994+0000","last_peered":"2023-07-18T03:40:42.175994+0000","last_clean":"2023-07-18T03:40:42.175994+0000","last_became_active":"2023-07-18T03:40:39.944095+0000","last_became_peered":"2023-07-18T03:40:39.944095+0000","last_unstale":"2023-07-18T03:40:42.175994+0000","last_undegraded":"2023-07-18T03:40:42.175994+0000","last_fullsized":"2023-07-18T03:40:42.175994+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:50.816530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958857+0000","last_change":"2023-07-18T03:40:40.965591+0000","last_active":"2023-07-18T03:40:41.958857+0000","last_peered":"2023-07-18T03:40:41.958857+0000","last_clean":"2023-07-18T03:40:41.958857+0000","last_became_active":"2023-07-18T03:40:40.965324+0000","last_became_peered":"2023-07-18T03:40:40.965324+0000","last_unstale":"2023-07-18T03:40:41.958857+0000","last_undegraded":"2023-07-18T03:40:41.958857+0000","last_fullsized":"2023-07-18T03:40:41.958857+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:57:48.782958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.983554+0000","last_change":"2023-07-18T03:40:39.950059+0000","last_active":"2023-07-18T03:40:40.983554+0000","last_peered":"2023-07-18T03:40:40.983554+0000","last_clean":"2023-07-18T03:40:40.983554+0000","last_became_active":"2023-07-18T03:40:39.949463+0000","last_became_peered":"2023-07-18T03:40:39.949463+0000","last_unstale":"2023-07-18T03:40:40.983554+0000","last_undegraded":"2023-07-18T03:40:40.983554+0000","last_fullsized":"2023-07-18T03:40:40.983554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:45:37.491688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176450+0000","last_change":"2023-07-18T03:40:40.964104+0000","last_active":"2023-07-18T03:40:42.176450+0000","last_peered":"2023-07-18T03:40:42.176450+0000","last_clean":"2023-07-18T03:40:42.176450+0000","last_became_active":"2023-07-18T03:40:40.963764+0000","last_became_peered":"2023-07-18T03:40:40.963764+0000","last_unstale":"2023-07-18T03:40:42.176450+0000","last_undegraded":"2023-07-18T03:40:42.176450+0000","last_fullsized":"2023-07-18T03:40:42.176450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:22:05.598835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175869+0000","last_change":"2023-07-18T03:40:39.954159+0000","last_active":"2023-07-18T03:40:42.175869+0000","last_peered":"2023-07-18T03:40:42.175869+0000","last_clean":"2023-07-18T03:40:42.175869+0000","last_became_active":"2023-07-18T03:40:39.953864+0000","last_became_peered":"2023-07-18T03:40:39.953864+0000","last_unstale":"2023-07-18T03:40:42.175869+0000","last_undegraded":"2023-07-18T03:40:42.175869+0000","last_fullsized":"2023-07-18T03:40:42.175869+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:23:00.111799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175846+0000","last_change":"2023-07-18T03:40:40.971335+0000","last_active":"2023-07-18T03:40:42.175846+0000","last_peered":"2023-07-18T03:40:42.175846+0000","last_clean":"2023-07-18T03:40:42.175846+0000","last_became_active":"2023-07-18T03:40:40.969388+0000","last_became_peered":"2023-07-18T03:40:40.969388+0000","last_unstale":"2023-07-18T03:40:42.175846+0000","last_undegraded":"2023-07-18T03:40:42.175846+0000","last_fullsized":"2023-07-18T03:40:42.175846+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:50:13.047904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175350+0000","last_change":"2023-07-18T03:40:39.956747+0000","last_active":"2023-07-18T03:40:42.175350+0000","last_peered":"2023-07-18T03:40:42.175350+0000","last_clean":"2023-07-18T03:40:42.175350+0000","last_became_active":"2023-07-18T03:40:39.956458+0000","last_became_peered":"2023-07-18T03:40:39.956458+0000","last_unstale":"2023-07-18T03:40:42.175350+0000","last_undegraded":"2023-07-18T03:40:42.175350+0000","last_fullsized":"2023-07-18T03:40:42.175350+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:43:11.769264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.977943+0000","last_change":"2023-07-18T03:40:40.967376+0000","last_active":"2023-07-18T03:40:40.977943+0000","last_peered":"2023-07-18T03:40:40.977943+0000","last_clean":"2023-07-18T03:40:40.977943+0000","last_became_active":"2023-07-18T03:40:40.967073+0000","last_became_peered":"2023-07-18T03:40:40.967073+0000","last_unstale":"2023-07-18T03:40:40.977943+0000","last_undegraded":"2023-07-18T03:40:40.977943+0000","last_fullsized":"2023-07-18T03:40:40.977943+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:31.758392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175675+0000","last_change":"2023-07-18T03:40:39.943090+0000","last_active":"2023-07-18T03:40:42.175675+0000","last_peered":"2023-07-18T03:40:42.175675+0000","last_clean":"2023-07-18T03:40:42.175675+0000","last_became_active":"2023-07-18T03:40:39.942761+0000","last_became_peered":"2023-07-18T03:40:39.942761+0000","last_unstale":"2023-07-18T03:40:42.175675+0000","last_undegraded":"2023-07-18T03:40:42.175675+0000","last_fullsized":"2023-07-18T03:40:42.175675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:22:41.585488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175600+0000","last_change":"2023-07-18T03:40:40.970998+0000","last_active":"2023-07-18T03:40:42.175600+0000","last_peered":"2023-07-18T03:40:42.175600+0000","last_clean":"2023-07-18T03:40:42.175600+0000","last_became_active":"2023-07-18T03:40:40.970167+0000","last_became_peered":"2023-07-18T03:40:40.970167+0000","last_unstale":"2023-07-18T03:40:42.175600+0000","last_undegraded":"2023-07-18T03:40:42.175600+0000","last_fullsized":"2023-07-18T03:40:42.175600+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:05:53.356207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.978514+0000","last_change":"2023-07-18T03:40:39.948252+0000","last_active":"2023-07-18T03:40:40.978514+0000","last_peered":"2023-07-18T03:40:40.978514+0000","last_clean":"2023-07-18T03:40:40.978514+0000","last_became_active":"2023-07-18T03:40:39.947975+0000","last_became_peered":"2023-07-18T03:40:39.947975+0000","last_unstale":"2023-07-18T03:40:40.978514+0000","last_undegraded":"2023-07-18T03:40:40.978514+0000","last_fullsized":"2023-07-18T03:40:40.978514+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:28:21.251676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.978569+0000","last_change":"2023-07-18T03:40:40.968117+0000","last_active":"2023-07-18T03:40:40.978569+0000","last_peered":"2023-07-18T03:40:40.978569+0000","last_clean":"2023-07-18T03:40:40.978569+0000","last_became_active":"2023-07-18T03:40:40.967596+0000","last_became_peered":"2023-07-18T03:40:40.967596+0000","last_unstale":"2023-07-18T03:40:40.978569+0000","last_undegraded":"2023-07-18T03:40:40.978569+0000","last_fullsized":"2023-07-18T03:40:40.978569+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:49:25.710002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175444+0000","last_change":"2023-07-18T03:40:39.956348+0000","last_active":"2023-07-18T03:40:42.175444+0000","last_peered":"2023-07-18T03:40:42.175444+0000","last_clean":"2023-07-18T03:40:42.175444+0000","last_became_active":"2023-07-18T03:40:39.955890+0000","last_became_peered":"2023-07-18T03:40:39.955890+0000","last_unstale":"2023-07-18T03:40:42.175444+0000","last_undegraded":"2023-07-18T03:40:42.175444+0000","last_fullsized":"2023-07-18T03:40:42.175444+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:54:13.800509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.954807+0000","last_change":"2023-07-18T03:40:40.956173+0000","last_active":"2023-07-18T03:40:41.954807+0000","last_peered":"2023-07-18T03:40:41.954807+0000","last_clean":"2023-07-18T03:40:41.954807+0000","last_became_active":"2023-07-18T03:40:40.955906+0000","last_became_peered":"2023-07-18T03:40:40.955906+0000","last_unstale":"2023-07-18T03:40:41.954807+0000","last_undegraded":"2023-07-18T03:40:41.954807+0000","last_fullsized":"2023-07-18T03:40:41.954807+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:20:49.935393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.956686+0000","last_change":"2023-07-18T03:40:39.943211+0000","last_active":"2023-07-18T03:40:41.956686+0000","last_peered":"2023-07-18T03:40:41.956686+0000","last_clean":"2023-07-18T03:40:41.956686+0000","last_became_active":"2023-07-18T03:40:39.942910+0000","last_became_peered":"2023-07-18T03:40:39.942910+0000","last_unstale":"2023-07-18T03:40:41.956686+0000","last_undegraded":"2023-07-18T03:40:41.956686+0000","last_fullsized":"2023-07-18T03:40:41.956686+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:43:27.533664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958617+0000","last_change":"2023-07-18T03:40:40.968844+0000","last_active":"2023-07-18T03:40:41.958617+0000","last_peered":"2023-07-18T03:40:41.958617+0000","last_clean":"2023-07-18T03:40:41.958617+0000","last_became_active":"2023-07-18T03:40:40.968560+0000","last_became_peered":"2023-07-18T03:40:40.968560+0000","last_unstale":"2023-07-18T03:40:41.958617+0000","last_undegraded":"2023-07-18T03:40:41.958617+0000","last_fullsized":"2023-07-18T03:40:41.958617+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:40:55.796859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176264+0000","last_change":"2023-07-18T03:40:39.946852+0000","last_active":"2023-07-18T03:40:42.176264+0000","last_peered":"2023-07-18T03:40:42.176264+0000","last_clean":"2023-07-18T03:40:42.176264+0000","last_became_active":"2023-07-18T03:40:39.946570+0000","last_became_peered":"2023-07-18T03:40:39.946570+0000","last_unstale":"2023-07-18T03:40:42.176264+0000","last_undegraded":"2023-07-18T03:40:42.176264+0000","last_fullsized":"2023-07-18T03:40:42.176264+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:03:00.670608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.954972+0000","last_change":"2023-07-18T03:40:40.962602+0000","last_active":"2023-07-18T03:40:41.954972+0000","last_peered":"2023-07-18T03:40:41.954972+0000","last_clean":"2023-07-18T03:40:41.954972+0000","last_became_active":"2023-07-18T03:40:40.962288+0000","last_became_peered":"2023-07-18T03:40:40.962288+0000","last_unstale":"2023-07-18T03:40:41.954972+0000","last_undegraded":"2023-07-18T03:40:41.954972+0000","last_fullsized":"2023-07-18T03:40:41.954972+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:20:30.773061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176145+0000","last_change":"2023-07-18T03:40:39.950796+0000","last_active":"2023-07-18T03:40:42.176145+0000","last_peered":"2023-07-18T03:40:42.176145+0000","last_clean":"2023-07-18T03:40:42.176145+0000","last_became_active":"2023-07-18T03:40:39.950502+0000","last_became_peered":"2023-07-18T03:40:39.950502+0000","last_unstale":"2023-07-18T03:40:42.176145+0000","last_undegraded":"2023-07-18T03:40:42.176145+0000","last_fullsized":"2023-07-18T03:40:42.176145+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:23:26.305569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176633+0000","last_change":"2023-07-18T03:40:40.964038+0000","last_active":"2023-07-18T03:40:42.176633+0000","last_peered":"2023-07-18T03:40:42.176633+0000","last_clean":"2023-07-18T03:40:42.176633+0000","last_became_active":"2023-07-18T03:40:40.963745+0000","last_became_peered":"2023-07-18T03:40:40.963745+0000","last_unstale":"2023-07-18T03:40:42.176633+0000","last_undegraded":"2023-07-18T03:40:42.176633+0000","last_fullsized":"2023-07-18T03:40:42.176633+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:55:37.610565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955110+0000","last_change":"2023-07-18T03:40:39.941612+0000","last_active":"2023-07-18T03:40:41.955110+0000","last_peered":"2023-07-18T03:40:41.955110+0000","last_clean":"2023-07-18T03:40:41.955110+0000","last_became_active":"2023-07-18T03:40:39.941341+0000","last_became_peered":"2023-07-18T03:40:39.941341+0000","last_unstale":"2023-07-18T03:40:41.955110+0000","last_undegraded":"2023-07-18T03:40:41.955110+0000","last_fullsized":"2023-07-18T03:40:41.955110+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:15:28.593173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175682+0000","last_change":"2023-07-18T03:40:40.959113+0000","last_active":"2023-07-18T03:40:42.175682+0000","last_peered":"2023-07-18T03:40:42.175682+0000","last_clean":"2023-07-18T03:40:42.175682+0000","last_became_active":"2023-07-18T03:40:40.958791+0000","last_became_peered":"2023-07-18T03:40:40.958791+0000","last_unstale":"2023-07-18T03:40:42.175682+0000","last_undegraded":"2023-07-18T03:40:42.175682+0000","last_fullsized":"2023-07-18T03:40:42.175682+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:20:49.935393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.956685+0000","last_change":"2023-07-18T03:40:39.949108+0000","last_active":"2023-07-18T03:40:40.956685+0000","last_peered":"2023-07-18T03:40:40.956685+0000","last_clean":"2023-07-18T03:40:40.956685+0000","last_became_active":"2023-07-18T03:40:39.948783+0000","last_became_peered":"2023-07-18T03:40:39.948783+0000","last_unstale":"2023-07-18T03:40:40.956685+0000","last_undegraded":"2023-07-18T03:40:40.956685+0000","last_fullsized":"2023-07-18T03:40:40.956685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:53:39.039261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175796+0000","last_change":"2023-07-18T03:40:40.963366+0000","last_active":"2023-07-18T03:40:42.175796+0000","last_peered":"2023-07-18T03:40:42.175796+0000","last_clean":"2023-07-18T03:40:42.175796+0000","last_became_active":"2023-07-18T03:40:40.963056+0000","last_became_peered":"2023-07-18T03:40:40.963056+0000","last_unstale":"2023-07-18T03:40:42.175796+0000","last_undegraded":"2023-07-18T03:40:42.175796+0000","last_fullsized":"2023-07-18T03:40:42.175796+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:07:25.341574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175480+0000","last_change":"2023-07-18T03:40:39.954508+0000","last_active":"2023-07-18T03:40:42.175480+0000","last_peered":"2023-07-18T03:40:42.175480+0000","last_clean":"2023-07-18T03:40:42.175480+0000","last_became_active":"2023-07-18T03:40:39.954255+0000","last_became_peered":"2023-07-18T03:40:39.954255+0000","last_unstale":"2023-07-18T03:40:42.175480+0000","last_undegraded":"2023-07-18T03:40:42.175480+0000","last_fullsized":"2023-07-18T03:40:42.175480+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:40:42.495459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955226+0000","last_change":"2023-07-18T03:40:40.966165+0000","last_active":"2023-07-18T03:40:41.955226+0000","last_peered":"2023-07-18T03:40:41.955226+0000","last_clean":"2023-07-18T03:40:41.955226+0000","last_became_active":"2023-07-18T03:40:40.965877+0000","last_became_peered":"2023-07-18T03:40:40.965877+0000","last_unstale":"2023-07-18T03:40:41.955226+0000","last_undegraded":"2023-07-18T03:40:41.955226+0000","last_fullsized":"2023-07-18T03:40:41.955226+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:28:30.323138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955844+0000","last_change":"2023-07-18T03:40:39.948052+0000","last_active":"2023-07-18T03:40:41.955844+0000","last_peered":"2023-07-18T03:40:41.955844+0000","last_clean":"2023-07-18T03:40:41.955844+0000","last_became_active":"2023-07-18T03:40:39.947595+0000","last_became_peered":"2023-07-18T03:40:39.947595+0000","last_unstale":"2023-07-18T03:40:41.955844+0000","last_undegraded":"2023-07-18T03:40:41.955844+0000","last_fullsized":"2023-07-18T03:40:41.955844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:26:53.934794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175266+0000","last_change":"2023-07-18T03:40:40.972063+0000","last_active":"2023-07-18T03:40:42.175266+0000","last_peered":"2023-07-18T03:40:42.175266+0000","last_clean":"2023-07-18T03:40:42.175266+0000","last_became_active":"2023-07-18T03:40:40.971793+0000","last_became_peered":"2023-07-18T03:40:40.971793+0000","last_unstale":"2023-07-18T03:40:42.175266+0000","last_undegraded":"2023-07-18T03:40:42.175266+0000","last_fullsized":"2023-07-18T03:40:42.175266+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:31:09.653739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.956690+0000","last_change":"2023-07-18T03:40:39.949161+0000","last_active":"2023-07-18T03:40:40.956690+0000","last_peered":"2023-07-18T03:40:40.956690+0000","last_clean":"2023-07-18T03:40:40.956690+0000","last_became_active":"2023-07-18T03:40:39.948853+0000","last_became_peered":"2023-07-18T03:40:39.948853+0000","last_unstale":"2023-07-18T03:40:40.956690+0000","last_undegraded":"2023-07-18T03:40:40.956690+0000","last_fullsized":"2023-07-18T03:40:40.956690+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:20:42.256618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958759+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:40:41.958759+0000","last_peered":"2023-07-18T03:40:41.958759+0000","last_clean":"2023-07-18T03:40:41.958759+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:40:41.958759+0000","last_undegraded":"2023-07-18T03:40:41.958759+0000","last_fullsized":"2023-07-18T03:40:41.958759+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:21:24.489457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176656+0000","last_change":"2023-07-18T03:40:40.964542+0000","last_active":"2023-07-18T03:40:42.176656+0000","last_peered":"2023-07-18T03:40:42.176656+0000","last_clean":"2023-07-18T03:40:42.176656+0000","last_became_active":"2023-07-18T03:40:40.964223+0000","last_became_peered":"2023-07-18T03:40:40.964223+0000","last_unstale":"2023-07-18T03:40:42.176656+0000","last_undegraded":"2023-07-18T03:40:42.176656+0000","last_fullsized":"2023-07-18T03:40:42.176656+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:53:07.258459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.956851+0000","last_change":"2023-07-18T03:40:39.950158+0000","last_active":"2023-07-18T03:40:40.956851+0000","last_peered":"2023-07-18T03:40:40.956851+0000","last_clean":"2023-07-18T03:40:40.956851+0000","last_became_active":"2023-07-18T03:40:39.949883+0000","last_became_peered":"2023-07-18T03:40:39.949883+0000","last_unstale":"2023-07-18T03:40:40.956851+0000","last_undegraded":"2023-07-18T03:40:40.956851+0000","last_fullsized":"2023-07-18T03:40:40.956851+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:32:34.975883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.959028+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:40:40.959028+0000","last_peered":"2023-07-18T03:40:40.959028+0000","last_clean":"2023-07-18T03:40:40.959028+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:40:40.959028+0000","last_undegraded":"2023-07-18T03:40:40.959028+0000","last_fullsized":"2023-07-18T03:40:40.959028+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:43:15.003936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.954995+0000","last_change":"2023-07-18T03:40:40.965589+0000","last_active":"2023-07-18T03:40:41.954995+0000","last_peered":"2023-07-18T03:40:41.954995+0000","last_clean":"2023-07-18T03:40:41.954995+0000","last_became_active":"2023-07-18T03:40:40.965290+0000","last_became_peered":"2023-07-18T03:40:40.965290+0000","last_unstale":"2023-07-18T03:40:41.954995+0000","last_undegraded":"2023-07-18T03:40:41.954995+0000","last_fullsized":"2023-07-18T03:40:41.954995+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:46:38.287766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176148+0000","last_change":"2023-07-18T03:40:39.942958+0000","last_active":"2023-07-18T03:40:42.176148+0000","last_peered":"2023-07-18T03:40:42.176148+0000","last_clean":"2023-07-18T03:40:42.176148+0000","last_became_active":"2023-07-18T03:40:39.942655+0000","last_became_peered":"2023-07-18T03:40:39.942655+0000","last_unstale":"2023-07-18T03:40:42.176148+0000","last_undegraded":"2023-07-18T03:40:42.176148+0000","last_fullsized":"2023-07-18T03:40:42.176148+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:15:22.492626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955064+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:40:41.955064+0000","last_peered":"2023-07-18T03:40:41.955064+0000","last_clean":"2023-07-18T03:40:41.955064+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:40:41.955064+0000","last_undegraded":"2023-07-18T03:40:41.955064+0000","last_fullsized":"2023-07-18T03:40:41.955064+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:00:50.073596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":220,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175478+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:40:42.175478+0000","last_peered":"2023-07-18T03:40:42.175478+0000","last_clean":"2023-07-18T03:40:42.175478+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:40:42.175478+0000","last_undegraded":"2023-07-18T03:40:42.175478+0000","last_fullsized":"2023-07-18T03:40:42.175478+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:39.153889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.975518+0000","last_change":"2023-07-18T03:40:40.965001+0000","last_active":"2023-07-18T03:40:40.975518+0000","last_peered":"2023-07-18T03:40:40.975518+0000","last_clean":"2023-07-18T03:40:40.975518+0000","last_became_active":"2023-07-18T03:40:40.964710+0000","last_became_peered":"2023-07-18T03:40:40.964710+0000","last_unstale":"2023-07-18T03:40:40.975518+0000","last_undegraded":"2023-07-18T03:40:40.975518+0000","last_fullsized":"2023-07-18T03:40:40.975518+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:51:15.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175116+0000","last_change":"2023-07-18T03:40:39.953906+0000","last_active":"2023-07-18T03:40:42.175116+0000","last_peered":"2023-07-18T03:40:42.175116+0000","last_clean":"2023-07-18T03:40:42.175116+0000","last_became_active":"2023-07-18T03:40:39.953567+0000","last_became_peered":"2023-07-18T03:40:39.953567+0000","last_unstale":"2023-07-18T03:40:42.175116+0000","last_undegraded":"2023-07-18T03:40:42.175116+0000","last_fullsized":"2023-07-18T03:40:42.175116+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:00.519303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176241+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:40:42.176241+0000","last_peered":"2023-07-18T03:40:42.176241+0000","last_clean":"2023-07-18T03:40:42.176241+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:40:42.176241+0000","last_undegraded":"2023-07-18T03:40:42.176241+0000","last_fullsized":"2023-07-18T03:40:42.176241+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:26:11.213686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175093+0000","last_change":"2023-07-18T03:40:40.968119+0000","last_active":"2023-07-18T03:40:42.175093+0000","last_peered":"2023-07-18T03:40:42.175093+0000","last_clean":"2023-07-18T03:40:42.175093+0000","last_became_active":"2023-07-18T03:40:40.967788+0000","last_became_peered":"2023-07-18T03:40:40.967788+0000","last_unstale":"2023-07-18T03:40:42.175093+0000","last_undegraded":"2023-07-18T03:40:42.175093+0000","last_fullsized":"2023-07-18T03:40:42.175093+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:46:35.942082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175670+0000","last_change":"2023-07-18T03:40:39.955933+0000","last_active":"2023-07-18T03:40:42.175670+0000","last_peered":"2023-07-18T03:40:42.175670+0000","last_clean":"2023-07-18T03:40:42.175670+0000","last_became_active":"2023-07-18T03:40:39.955566+0000","last_became_peered":"2023-07-18T03:40:39.955566+0000","last_unstale":"2023-07-18T03:40:42.175670+0000","last_undegraded":"2023-07-18T03:40:42.175670+0000","last_fullsized":"2023-07-18T03:40:42.175670+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:06:20.912545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176431+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:40:42.176431+0000","last_peered":"2023-07-18T03:40:42.176431+0000","last_clean":"2023-07-18T03:40:42.176431+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:40:42.176431+0000","last_undegraded":"2023-07-18T03:40:42.176431+0000","last_fullsized":"2023-07-18T03:40:42.176431+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:27.768668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176557+0000","last_change":"2023-07-18T03:40:40.970205+0000","last_active":"2023-07-18T03:40:42.176557+0000","last_peered":"2023-07-18T03:40:42.176557+0000","last_clean":"2023-07-18T03:40:42.176557+0000","last_became_active":"2023-07-18T03:40:40.969812+0000","last_became_peered":"2023-07-18T03:40:40.969812+0000","last_unstale":"2023-07-18T03:40:42.176557+0000","last_undegraded":"2023-07-18T03:40:42.176557+0000","last_fullsized":"2023-07-18T03:40:42.176557+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:05:16.859125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176039+0000","last_change":"2023-07-18T03:40:39.956778+0000","last_active":"2023-07-18T03:40:42.176039+0000","last_peered":"2023-07-18T03:40:42.176039+0000","last_clean":"2023-07-18T03:40:42.176039+0000","last_became_active":"2023-07-18T03:40:39.956487+0000","last_became_peered":"2023-07-18T03:40:39.956487+0000","last_unstale":"2023-07-18T03:40:42.176039+0000","last_undegraded":"2023-07-18T03:40:42.176039+0000","last_fullsized":"2023-07-18T03:40:42.176039+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:15:28.593173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176713+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:40:42.176713+0000","last_peered":"2023-07-18T03:40:42.176713+0000","last_clean":"2023-07-18T03:40:42.176713+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:40:42.176713+0000","last_undegraded":"2023-07-18T03:40:42.176713+0000","last_fullsized":"2023-07-18T03:40:42.176713+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:25:03.136061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.978002+0000","last_change":"2023-07-18T03:40:40.974072+0000","last_active":"2023-07-18T03:40:40.978002+0000","last_peered":"2023-07-18T03:40:40.978002+0000","last_clean":"2023-07-18T03:40:40.978002+0000","last_became_active":"2023-07-18T03:40:40.973819+0000","last_became_peered":"2023-07-18T03:40:40.973819+0000","last_unstale":"2023-07-18T03:40:40.978002+0000","last_undegraded":"2023-07-18T03:40:40.978002+0000","last_fullsized":"2023-07-18T03:40:40.978002+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:27:14.738764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176668+0000","last_change":"2023-07-18T03:40:39.946448+0000","last_active":"2023-07-18T03:40:42.176668+0000","last_peered":"2023-07-18T03:40:42.176668+0000","last_clean":"2023-07-18T03:40:42.176668+0000","last_became_active":"2023-07-18T03:40:39.946158+0000","last_became_peered":"2023-07-18T03:40:39.946158+0000","last_unstale":"2023-07-18T03:40:42.176668+0000","last_undegraded":"2023-07-18T03:40:42.176668+0000","last_fullsized":"2023-07-18T03:40:42.176668+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:47:14.934558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175956+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:40:42.175956+0000","last_peered":"2023-07-18T03:40:42.175956+0000","last_clean":"2023-07-18T03:40:42.175956+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:40:42.175956+0000","last_undegraded":"2023-07-18T03:40:42.175956+0000","last_fullsized":"2023-07-18T03:40:42.175956+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:09:53.507284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175435+0000","last_change":"2023-07-18T03:40:40.979329+0000","last_active":"2023-07-18T03:40:42.175435+0000","last_peered":"2023-07-18T03:40:42.175435+0000","last_clean":"2023-07-18T03:40:42.175435+0000","last_became_active":"2023-07-18T03:40:40.979114+0000","last_became_peered":"2023-07-18T03:40:40.979114+0000","last_unstale":"2023-07-18T03:40:42.175435+0000","last_undegraded":"2023-07-18T03:40:42.175435+0000","last_fullsized":"2023-07-18T03:40:42.175435+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:05:36.754409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958573+0000","last_change":"2023-07-18T03:40:39.949191+0000","last_active":"2023-07-18T03:40:41.958573+0000","last_peered":"2023-07-18T03:40:41.958573+0000","last_clean":"2023-07-18T03:40:41.958573+0000","last_became_active":"2023-07-18T03:40:39.948816+0000","last_became_peered":"2023-07-18T03:40:39.948816+0000","last_unstale":"2023-07-18T03:40:41.958573+0000","last_undegraded":"2023-07-18T03:40:41.958573+0000","last_fullsized":"2023-07-18T03:40:41.958573+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:09.912282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176170+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:40:42.176170+0000","last_peered":"2023-07-18T03:40:42.176170+0000","last_clean":"2023-07-18T03:40:42.176170+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:40:42.176170+0000","last_undegraded":"2023-07-18T03:40:42.176170+0000","last_fullsized":"2023-07-18T03:40:42.176170+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:30:06.650785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1144,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1171456,"data_stored":837345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85599999999999998,"5min":1.167,"15min":1.167},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.68700000000000006}]},{"osd":1,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71999999999999997,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.71999999999999997}]},{"osd":2,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.62},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.873,"15min":0.89500000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.2570000000000001,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.746,"15min":0.753},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1419999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0249999999999999,"5min":1.365,"15min":1.365},"last":0.67000000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.0010000000000003,"5min":4.0010000000000003,"15min":4.0010000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.495}]},{"osd":5,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.72899999999999998,"15min":0.71199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.2989999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.77700000000000002,"15min":0.76000000000000001},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.64500000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.81699999999999995,"15min":0.80000000000000004},"min":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0609999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.55800000000000005}]}]},{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":577536,"data_stored":247505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.73799999999999999,"15min":0.754},"min":{"1min":0.51400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84699999999999998,"5min":2.427,"15min":2.427},"last":0.83499999999999996}]},{"osd":1,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.84099999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76700000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.873},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95999999999999996,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.72199999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91300000000000003,"5min":1.034,"15min":1.034},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96399999999999997,"5min":1.113,"15min":1.113},"last":0.78700000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90400000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66500000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55900000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81000000000000005,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0820000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.90800000000000003}]}]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":573440,"data_stored":241786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.432,"15min":0.441},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.46300000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.505},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.58899999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60599999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.69999999999999996,"15min":0.72799999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.66100000000000003,"5min":3.956,"15min":3.956},"last":0.433},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.68400000000000005,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.755,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.58099999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.94399999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.86599999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.63200000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.92900000000000005,"5min":1.22,"15min":1.22},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.88200000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625}]},{"osd":6,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91300000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90400000000000003,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.81799999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.94599999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.79900000000000004,"15min":0.81000000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84899999999999998,"5min":1.272,"15min":1.272},"last":0.66200000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1132,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1159168,"data_stored":830728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64500000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.438},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.46100000000000002,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.434,"15min":0.435},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.60099999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40999999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.65700000000000003,"5min":0.748,"15min":0.748},"last":0.47699999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90400000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.59399999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":0.73799999999999999},"min":{"1min":0.57199999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84999999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.747,"15min":0.754},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88400000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.754,"15min":0.755},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.94999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58299999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.61499999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.01,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.626},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92000000000000004,"5min":1.042,"15min":1.042},"last":0.57099999999999995}]}]},{"osd":4,"up_from":14,"seq":60129542175,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":573440,"data_stored":247435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48599999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.47499999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.434}]},{"osd":2,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89400000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.81699999999999995,"5min":1.032,"15min":1.032},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67000000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.70399999999999996,"15min":0.71599999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.78100000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92700000000000005,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92200000000000004,"5min":0.999,"15min":0.999},"last":0.70099999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":557056,"data_stored":240200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.67500000000000004,"15min":0.63700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.68600000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.74099999999999999,"15min":0.68700000000000006},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":8.3780000000000001,"5min":8.3780000000000001,"15min":8.3780000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.74299999999999999,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.82599999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.77200000000000002,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":4.8769999999999998,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.65800000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70499999999999996,"15min":0.69499999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.70199999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.755,"15min":0.73899999999999999},"min":{"1min":0.41199999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.9690000000000003,"5min":4.9690000000000003,"15min":4.9690000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.6829999999999998,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71399999999999997,"15min":0.71899999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90800000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.83899999999999997,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.3140000000000001,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":0.56699999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.80300000000000005,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":3.7389999999999999,"5min":3.7389999999999999,"15min":3.7389999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94799999999999995,"5min":1.472,"15min":1.472},"last":0.59899999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":577536,"data_stored":246909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.47499999999999998,"15min":0.48899999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":1.389,"15min":1.389},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.64500000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.84099999999999997,"15min":0.90300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.74199999999999999,"5min":8.3040000000000003,"15min":8.3040000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.66700000000000004,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.77300000000000002,"15min":0.81899999999999995},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63100000000000001,"5min":8.0229999999999997,"15min":8.0229999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.88,"15min":0.93799999999999994},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.70999999999999996,"5min":8.359,"15min":8.359},"last":0.46600000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97699999999999998,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78400000000000003,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0129999999999999,"5min":1.429,"15min":1.429},"last":0.77900000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.117,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.754,"15min":0.755},"min":{"1min":0.623,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0920000000000001,"5min":1.266,"15min":1.266},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95199999999999996,"5min":1.752,"15min":1.752},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.626,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0449999999999999,"5min":1.454,"15min":1.454},"last":0.76100000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.58499999999999996,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.94299999999999995,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0249999999999999,"5min":1.222,"15min":1.222},"last":0.60099999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":581632,"data_stored":248417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.40300000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.503,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.498,"15min":0.497},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72499999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.46000000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.48299999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.622,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.58799999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78700000000000003,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56999999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0580000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.038,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67000000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.60799999999999998,"5min":0.504,"15min":0.504},"max":{"1min":1.0209999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.61699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:40:45.402 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:40:45.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.640+0000 7fa0445ff700 1 Processor -- start 2023-07-18T03:40:45.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.640+0000 7fa0445ff700 1 -- start start 2023-07-18T03:40:45.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa0445ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa03c0b4510 0x7fa03c0b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:45.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa0445ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa03c0b5d90 0x7fa03c0cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:45.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa0445ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03c0b7f50 con 0x7fa03c0d0b10 2023-07-18T03:40:45.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa0445ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa03c0b48e0 con 0x7fa03c0b4510 2023-07-18T03:40:45.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa0445ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c0b7340 con 0x7fa03c0b5d90 2023-07-18T03:40:45.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa04239b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa03c0b4510 0x7fa03c0b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:45.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa04239b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa03c0b4510 0x7fa03c0b6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35594/0 (socket says 172.21.15.49:35594) 2023-07-18T03:40:45.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa04239b700 1 -- 172.21.15.49:0/3177766376 learned_addr learned my addr 172.21.15.49:0/3177766376 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:45.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa041b9a700 1 --2- 172.21.15.49:0/3177766376 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa03c0b5d90 0x7fa03c0cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:45.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa04239b700 1 -- 172.21.15.49:0/3177766376 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa03c0b5d90 msgr2=0x7fa03c0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:45.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa04239b700 1 --2- 172.21.15.49:0/3177766376 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa03c0b5d90 0x7fa03c0cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa04239b700 1 -- 172.21.15.49:0/3177766376 >> v1:172.21.15.107:6789/0 conn(0x7fa03c0d0b10 legacy=0x7fa03c0b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:45.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.642+0000 7fa04239b700 1 -- 172.21.15.49:0/3177766376 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c0b7620 con 0x7fa03c0b4510 2023-07-18T03:40:45.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.643+0000 7fa04239b700 1 --2- 172.21.15.49:0/3177766376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c0b4510 0x7fa03c0b6c80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa0380098e0 tx=0x7fa038009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc76a1112ef0f7d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:45.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.643+0000 7fa041399700 1 -- 172.21.15.49:0/3177766376 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03800b8a0 con 0x7fa03c0b4510 2023-07-18T03:40:45.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.643+0000 7fa041399700 1 -- 172.21.15.49:0/3177766376 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03800ba00 con 0x7fa03c0b4510 2023-07-18T03:40:45.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.643+0000 7fa041399700 1 -- 172.21.15.49:0/3177766376 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03800bdb0 con 0x7fa03c0b4510 2023-07-18T03:40:45.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.643+0000 7fa0445ff700 1 -- 172.21.15.49:0/3177766376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c0b4510 msgr2=0x7fa03c0b6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:45.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.643+0000 7fa0445ff700 1 --2- 172.21.15.49:0/3177766376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c0b4510 0x7fa03c0b6c80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa0380098e0 tx=0x7fa038009bf0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.644+0000 7fa0445ff700 1 -- 172.21.15.49:0/3177766376 shutdown_connections 2023-07-18T03:40:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.644+0000 7fa0445ff700 1 --2- 172.21.15.49:0/3177766376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c0b4510 0x7fa03c0b6c80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.644+0000 7fa0445ff700 1 --2- 172.21.15.49:0/3177766376 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa03c0b5d90 0x7fa03c0cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.644+0000 7fa0445ff700 1 -- 172.21.15.49:0/3177766376 >> 172.21.15.49:0/3177766376 conn(0x7fa03c09c1d0 msgr2=0x7fa03c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.644+0000 7fa0445ff700 1 -- 172.21.15.49:0/3177766376 shutdown_connections 2023-07-18T03:40:45.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.644+0000 7fa0445ff700 1 -- 172.21.15.49:0/3177766376 wait complete. 2023-07-18T03:40:45.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.646+0000 7fa0445ff700 1 Processor -- start 2023-07-18T03:40:45.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.646+0000 7fa0445ff700 1 -- start start 2023-07-18T03:40:45.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa0445ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa03c0b4510 0x7fa03c1588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:45.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa0445ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c0b5d90 0x7fa03c155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:45.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa0445ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa03c0d0b10 0x7fa03c09e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:45.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa0445ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c157850 con 0x7fa03c0b5d90 2023-07-18T03:40:45.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa0445ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c157b30 con 0x7fa03c0b4510 2023-07-18T03:40:45.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa0445ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c157e10 con 0x7fa03c0d0b10 2023-07-18T03:40:45.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa041b9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c0b5d90 0x7fa03c155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:45.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa041b9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c0b5d90 0x7fa03c155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35600/0 (socket says 172.21.15.49:35600) 2023-07-18T03:40:45.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa041b9a700 1 -- 172.21.15.49:0/2857783714 learned_addr learned my addr 172.21.15.49:0/2857783714 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:45.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa04239b700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa03c0b4510 0x7fa03c1588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:45.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa042b9c700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa03c0d0b10 0x7fa03c09e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:45.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa041b9a700 1 -- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa03c0d0b10 msgr2=0x7fa03c09e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:45.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa041b9a700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa03c0d0b10 0x7fa03c09e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa041b9a700 1 -- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa03c0b4510 msgr2=0x7fa03c1588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:45.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa041b9a700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa03c0b4510 0x7fa03c1588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa041b9a700 1 -- 172.21.15.49:0/2857783714 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0380094f0 con 0x7fa03c0b5d90 2023-07-18T03:40:45.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.647+0000 7fa042b9c700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa03c0d0b10 0x7fa03c09e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:45.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.648+0000 7fa041b9a700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c0b5d90 0x7fa03c155260 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00e810 tx=0x7fa02c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:45.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.648+0000 7fa032ffd700 1 -- 172.21.15.49:0/2857783714 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c013610 con 0x7fa03c0b5d90 2023-07-18T03:40:45.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.648+0000 7fa032ffd700 1 -- 172.21.15.49:0/2857783714 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02c013770 con 0x7fa03c0b5d90 2023-07-18T03:40:45.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.648+0000 7fa032ffd700 1 -- 172.21.15.49:0/2857783714 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c013b20 con 0x7fa03c0b5d90 2023-07-18T03:40:45.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.648+0000 7fa0445ff700 1 -- 172.21.15.49:0/2857783714 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa03c1580f0 con 0x7fa03c0b5d90 2023-07-18T03:40:45.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.648+0000 7fa0445ff700 1 -- 172.21.15.49:0/2857783714 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03c09ed40 con 0x7fa03c0b5d90 2023-07-18T03:40:45.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.649+0000 7fa0445ff700 1 -- 172.21.15.49:0/2857783714 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03c04e260 con 0x7fa03c0b5d90 2023-07-18T03:40:45.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.650+0000 7fa032ffd700 1 -- 172.21.15.49:0/2857783714 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fa02c013c80 con 0x7fa03c0b5d90 2023-07-18T03:40:45.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.651+0000 7fa032ffd700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa02006ad70 0x7fa02006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:45.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.651+0000 7fa032ffd700 1 -- 172.21.15.49:0/2857783714 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa02c0892a0 con 0x7fa03c0b5d90 2023-07-18T03:40:45.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.652+0000 7fa04239b700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa02006ad70 0x7fa02006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:45.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.652+0000 7fa04239b700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa02006ad70 0x7fa02006d220 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa038009c30 tx=0x7fa038008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:45.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.655+0000 7fa032ffd700 1 -- 172.21.15.49:0/2857783714 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa02c057bf0 con 0x7fa03c0b5d90 2023-07-18T03:40:45.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.860+0000 7fa0445ff700 1 -- 172.21.15.49:0/2857783714 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa03c157ca0 con 0x7fa02006ad70 2023-07-18T03:40:45.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.869+0000 7fa032ffd700 1 -- 172.21.15.49:0/2857783714 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311762 (secure 0 0 0) 0x7fa03c157ca0 con 0x7fa02006ad70 2023-07-18T03:40:45.868 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:45.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.874+0000 7fa0445ff700 1 -- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa02006ad70 msgr2=0x7fa02006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:45.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.874+0000 7fa0445ff700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa02006ad70 0x7fa02006d220 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa038009c30 tx=0x7fa038008030 comp rx=0 tx=0).stop 2023-07-18T03:40:45.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.874+0000 7fa0445ff700 1 -- 172.21.15.49:0/2857783714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c0b5d90 msgr2=0x7fa03c155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:45.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.874+0000 7fa0445ff700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c0b5d90 0x7fa03c155260 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00e810 tx=0x7fa02c00eb20 comp rx=0 tx=0).stop 2023-07-18T03:40:45.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.874+0000 7fa0445ff700 1 -- 172.21.15.49:0/2857783714 shutdown_connections 2023-07-18T03:40:45.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.874+0000 7fa0445ff700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa03c0d0b10 0x7fa03c09e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.874+0000 7fa0445ff700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa03c0b4510 0x7fa03c1588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.874+0000 7fa0445ff700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa02006ad70 0x7fa02006d220 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.874+0000 7fa0445ff700 1 --2- 172.21.15.49:0/2857783714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa03c0b5d90 0x7fa03c155260 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:45.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.874+0000 7fa0445ff700 1 -- 172.21.15.49:0/2857783714 >> 172.21.15.49:0/2857783714 conn(0x7fa03c09c1d0 msgr2=0x7fa03c0b83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:45.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.874+0000 7fa0445ff700 1 -- 172.21.15.49:0/2857783714 shutdown_connections 2023-07-18T03:40:45.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:45.874+0000 7fa0445ff700 1 -- 172.21.15.49:0/2857783714 wait complete. 2023-07-18T03:40:45.871 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:40:45.888 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2023-07-18T03:40:44.127550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237636,"kb_used_data":5636,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754737084,"statfs":{"total":773094113280,"available":772850774016,"internally_reserved":0,"allocated":5771264,"data_stored":3140325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.203371"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955090+0000","last_change":"2023-07-18T03:40:40.961987+0000","last_active":"2023-07-18T03:40:41.955090+0000","last_peered":"2023-07-18T03:40:41.955090+0000","last_clean":"2023-07-18T03:40:41.955090+0000","last_became_active":"2023-07-18T03:40:40.961699+0000","last_became_peered":"2023-07-18T03:40:40.961699+0000","last_unstale":"2023-07-18T03:40:41.955090+0000","last_undegraded":"2023-07-18T03:40:41.955090+0000","last_fullsized":"2023-07-18T03:40:41.955090+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:12:05.109987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175566+0000","last_change":"2023-07-18T03:40:40.962694+0000","last_active":"2023-07-18T03:40:42.175566+0000","last_peered":"2023-07-18T03:40:42.175566+0000","last_clean":"2023-07-18T03:40:42.175566+0000","last_became_active":"2023-07-18T03:40:40.962429+0000","last_became_peered":"2023-07-18T03:40:40.962429+0000","last_unstale":"2023-07-18T03:40:42.175566+0000","last_undegraded":"2023-07-18T03:40:42.175566+0000","last_fullsized":"2023-07-18T03:40:42.175566+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:39:12.540003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175659+0000","last_change":"2023-07-18T03:40:40.973776+0000","last_active":"2023-07-18T03:40:42.175659+0000","last_peered":"2023-07-18T03:40:42.175659+0000","last_clean":"2023-07-18T03:40:42.175659+0000","last_became_active":"2023-07-18T03:40:40.973451+0000","last_became_peered":"2023-07-18T03:40:40.973451+0000","last_unstale":"2023-07-18T03:40:42.175659+0000","last_undegraded":"2023-07-18T03:40:42.175659+0000","last_fullsized":"2023-07-18T03:40:42.175659+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:20:53.697417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175815+0000","last_change":"2023-07-18T03:40:40.966335+0000","last_active":"2023-07-18T03:40:42.175815+0000","last_peered":"2023-07-18T03:40:42.175815+0000","last_clean":"2023-07-18T03:40:42.175815+0000","last_became_active":"2023-07-18T03:40:40.966041+0000","last_became_peered":"2023-07-18T03:40:40.966041+0000","last_unstale":"2023-07-18T03:40:42.175815+0000","last_undegraded":"2023-07-18T03:40:42.175815+0000","last_fullsized":"2023-07-18T03:40:42.175815+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:05.139386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958766+0000","last_change":"2023-07-18T03:40:40.949371+0000","last_active":"2023-07-18T03:40:41.958766+0000","last_peered":"2023-07-18T03:40:41.958766+0000","last_clean":"2023-07-18T03:40:41.958766+0000","last_became_active":"2023-07-18T03:40:40.948989+0000","last_became_peered":"2023-07-18T03:40:40.948989+0000","last_unstale":"2023-07-18T03:40:41.958766+0000","last_undegraded":"2023-07-18T03:40:41.958766+0000","last_fullsized":"2023-07-18T03:40:41.958766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:16:23.107948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955132+0000","last_change":"2023-07-18T03:40:40.957623+0000","last_active":"2023-07-18T03:40:41.955132+0000","last_peered":"2023-07-18T03:40:41.955132+0000","last_clean":"2023-07-18T03:40:41.955132+0000","last_became_active":"2023-07-18T03:40:40.957363+0000","last_became_peered":"2023-07-18T03:40:40.957363+0000","last_unstale":"2023-07-18T03:40:41.955132+0000","last_undegraded":"2023-07-18T03:40:41.955132+0000","last_fullsized":"2023-07-18T03:40:41.955132+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:23:27.311259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175474+0000","last_change":"2023-07-18T03:40:40.966144+0000","last_active":"2023-07-18T03:40:42.175474+0000","last_peered":"2023-07-18T03:40:42.175474+0000","last_clean":"2023-07-18T03:40:42.175474+0000","last_became_active":"2023-07-18T03:40:40.965812+0000","last_became_peered":"2023-07-18T03:40:40.965812+0000","last_unstale":"2023-07-18T03:40:42.175474+0000","last_undegraded":"2023-07-18T03:40:42.175474+0000","last_fullsized":"2023-07-18T03:40:42.175474+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:25:54.877763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.978588+0000","last_change":"2023-07-18T03:40:40.971844+0000","last_active":"2023-07-18T03:40:40.978588+0000","last_peered":"2023-07-18T03:40:40.978588+0000","last_clean":"2023-07-18T03:40:40.978588+0000","last_became_active":"2023-07-18T03:40:40.971231+0000","last_became_peered":"2023-07-18T03:40:40.971231+0000","last_unstale":"2023-07-18T03:40:40.978588+0000","last_undegraded":"2023-07-18T03:40:40.978588+0000","last_fullsized":"2023-07-18T03:40:40.978588+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:53:38.313583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.982771+0000","last_change":"2023-07-18T03:40:40.977579+0000","last_active":"2023-07-18T03:40:40.982771+0000","last_peered":"2023-07-18T03:40:40.982771+0000","last_clean":"2023-07-18T03:40:40.982771+0000","last_became_active":"2023-07-18T03:40:40.977307+0000","last_became_peered":"2023-07-18T03:40:40.977307+0000","last_unstale":"2023-07-18T03:40:40.982771+0000","last_undegraded":"2023-07-18T03:40:40.982771+0000","last_fullsized":"2023-07-18T03:40:40.982771+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:30:32.044631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955561+0000","last_change":"2023-07-18T03:40:40.952693+0000","last_active":"2023-07-18T03:40:41.955561+0000","last_peered":"2023-07-18T03:40:41.955561+0000","last_clean":"2023-07-18T03:40:41.955561+0000","last_became_active":"2023-07-18T03:40:40.952384+0000","last_became_peered":"2023-07-18T03:40:40.952384+0000","last_unstale":"2023-07-18T03:40:41.955561+0000","last_undegraded":"2023-07-18T03:40:41.955561+0000","last_fullsized":"2023-07-18T03:40:41.955561+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:38:48.622343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.956398+0000","last_change":"2023-07-18T03:40:40.962995+0000","last_active":"2023-07-18T03:40:41.956398+0000","last_peered":"2023-07-18T03:40:41.956398+0000","last_clean":"2023-07-18T03:40:41.956398+0000","last_became_active":"2023-07-18T03:40:40.962702+0000","last_became_peered":"2023-07-18T03:40:40.962702+0000","last_unstale":"2023-07-18T03:40:41.956398+0000","last_undegraded":"2023-07-18T03:40:41.956398+0000","last_fullsized":"2023-07-18T03:40:41.956398+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:03:23.647349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.981591+0000","last_change":"2023-07-18T03:40:40.974913+0000","last_active":"2023-07-18T03:40:40.981591+0000","last_peered":"2023-07-18T03:40:40.981591+0000","last_clean":"2023-07-18T03:40:40.981591+0000","last_became_active":"2023-07-18T03:40:40.974618+0000","last_became_peered":"2023-07-18T03:40:40.974618+0000","last_unstale":"2023-07-18T03:40:40.981591+0000","last_undegraded":"2023-07-18T03:40:40.981591+0000","last_fullsized":"2023-07-18T03:40:40.981591+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:45:04.720217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.954950+0000","last_change":"2023-07-18T03:40:40.952736+0000","last_active":"2023-07-18T03:40:41.954950+0000","last_peered":"2023-07-18T03:40:41.954950+0000","last_clean":"2023-07-18T03:40:41.954950+0000","last_became_active":"2023-07-18T03:40:40.952398+0000","last_became_peered":"2023-07-18T03:40:40.952398+0000","last_unstale":"2023-07-18T03:40:41.954950+0000","last_undegraded":"2023-07-18T03:40:41.954950+0000","last_fullsized":"2023-07-18T03:40:41.954950+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:22:06.699892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175176+0000","last_change":"2023-07-18T03:40:40.973447+0000","last_active":"2023-07-18T03:40:42.175176+0000","last_peered":"2023-07-18T03:40:42.175176+0000","last_clean":"2023-07-18T03:40:42.175176+0000","last_became_active":"2023-07-18T03:40:40.973162+0000","last_became_peered":"2023-07-18T03:40:40.973162+0000","last_unstale":"2023-07-18T03:40:42.175176+0000","last_undegraded":"2023-07-18T03:40:42.175176+0000","last_fullsized":"2023-07-18T03:40:42.175176+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:30:25.988049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175662+0000","last_change":"2023-07-18T03:40:40.957906+0000","last_active":"2023-07-18T03:40:42.175662+0000","last_peered":"2023-07-18T03:40:42.175662+0000","last_clean":"2023-07-18T03:40:42.175662+0000","last_became_active":"2023-07-18T03:40:40.957647+0000","last_became_peered":"2023-07-18T03:40:40.957647+0000","last_unstale":"2023-07-18T03:40:42.175662+0000","last_undegraded":"2023-07-18T03:40:42.175662+0000","last_fullsized":"2023-07-18T03:40:42.175662+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:08:06.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.954809+0000","last_change":"2023-07-18T03:40:40.954836+0000","last_active":"2023-07-18T03:40:41.954809+0000","last_peered":"2023-07-18T03:40:41.954809+0000","last_clean":"2023-07-18T03:40:41.954809+0000","last_became_active":"2023-07-18T03:40:40.954559+0000","last_became_peered":"2023-07-18T03:40:40.954559+0000","last_unstale":"2023-07-18T03:40:41.954809+0000","last_undegraded":"2023-07-18T03:40:41.954809+0000","last_fullsized":"2023-07-18T03:40:41.954809+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:43:12.774955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175821+0000","last_change":"2023-07-18T03:40:40.970811+0000","last_active":"2023-07-18T03:40:42.175821+0000","last_peered":"2023-07-18T03:40:42.175821+0000","last_clean":"2023-07-18T03:40:42.175821+0000","last_became_active":"2023-07-18T03:40:40.969518+0000","last_became_peered":"2023-07-18T03:40:40.969518+0000","last_unstale":"2023-07-18T03:40:42.175821+0000","last_undegraded":"2023-07-18T03:40:42.175821+0000","last_fullsized":"2023-07-18T03:40:42.175821+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:47:16.011690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175900+0000","last_change":"2023-07-18T03:40:40.962259+0000","last_active":"2023-07-18T03:40:42.175900+0000","last_peered":"2023-07-18T03:40:42.175900+0000","last_clean":"2023-07-18T03:40:42.175900+0000","last_became_active":"2023-07-18T03:40:40.961970+0000","last_became_peered":"2023-07-18T03:40:40.961970+0000","last_unstale":"2023-07-18T03:40:42.175900+0000","last_undegraded":"2023-07-18T03:40:42.175900+0000","last_fullsized":"2023-07-18T03:40:42.175900+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:52:23.122594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175955+0000","last_change":"2023-07-18T03:40:40.971561+0000","last_active":"2023-07-18T03:40:42.175955+0000","last_peered":"2023-07-18T03:40:42.175955+0000","last_clean":"2023-07-18T03:40:42.175955+0000","last_became_active":"2023-07-18T03:40:40.969715+0000","last_became_peered":"2023-07-18T03:40:40.969715+0000","last_unstale":"2023-07-18T03:40:42.175955+0000","last_undegraded":"2023-07-18T03:40:42.175955+0000","last_fullsized":"2023-07-18T03:40:42.175955+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:38:26.570999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175777+0000","last_change":"2023-07-18T03:40:40.960847+0000","last_active":"2023-07-18T03:40:42.175777+0000","last_peered":"2023-07-18T03:40:42.175777+0000","last_clean":"2023-07-18T03:40:42.175777+0000","last_became_active":"2023-07-18T03:40:40.960592+0000","last_became_peered":"2023-07-18T03:40:40.960592+0000","last_unstale":"2023-07-18T03:40:42.175777+0000","last_undegraded":"2023-07-18T03:40:42.175777+0000","last_fullsized":"2023-07-18T03:40:42.175777+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:22:06.699892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175853+0000","last_change":"2023-07-18T03:40:40.958959+0000","last_active":"2023-07-18T03:40:42.175853+0000","last_peered":"2023-07-18T03:40:42.175853+0000","last_clean":"2023-07-18T03:40:42.175853+0000","last_became_active":"2023-07-18T03:40:40.958659+0000","last_became_peered":"2023-07-18T03:40:40.958659+0000","last_unstale":"2023-07-18T03:40:42.175853+0000","last_undegraded":"2023-07-18T03:40:42.175853+0000","last_fullsized":"2023-07-18T03:40:42.175853+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:20:30.773061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.977967+0000","last_change":"2023-07-18T03:40:40.970419+0000","last_active":"2023-07-18T03:40:40.977967+0000","last_peered":"2023-07-18T03:40:40.977967+0000","last_clean":"2023-07-18T03:40:40.977967+0000","last_became_active":"2023-07-18T03:40:40.970138+0000","last_became_peered":"2023-07-18T03:40:40.970138+0000","last_unstale":"2023-07-18T03:40:40.977967+0000","last_undegraded":"2023-07-18T03:40:40.977967+0000","last_fullsized":"2023-07-18T03:40:40.977967+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:21.834512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.956022+0000","last_change":"2023-07-18T03:40:40.961886+0000","last_active":"2023-07-18T03:40:41.956022+0000","last_peered":"2023-07-18T03:40:41.956022+0000","last_clean":"2023-07-18T03:40:41.956022+0000","last_became_active":"2023-07-18T03:40:40.961593+0000","last_became_peered":"2023-07-18T03:40:40.961593+0000","last_unstale":"2023-07-18T03:40:41.956022+0000","last_undegraded":"2023-07-18T03:40:41.956022+0000","last_fullsized":"2023-07-18T03:40:41.956022+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:03:49.133039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175046+0000","last_change":"2023-07-18T03:40:40.969148+0000","last_active":"2023-07-18T03:40:42.175046+0000","last_peered":"2023-07-18T03:40:42.175046+0000","last_clean":"2023-07-18T03:40:42.175046+0000","last_became_active":"2023-07-18T03:40:40.968878+0000","last_became_peered":"2023-07-18T03:40:40.968878+0000","last_unstale":"2023-07-18T03:40:42.175046+0000","last_undegraded":"2023-07-18T03:40:42.175046+0000","last_fullsized":"2023-07-18T03:40:42.175046+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:17:22.645893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175774+0000","last_change":"2023-07-18T03:40:40.980878+0000","last_active":"2023-07-18T03:40:42.175774+0000","last_peered":"2023-07-18T03:40:42.175774+0000","last_clean":"2023-07-18T03:40:42.175774+0000","last_became_active":"2023-07-18T03:40:40.980407+0000","last_became_peered":"2023-07-18T03:40:40.980407+0000","last_unstale":"2023-07-18T03:40:42.175774+0000","last_undegraded":"2023-07-18T03:40:42.175774+0000","last_fullsized":"2023-07-18T03:40:42.175774+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:45:07.583653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.981293+0000","last_change":"2023-07-18T03:40:40.974638+0000","last_active":"2023-07-18T03:40:40.981293+0000","last_peered":"2023-07-18T03:40:40.981293+0000","last_clean":"2023-07-18T03:40:40.981293+0000","last_became_active":"2023-07-18T03:40:40.974268+0000","last_became_peered":"2023-07-18T03:40:40.974268+0000","last_unstale":"2023-07-18T03:40:40.981293+0000","last_undegraded":"2023-07-18T03:40:40.981293+0000","last_fullsized":"2023-07-18T03:40:40.981293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:03:17.053200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955774+0000","last_change":"2023-07-18T03:40:40.973345+0000","last_active":"2023-07-18T03:40:41.955774+0000","last_peered":"2023-07-18T03:40:41.955774+0000","last_clean":"2023-07-18T03:40:41.955774+0000","last_became_active":"2023-07-18T03:40:40.973095+0000","last_became_peered":"2023-07-18T03:40:40.973095+0000","last_unstale":"2023-07-18T03:40:41.955774+0000","last_undegraded":"2023-07-18T03:40:41.955774+0000","last_fullsized":"2023-07-18T03:40:41.955774+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:23:20.968811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175956+0000","last_change":"2023-07-18T03:40:40.968398+0000","last_active":"2023-07-18T03:40:42.175956+0000","last_peered":"2023-07-18T03:40:42.175956+0000","last_clean":"2023-07-18T03:40:42.175956+0000","last_became_active":"2023-07-18T03:40:40.968140+0000","last_became_peered":"2023-07-18T03:40:40.968140+0000","last_unstale":"2023-07-18T03:40:42.175956+0000","last_undegraded":"2023-07-18T03:40:42.175956+0000","last_fullsized":"2023-07-18T03:40:42.175956+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:46:38.287766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175569+0000","last_change":"2023-07-18T03:40:40.964061+0000","last_active":"2023-07-18T03:40:42.175569+0000","last_peered":"2023-07-18T03:40:42.175569+0000","last_clean":"2023-07-18T03:40:42.175569+0000","last_became_active":"2023-07-18T03:40:40.963746+0000","last_became_peered":"2023-07-18T03:40:40.963746+0000","last_unstale":"2023-07-18T03:40:42.175569+0000","last_undegraded":"2023-07-18T03:40:42.175569+0000","last_fullsized":"2023-07-18T03:40:42.175569+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:00:25.826683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176160+0000","last_change":"2023-07-18T03:40:40.967820+0000","last_active":"2023-07-18T03:40:42.176160+0000","last_peered":"2023-07-18T03:40:42.176160+0000","last_clean":"2023-07-18T03:40:42.176160+0000","last_became_active":"2023-07-18T03:40:40.967515+0000","last_became_peered":"2023-07-18T03:40:40.967515+0000","last_unstale":"2023-07-18T03:40:42.176160+0000","last_undegraded":"2023-07-18T03:40:42.176160+0000","last_fullsized":"2023-07-18T03:40:42.176160+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:19.679560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175341+0000","last_change":"2023-07-18T03:40:40.973531+0000","last_active":"2023-07-18T03:40:42.175341+0000","last_peered":"2023-07-18T03:40:42.175341+0000","last_clean":"2023-07-18T03:40:42.175341+0000","last_became_active":"2023-07-18T03:40:40.973212+0000","last_became_peered":"2023-07-18T03:40:40.973212+0000","last_unstale":"2023-07-18T03:40:42.175341+0000","last_undegraded":"2023-07-18T03:40:42.175341+0000","last_fullsized":"2023-07-18T03:40:42.175341+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:39.170107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175173+0000","last_change":"2023-07-18T03:40:40.970303+0000","last_active":"2023-07-18T03:40:42.175173+0000","last_peered":"2023-07-18T03:40:42.175173+0000","last_clean":"2023-07-18T03:40:42.175173+0000","last_became_active":"2023-07-18T03:40:40.970041+0000","last_became_peered":"2023-07-18T03:40:40.970041+0000","last_unstale":"2023-07-18T03:40:42.175173+0000","last_undegraded":"2023-07-18T03:40:42.175173+0000","last_fullsized":"2023-07-18T03:40:42.175173+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:16:07.234893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175221+0000","last_change":"2023-07-18T03:40:40.962657+0000","last_active":"2023-07-18T03:40:42.175221+0000","last_peered":"2023-07-18T03:40:42.175221+0000","last_clean":"2023-07-18T03:40:42.175221+0000","last_became_active":"2023-07-18T03:40:40.962363+0000","last_became_peered":"2023-07-18T03:40:40.962363+0000","last_unstale":"2023-07-18T03:40:42.175221+0000","last_undegraded":"2023-07-18T03:40:42.175221+0000","last_fullsized":"2023-07-18T03:40:42.175221+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:16:10.691858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176164+0000","last_change":"2023-07-18T03:40:39.945226+0000","last_active":"2023-07-18T03:40:42.176164+0000","last_peered":"2023-07-18T03:40:42.176164+0000","last_clean":"2023-07-18T03:40:42.176164+0000","last_became_active":"2023-07-18T03:40:39.944923+0000","last_became_peered":"2023-07-18T03:40:39.944923+0000","last_unstale":"2023-07-18T03:40:42.176164+0000","last_undegraded":"2023-07-18T03:40:42.176164+0000","last_fullsized":"2023-07-18T03:40:42.176164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:28:29.317447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176528+0000","last_change":"2023-07-18T03:40:40.965946+0000","last_active":"2023-07-18T03:40:42.176528+0000","last_peered":"2023-07-18T03:40:42.176528+0000","last_clean":"2023-07-18T03:40:42.176528+0000","last_became_active":"2023-07-18T03:40:40.965616+0000","last_became_peered":"2023-07-18T03:40:40.965616+0000","last_unstale":"2023-07-18T03:40:42.176528+0000","last_undegraded":"2023-07-18T03:40:42.176528+0000","last_fullsized":"2023-07-18T03:40:42.176528+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:20:47.935526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955256+0000","last_change":"2023-07-18T03:40:39.950318+0000","last_active":"2023-07-18T03:40:41.955256+0000","last_peered":"2023-07-18T03:40:41.955256+0000","last_clean":"2023-07-18T03:40:41.955256+0000","last_became_active":"2023-07-18T03:40:39.950056+0000","last_became_peered":"2023-07-18T03:40:39.950056+0000","last_unstale":"2023-07-18T03:40:41.955256+0000","last_undegraded":"2023-07-18T03:40:41.955256+0000","last_fullsized":"2023-07-18T03:40:41.955256+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:32:17.247797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175126+0000","last_change":"2023-07-18T03:40:40.958875+0000","last_active":"2023-07-18T03:40:42.175126+0000","last_peered":"2023-07-18T03:40:42.175126+0000","last_clean":"2023-07-18T03:40:42.175126+0000","last_became_active":"2023-07-18T03:40:40.958569+0000","last_became_peered":"2023-07-18T03:40:40.958569+0000","last_unstale":"2023-07-18T03:40:42.175126+0000","last_undegraded":"2023-07-18T03:40:42.175126+0000","last_fullsized":"2023-07-18T03:40:42.175126+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:18:40.983271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955893+0000","last_change":"2023-07-18T03:40:39.943162+0000","last_active":"2023-07-18T03:40:41.955893+0000","last_peered":"2023-07-18T03:40:41.955893+0000","last_clean":"2023-07-18T03:40:41.955893+0000","last_became_active":"2023-07-18T03:40:39.942873+0000","last_became_peered":"2023-07-18T03:40:39.942873+0000","last_unstale":"2023-07-18T03:40:41.955893+0000","last_undegraded":"2023-07-18T03:40:41.955893+0000","last_fullsized":"2023-07-18T03:40:41.955893+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:11:46.013861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.981199+0000","last_change":"2023-07-18T03:40:40.976222+0000","last_active":"2023-07-18T03:40:40.981199+0000","last_peered":"2023-07-18T03:40:40.981199+0000","last_clean":"2023-07-18T03:40:40.981199+0000","last_became_active":"2023-07-18T03:40:40.975964+0000","last_became_peered":"2023-07-18T03:40:40.975964+0000","last_unstale":"2023-07-18T03:40:40.981199+0000","last_undegraded":"2023-07-18T03:40:40.981199+0000","last_fullsized":"2023-07-18T03:40:40.981199+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:29:56.985236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176637+0000","last_change":"2023-07-18T03:40:39.952921+0000","last_active":"2023-07-18T03:40:42.176637+0000","last_peered":"2023-07-18T03:40:42.176637+0000","last_clean":"2023-07-18T03:40:42.176637+0000","last_became_active":"2023-07-18T03:40:39.952687+0000","last_became_peered":"2023-07-18T03:40:39.952687+0000","last_unstale":"2023-07-18T03:40:42.176637+0000","last_undegraded":"2023-07-18T03:40:42.176637+0000","last_fullsized":"2023-07-18T03:40:42.176637+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:04:51.980579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958827+0000","last_change":"2023-07-18T03:40:40.973478+0000","last_active":"2023-07-18T03:40:41.958827+0000","last_peered":"2023-07-18T03:40:41.958827+0000","last_clean":"2023-07-18T03:40:41.958827+0000","last_became_active":"2023-07-18T03:40:40.973208+0000","last_became_peered":"2023-07-18T03:40:40.973208+0000","last_unstale":"2023-07-18T03:40:41.958827+0000","last_undegraded":"2023-07-18T03:40:41.958827+0000","last_fullsized":"2023-07-18T03:40:41.958827+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:14:33.202960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175779+0000","last_change":"2023-07-18T03:40:39.954484+0000","last_active":"2023-07-18T03:40:42.175779+0000","last_peered":"2023-07-18T03:40:42.175779+0000","last_clean":"2023-07-18T03:40:42.175779+0000","last_became_active":"2023-07-18T03:40:39.954155+0000","last_became_peered":"2023-07-18T03:40:39.954155+0000","last_unstale":"2023-07-18T03:40:42.175779+0000","last_undegraded":"2023-07-18T03:40:42.175779+0000","last_fullsized":"2023-07-18T03:40:42.175779+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:48:06.694651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176597+0000","last_change":"2023-07-18T03:40:40.966808+0000","last_active":"2023-07-18T03:40:42.176597+0000","last_peered":"2023-07-18T03:40:42.176597+0000","last_clean":"2023-07-18T03:40:42.176597+0000","last_became_active":"2023-07-18T03:40:40.966529+0000","last_became_peered":"2023-07-18T03:40:40.966529+0000","last_unstale":"2023-07-18T03:40:42.176597+0000","last_undegraded":"2023-07-18T03:40:42.176597+0000","last_fullsized":"2023-07-18T03:40:42.176597+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:02:01.209598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958529+0000","last_change":"2023-07-18T03:40:39.945758+0000","last_active":"2023-07-18T03:40:41.958529+0000","last_peered":"2023-07-18T03:40:41.958529+0000","last_clean":"2023-07-18T03:40:41.958529+0000","last_became_active":"2023-07-18T03:40:39.945472+0000","last_became_peered":"2023-07-18T03:40:39.945472+0000","last_unstale":"2023-07-18T03:40:41.958529+0000","last_undegraded":"2023-07-18T03:40:41.958529+0000","last_fullsized":"2023-07-18T03:40:41.958529+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:20.448131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.981311+0000","last_change":"2023-07-18T03:40:40.974986+0000","last_active":"2023-07-18T03:40:40.981311+0000","last_peered":"2023-07-18T03:40:40.981311+0000","last_clean":"2023-07-18T03:40:40.981311+0000","last_became_active":"2023-07-18T03:40:40.974134+0000","last_became_peered":"2023-07-18T03:40:40.974134+0000","last_unstale":"2023-07-18T03:40:40.981311+0000","last_undegraded":"2023-07-18T03:40:40.981311+0000","last_fullsized":"2023-07-18T03:40:40.981311+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:05:22.268526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175314+0000","last_change":"2023-07-18T03:40:39.945758+0000","last_active":"2023-07-18T03:40:42.175314+0000","last_peered":"2023-07-18T03:40:42.175314+0000","last_clean":"2023-07-18T03:40:42.175314+0000","last_became_active":"2023-07-18T03:40:39.945260+0000","last_became_peered":"2023-07-18T03:40:39.945260+0000","last_unstale":"2023-07-18T03:40:42.175314+0000","last_undegraded":"2023-07-18T03:40:42.175314+0000","last_fullsized":"2023-07-18T03:40:42.175314+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:27:52.130590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175770+0000","last_change":"2023-07-18T03:40:40.973616+0000","last_active":"2023-07-18T03:40:42.175770+0000","last_peered":"2023-07-18T03:40:42.175770+0000","last_clean":"2023-07-18T03:40:42.175770+0000","last_became_active":"2023-07-18T03:40:40.973351+0000","last_became_peered":"2023-07-18T03:40:40.973351+0000","last_unstale":"2023-07-18T03:40:42.175770+0000","last_undegraded":"2023-07-18T03:40:42.175770+0000","last_fullsized":"2023-07-18T03:40:42.175770+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:47:58.326201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175113+0000","last_change":"2023-07-18T03:40:39.953906+0000","last_active":"2023-07-18T03:40:42.175113+0000","last_peered":"2023-07-18T03:40:42.175113+0000","last_clean":"2023-07-18T03:40:42.175113+0000","last_became_active":"2023-07-18T03:40:39.953591+0000","last_became_peered":"2023-07-18T03:40:39.953591+0000","last_unstale":"2023-07-18T03:40:42.175113+0000","last_undegraded":"2023-07-18T03:40:42.175113+0000","last_fullsized":"2023-07-18T03:40:42.175113+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:53:21.854085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958391+0000","last_change":"2023-07-18T03:40:40.965005+0000","last_active":"2023-07-18T03:40:41.958391+0000","last_peered":"2023-07-18T03:40:41.958391+0000","last_clean":"2023-07-18T03:40:41.958391+0000","last_became_active":"2023-07-18T03:40:40.964741+0000","last_became_peered":"2023-07-18T03:40:40.964741+0000","last_unstale":"2023-07-18T03:40:41.958391+0000","last_undegraded":"2023-07-18T03:40:41.958391+0000","last_fullsized":"2023-07-18T03:40:41.958391+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:09:36.480771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175841+0000","last_change":"2023-07-18T03:40:39.943379+0000","last_active":"2023-07-18T03:40:42.175841+0000","last_peered":"2023-07-18T03:40:42.175841+0000","last_clean":"2023-07-18T03:40:42.175841+0000","last_became_active":"2023-07-18T03:40:39.943077+0000","last_became_peered":"2023-07-18T03:40:39.943077+0000","last_unstale":"2023-07-18T03:40:42.175841+0000","last_undegraded":"2023-07-18T03:40:42.175841+0000","last_fullsized":"2023-07-18T03:40:42.175841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:10:23.919030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176263+0000","last_change":"2023-07-18T03:40:40.961277+0000","last_active":"2023-07-18T03:40:42.176263+0000","last_peered":"2023-07-18T03:40:42.176263+0000","last_clean":"2023-07-18T03:40:42.176263+0000","last_became_active":"2023-07-18T03:40:40.960859+0000","last_became_peered":"2023-07-18T03:40:40.960859+0000","last_unstale":"2023-07-18T03:40:42.176263+0000","last_undegraded":"2023-07-18T03:40:42.176263+0000","last_fullsized":"2023-07-18T03:40:42.176263+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:33:51.672942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958745+0000","last_change":"2023-07-18T03:40:39.956587+0000","last_active":"2023-07-18T03:40:41.958745+0000","last_peered":"2023-07-18T03:40:41.958745+0000","last_clean":"2023-07-18T03:40:41.958745+0000","last_became_active":"2023-07-18T03:40:39.956218+0000","last_became_peered":"2023-07-18T03:40:39.956218+0000","last_unstale":"2023-07-18T03:40:41.958745+0000","last_undegraded":"2023-07-18T03:40:41.958745+0000","last_fullsized":"2023-07-18T03:40:41.958745+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:43:27.074767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175313+0000","last_change":"2023-07-18T03:40:40.964462+0000","last_active":"2023-07-18T03:40:42.175313+0000","last_peered":"2023-07-18T03:40:42.175313+0000","last_clean":"2023-07-18T03:40:42.175313+0000","last_became_active":"2023-07-18T03:40:40.964234+0000","last_became_peered":"2023-07-18T03:40:40.964234+0000","last_unstale":"2023-07-18T03:40:42.175313+0000","last_undegraded":"2023-07-18T03:40:42.175313+0000","last_fullsized":"2023-07-18T03:40:42.175313+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:07:06.495446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175994+0000","last_change":"2023-07-18T03:40:39.944342+0000","last_active":"2023-07-18T03:40:42.175994+0000","last_peered":"2023-07-18T03:40:42.175994+0000","last_clean":"2023-07-18T03:40:42.175994+0000","last_became_active":"2023-07-18T03:40:39.944095+0000","last_became_peered":"2023-07-18T03:40:39.944095+0000","last_unstale":"2023-07-18T03:40:42.175994+0000","last_undegraded":"2023-07-18T03:40:42.175994+0000","last_fullsized":"2023-07-18T03:40:42.175994+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:50.816530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958857+0000","last_change":"2023-07-18T03:40:40.965591+0000","last_active":"2023-07-18T03:40:41.958857+0000","last_peered":"2023-07-18T03:40:41.958857+0000","last_clean":"2023-07-18T03:40:41.958857+0000","last_became_active":"2023-07-18T03:40:40.965324+0000","last_became_peered":"2023-07-18T03:40:40.965324+0000","last_unstale":"2023-07-18T03:40:41.958857+0000","last_undegraded":"2023-07-18T03:40:41.958857+0000","last_fullsized":"2023-07-18T03:40:41.958857+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:57:48.782958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.983554+0000","last_change":"2023-07-18T03:40:39.950059+0000","last_active":"2023-07-18T03:40:40.983554+0000","last_peered":"2023-07-18T03:40:40.983554+0000","last_clean":"2023-07-18T03:40:40.983554+0000","last_became_active":"2023-07-18T03:40:39.949463+0000","last_became_peered":"2023-07-18T03:40:39.949463+0000","last_unstale":"2023-07-18T03:40:40.983554+0000","last_undegraded":"2023-07-18T03:40:40.983554+0000","last_fullsized":"2023-07-18T03:40:40.983554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:45:37.491688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176450+0000","last_change":"2023-07-18T03:40:40.964104+0000","last_active":"2023-07-18T03:40:42.176450+0000","last_peered":"2023-07-18T03:40:42.176450+0000","last_clean":"2023-07-18T03:40:42.176450+0000","last_became_active":"2023-07-18T03:40:40.963764+0000","last_became_peered":"2023-07-18T03:40:40.963764+0000","last_unstale":"2023-07-18T03:40:42.176450+0000","last_undegraded":"2023-07-18T03:40:42.176450+0000","last_fullsized":"2023-07-18T03:40:42.176450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:22:05.598835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175869+0000","last_change":"2023-07-18T03:40:39.954159+0000","last_active":"2023-07-18T03:40:42.175869+0000","last_peered":"2023-07-18T03:40:42.175869+0000","last_clean":"2023-07-18T03:40:42.175869+0000","last_became_active":"2023-07-18T03:40:39.953864+0000","last_became_peered":"2023-07-18T03:40:39.953864+0000","last_unstale":"2023-07-18T03:40:42.175869+0000","last_undegraded":"2023-07-18T03:40:42.175869+0000","last_fullsized":"2023-07-18T03:40:42.175869+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:23:00.111799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175846+0000","last_change":"2023-07-18T03:40:40.971335+0000","last_active":"2023-07-18T03:40:42.175846+0000","last_peered":"2023-07-18T03:40:42.175846+0000","last_clean":"2023-07-18T03:40:42.175846+0000","last_became_active":"2023-07-18T03:40:40.969388+0000","last_became_peered":"2023-07-18T03:40:40.969388+0000","last_unstale":"2023-07-18T03:40:42.175846+0000","last_undegraded":"2023-07-18T03:40:42.175846+0000","last_fullsized":"2023-07-18T03:40:42.175846+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:50:13.047904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175350+0000","last_change":"2023-07-18T03:40:39.956747+0000","last_active":"2023-07-18T03:40:42.175350+0000","last_peered":"2023-07-18T03:40:42.175350+0000","last_clean":"2023-07-18T03:40:42.175350+0000","last_became_active":"2023-07-18T03:40:39.956458+0000","last_became_peered":"2023-07-18T03:40:39.956458+0000","last_unstale":"2023-07-18T03:40:42.175350+0000","last_undegraded":"2023-07-18T03:40:42.175350+0000","last_fullsized":"2023-07-18T03:40:42.175350+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:43:11.769264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.977943+0000","last_change":"2023-07-18T03:40:40.967376+0000","last_active":"2023-07-18T03:40:40.977943+0000","last_peered":"2023-07-18T03:40:40.977943+0000","last_clean":"2023-07-18T03:40:40.977943+0000","last_became_active":"2023-07-18T03:40:40.967073+0000","last_became_peered":"2023-07-18T03:40:40.967073+0000","last_unstale":"2023-07-18T03:40:40.977943+0000","last_undegraded":"2023-07-18T03:40:40.977943+0000","last_fullsized":"2023-07-18T03:40:40.977943+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:25:31.758392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175675+0000","last_change":"2023-07-18T03:40:39.943090+0000","last_active":"2023-07-18T03:40:42.175675+0000","last_peered":"2023-07-18T03:40:42.175675+0000","last_clean":"2023-07-18T03:40:42.175675+0000","last_became_active":"2023-07-18T03:40:39.942761+0000","last_became_peered":"2023-07-18T03:40:39.942761+0000","last_unstale":"2023-07-18T03:40:42.175675+0000","last_undegraded":"2023-07-18T03:40:42.175675+0000","last_fullsized":"2023-07-18T03:40:42.175675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:22:41.585488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175600+0000","last_change":"2023-07-18T03:40:40.970998+0000","last_active":"2023-07-18T03:40:42.175600+0000","last_peered":"2023-07-18T03:40:42.175600+0000","last_clean":"2023-07-18T03:40:42.175600+0000","last_became_active":"2023-07-18T03:40:40.970167+0000","last_became_peered":"2023-07-18T03:40:40.970167+0000","last_unstale":"2023-07-18T03:40:42.175600+0000","last_undegraded":"2023-07-18T03:40:42.175600+0000","last_fullsized":"2023-07-18T03:40:42.175600+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:05:53.356207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.978514+0000","last_change":"2023-07-18T03:40:39.948252+0000","last_active":"2023-07-18T03:40:40.978514+0000","last_peered":"2023-07-18T03:40:40.978514+0000","last_clean":"2023-07-18T03:40:40.978514+0000","last_became_active":"2023-07-18T03:40:39.947975+0000","last_became_peered":"2023-07-18T03:40:39.947975+0000","last_unstale":"2023-07-18T03:40:40.978514+0000","last_undegraded":"2023-07-18T03:40:40.978514+0000","last_fullsized":"2023-07-18T03:40:40.978514+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:28:21.251676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.978569+0000","last_change":"2023-07-18T03:40:40.968117+0000","last_active":"2023-07-18T03:40:40.978569+0000","last_peered":"2023-07-18T03:40:40.978569+0000","last_clean":"2023-07-18T03:40:40.978569+0000","last_became_active":"2023-07-18T03:40:40.967596+0000","last_became_peered":"2023-07-18T03:40:40.967596+0000","last_unstale":"2023-07-18T03:40:40.978569+0000","last_undegraded":"2023-07-18T03:40:40.978569+0000","last_fullsized":"2023-07-18T03:40:40.978569+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:49:25.710002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175444+0000","last_change":"2023-07-18T03:40:39.956348+0000","last_active":"2023-07-18T03:40:42.175444+0000","last_peered":"2023-07-18T03:40:42.175444+0000","last_clean":"2023-07-18T03:40:42.175444+0000","last_became_active":"2023-07-18T03:40:39.955890+0000","last_became_peered":"2023-07-18T03:40:39.955890+0000","last_unstale":"2023-07-18T03:40:42.175444+0000","last_undegraded":"2023-07-18T03:40:42.175444+0000","last_fullsized":"2023-07-18T03:40:42.175444+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:54:13.800509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.954807+0000","last_change":"2023-07-18T03:40:40.956173+0000","last_active":"2023-07-18T03:40:41.954807+0000","last_peered":"2023-07-18T03:40:41.954807+0000","last_clean":"2023-07-18T03:40:41.954807+0000","last_became_active":"2023-07-18T03:40:40.955906+0000","last_became_peered":"2023-07-18T03:40:40.955906+0000","last_unstale":"2023-07-18T03:40:41.954807+0000","last_undegraded":"2023-07-18T03:40:41.954807+0000","last_fullsized":"2023-07-18T03:40:41.954807+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:20:49.935393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.956686+0000","last_change":"2023-07-18T03:40:39.943211+0000","last_active":"2023-07-18T03:40:41.956686+0000","last_peered":"2023-07-18T03:40:41.956686+0000","last_clean":"2023-07-18T03:40:41.956686+0000","last_became_active":"2023-07-18T03:40:39.942910+0000","last_became_peered":"2023-07-18T03:40:39.942910+0000","last_unstale":"2023-07-18T03:40:41.956686+0000","last_undegraded":"2023-07-18T03:40:41.956686+0000","last_fullsized":"2023-07-18T03:40:41.956686+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:43:27.533664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958617+0000","last_change":"2023-07-18T03:40:40.968844+0000","last_active":"2023-07-18T03:40:41.958617+0000","last_peered":"2023-07-18T03:40:41.958617+0000","last_clean":"2023-07-18T03:40:41.958617+0000","last_became_active":"2023-07-18T03:40:40.968560+0000","last_became_peered":"2023-07-18T03:40:40.968560+0000","last_unstale":"2023-07-18T03:40:41.958617+0000","last_undegraded":"2023-07-18T03:40:41.958617+0000","last_fullsized":"2023-07-18T03:40:41.958617+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:40:55.796859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176264+0000","last_change":"2023-07-18T03:40:39.946852+0000","last_active":"2023-07-18T03:40:42.176264+0000","last_peered":"2023-07-18T03:40:42.176264+0000","last_clean":"2023-07-18T03:40:42.176264+0000","last_became_active":"2023-07-18T03:40:39.946570+0000","last_became_peered":"2023-07-18T03:40:39.946570+0000","last_unstale":"2023-07-18T03:40:42.176264+0000","last_undegraded":"2023-07-18T03:40:42.176264+0000","last_fullsized":"2023-07-18T03:40:42.176264+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:03:00.670608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.954972+0000","last_change":"2023-07-18T03:40:40.962602+0000","last_active":"2023-07-18T03:40:41.954972+0000","last_peered":"2023-07-18T03:40:41.954972+0000","last_clean":"2023-07-18T03:40:41.954972+0000","last_became_active":"2023-07-18T03:40:40.962288+0000","last_became_peered":"2023-07-18T03:40:40.962288+0000","last_unstale":"2023-07-18T03:40:41.954972+0000","last_undegraded":"2023-07-18T03:40:41.954972+0000","last_fullsized":"2023-07-18T03:40:41.954972+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:20:30.773061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176145+0000","last_change":"2023-07-18T03:40:39.950796+0000","last_active":"2023-07-18T03:40:42.176145+0000","last_peered":"2023-07-18T03:40:42.176145+0000","last_clean":"2023-07-18T03:40:42.176145+0000","last_became_active":"2023-07-18T03:40:39.950502+0000","last_became_peered":"2023-07-18T03:40:39.950502+0000","last_unstale":"2023-07-18T03:40:42.176145+0000","last_undegraded":"2023-07-18T03:40:42.176145+0000","last_fullsized":"2023-07-18T03:40:42.176145+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:23:26.305569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176633+0000","last_change":"2023-07-18T03:40:40.964038+0000","last_active":"2023-07-18T03:40:42.176633+0000","last_peered":"2023-07-18T03:40:42.176633+0000","last_clean":"2023-07-18T03:40:42.176633+0000","last_became_active":"2023-07-18T03:40:40.963745+0000","last_became_peered":"2023-07-18T03:40:40.963745+0000","last_unstale":"2023-07-18T03:40:42.176633+0000","last_undegraded":"2023-07-18T03:40:42.176633+0000","last_fullsized":"2023-07-18T03:40:42.176633+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:55:37.610565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955110+0000","last_change":"2023-07-18T03:40:39.941612+0000","last_active":"2023-07-18T03:40:41.955110+0000","last_peered":"2023-07-18T03:40:41.955110+0000","last_clean":"2023-07-18T03:40:41.955110+0000","last_became_active":"2023-07-18T03:40:39.941341+0000","last_became_peered":"2023-07-18T03:40:39.941341+0000","last_unstale":"2023-07-18T03:40:41.955110+0000","last_undegraded":"2023-07-18T03:40:41.955110+0000","last_fullsized":"2023-07-18T03:40:41.955110+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:15:28.593173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175682+0000","last_change":"2023-07-18T03:40:40.959113+0000","last_active":"2023-07-18T03:40:42.175682+0000","last_peered":"2023-07-18T03:40:42.175682+0000","last_clean":"2023-07-18T03:40:42.175682+0000","last_became_active":"2023-07-18T03:40:40.958791+0000","last_became_peered":"2023-07-18T03:40:40.958791+0000","last_unstale":"2023-07-18T03:40:42.175682+0000","last_undegraded":"2023-07-18T03:40:42.175682+0000","last_fullsized":"2023-07-18T03:40:42.175682+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:20:49.935393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.956685+0000","last_change":"2023-07-18T03:40:39.949108+0000","last_active":"2023-07-18T03:40:40.956685+0000","last_peered":"2023-07-18T03:40:40.956685+0000","last_clean":"2023-07-18T03:40:40.956685+0000","last_became_active":"2023-07-18T03:40:39.948783+0000","last_became_peered":"2023-07-18T03:40:39.948783+0000","last_unstale":"2023-07-18T03:40:40.956685+0000","last_undegraded":"2023-07-18T03:40:40.956685+0000","last_fullsized":"2023-07-18T03:40:40.956685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:53:39.039261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175796+0000","last_change":"2023-07-18T03:40:40.963366+0000","last_active":"2023-07-18T03:40:42.175796+0000","last_peered":"2023-07-18T03:40:42.175796+0000","last_clean":"2023-07-18T03:40:42.175796+0000","last_became_active":"2023-07-18T03:40:40.963056+0000","last_became_peered":"2023-07-18T03:40:40.963056+0000","last_unstale":"2023-07-18T03:40:42.175796+0000","last_undegraded":"2023-07-18T03:40:42.175796+0000","last_fullsized":"2023-07-18T03:40:42.175796+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:07:25.341574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175480+0000","last_change":"2023-07-18T03:40:39.954508+0000","last_active":"2023-07-18T03:40:42.175480+0000","last_peered":"2023-07-18T03:40:42.175480+0000","last_clean":"2023-07-18T03:40:42.175480+0000","last_became_active":"2023-07-18T03:40:39.954255+0000","last_became_peered":"2023-07-18T03:40:39.954255+0000","last_unstale":"2023-07-18T03:40:42.175480+0000","last_undegraded":"2023-07-18T03:40:42.175480+0000","last_fullsized":"2023-07-18T03:40:42.175480+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:40:42.495459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955226+0000","last_change":"2023-07-18T03:40:40.966165+0000","last_active":"2023-07-18T03:40:41.955226+0000","last_peered":"2023-07-18T03:40:41.955226+0000","last_clean":"2023-07-18T03:40:41.955226+0000","last_became_active":"2023-07-18T03:40:40.965877+0000","last_became_peered":"2023-07-18T03:40:40.965877+0000","last_unstale":"2023-07-18T03:40:41.955226+0000","last_undegraded":"2023-07-18T03:40:41.955226+0000","last_fullsized":"2023-07-18T03:40:41.955226+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:28:30.323138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955844+0000","last_change":"2023-07-18T03:40:39.948052+0000","last_active":"2023-07-18T03:40:41.955844+0000","last_peered":"2023-07-18T03:40:41.955844+0000","last_clean":"2023-07-18T03:40:41.955844+0000","last_became_active":"2023-07-18T03:40:39.947595+0000","last_became_peered":"2023-07-18T03:40:39.947595+0000","last_unstale":"2023-07-18T03:40:41.955844+0000","last_undegraded":"2023-07-18T03:40:41.955844+0000","last_fullsized":"2023-07-18T03:40:41.955844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:26:53.934794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175266+0000","last_change":"2023-07-18T03:40:40.972063+0000","last_active":"2023-07-18T03:40:42.175266+0000","last_peered":"2023-07-18T03:40:42.175266+0000","last_clean":"2023-07-18T03:40:42.175266+0000","last_became_active":"2023-07-18T03:40:40.971793+0000","last_became_peered":"2023-07-18T03:40:40.971793+0000","last_unstale":"2023-07-18T03:40:42.175266+0000","last_undegraded":"2023-07-18T03:40:42.175266+0000","last_fullsized":"2023-07-18T03:40:42.175266+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:31:09.653739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.956690+0000","last_change":"2023-07-18T03:40:39.949161+0000","last_active":"2023-07-18T03:40:40.956690+0000","last_peered":"2023-07-18T03:40:40.956690+0000","last_clean":"2023-07-18T03:40:40.956690+0000","last_became_active":"2023-07-18T03:40:39.948853+0000","last_became_peered":"2023-07-18T03:40:39.948853+0000","last_unstale":"2023-07-18T03:40:40.956690+0000","last_undegraded":"2023-07-18T03:40:40.956690+0000","last_fullsized":"2023-07-18T03:40:40.956690+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:20:42.256618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958759+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:40:41.958759+0000","last_peered":"2023-07-18T03:40:41.958759+0000","last_clean":"2023-07-18T03:40:41.958759+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:40:41.958759+0000","last_undegraded":"2023-07-18T03:40:41.958759+0000","last_fullsized":"2023-07-18T03:40:41.958759+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:21:24.489457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176656+0000","last_change":"2023-07-18T03:40:40.964542+0000","last_active":"2023-07-18T03:40:42.176656+0000","last_peered":"2023-07-18T03:40:42.176656+0000","last_clean":"2023-07-18T03:40:42.176656+0000","last_became_active":"2023-07-18T03:40:40.964223+0000","last_became_peered":"2023-07-18T03:40:40.964223+0000","last_unstale":"2023-07-18T03:40:42.176656+0000","last_undegraded":"2023-07-18T03:40:42.176656+0000","last_fullsized":"2023-07-18T03:40:42.176656+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:53:07.258459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.956851+0000","last_change":"2023-07-18T03:40:39.950158+0000","last_active":"2023-07-18T03:40:40.956851+0000","last_peered":"2023-07-18T03:40:40.956851+0000","last_clean":"2023-07-18T03:40:40.956851+0000","last_became_active":"2023-07-18T03:40:39.949883+0000","last_became_peered":"2023-07-18T03:40:39.949883+0000","last_unstale":"2023-07-18T03:40:40.956851+0000","last_undegraded":"2023-07-18T03:40:40.956851+0000","last_fullsized":"2023-07-18T03:40:40.956851+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:32:34.975883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.959028+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:40:40.959028+0000","last_peered":"2023-07-18T03:40:40.959028+0000","last_clean":"2023-07-18T03:40:40.959028+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:40:40.959028+0000","last_undegraded":"2023-07-18T03:40:40.959028+0000","last_fullsized":"2023-07-18T03:40:40.959028+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:43:15.003936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.954995+0000","last_change":"2023-07-18T03:40:40.965589+0000","last_active":"2023-07-18T03:40:41.954995+0000","last_peered":"2023-07-18T03:40:41.954995+0000","last_clean":"2023-07-18T03:40:41.954995+0000","last_became_active":"2023-07-18T03:40:40.965290+0000","last_became_peered":"2023-07-18T03:40:40.965290+0000","last_unstale":"2023-07-18T03:40:41.954995+0000","last_undegraded":"2023-07-18T03:40:41.954995+0000","last_fullsized":"2023-07-18T03:40:41.954995+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:46:38.287766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176148+0000","last_change":"2023-07-18T03:40:39.942958+0000","last_active":"2023-07-18T03:40:42.176148+0000","last_peered":"2023-07-18T03:40:42.176148+0000","last_clean":"2023-07-18T03:40:42.176148+0000","last_became_active":"2023-07-18T03:40:39.942655+0000","last_became_peered":"2023-07-18T03:40:39.942655+0000","last_unstale":"2023-07-18T03:40:42.176148+0000","last_undegraded":"2023-07-18T03:40:42.176148+0000","last_fullsized":"2023-07-18T03:40:42.176148+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:15:22.492626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.955064+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:40:41.955064+0000","last_peered":"2023-07-18T03:40:41.955064+0000","last_clean":"2023-07-18T03:40:41.955064+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:40:41.955064+0000","last_undegraded":"2023-07-18T03:40:41.955064+0000","last_fullsized":"2023-07-18T03:40:41.955064+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:00:50.073596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":220,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175478+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:40:42.175478+0000","last_peered":"2023-07-18T03:40:42.175478+0000","last_clean":"2023-07-18T03:40:42.175478+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:40:42.175478+0000","last_undegraded":"2023-07-18T03:40:42.175478+0000","last_fullsized":"2023-07-18T03:40:42.175478+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:39.153889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.975518+0000","last_change":"2023-07-18T03:40:40.965001+0000","last_active":"2023-07-18T03:40:40.975518+0000","last_peered":"2023-07-18T03:40:40.975518+0000","last_clean":"2023-07-18T03:40:40.975518+0000","last_became_active":"2023-07-18T03:40:40.964710+0000","last_became_peered":"2023-07-18T03:40:40.964710+0000","last_unstale":"2023-07-18T03:40:40.975518+0000","last_undegraded":"2023-07-18T03:40:40.975518+0000","last_fullsized":"2023-07-18T03:40:40.975518+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:51:15.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175116+0000","last_change":"2023-07-18T03:40:39.953906+0000","last_active":"2023-07-18T03:40:42.175116+0000","last_peered":"2023-07-18T03:40:42.175116+0000","last_clean":"2023-07-18T03:40:42.175116+0000","last_became_active":"2023-07-18T03:40:39.953567+0000","last_became_peered":"2023-07-18T03:40:39.953567+0000","last_unstale":"2023-07-18T03:40:42.175116+0000","last_undegraded":"2023-07-18T03:40:42.175116+0000","last_fullsized":"2023-07-18T03:40:42.175116+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:00.519303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176241+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:40:42.176241+0000","last_peered":"2023-07-18T03:40:42.176241+0000","last_clean":"2023-07-18T03:40:42.176241+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:40:42.176241+0000","last_undegraded":"2023-07-18T03:40:42.176241+0000","last_fullsized":"2023-07-18T03:40:42.176241+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:26:11.213686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175093+0000","last_change":"2023-07-18T03:40:40.968119+0000","last_active":"2023-07-18T03:40:42.175093+0000","last_peered":"2023-07-18T03:40:42.175093+0000","last_clean":"2023-07-18T03:40:42.175093+0000","last_became_active":"2023-07-18T03:40:40.967788+0000","last_became_peered":"2023-07-18T03:40:40.967788+0000","last_unstale":"2023-07-18T03:40:42.175093+0000","last_undegraded":"2023-07-18T03:40:42.175093+0000","last_fullsized":"2023-07-18T03:40:42.175093+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:46:35.942082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175670+0000","last_change":"2023-07-18T03:40:39.955933+0000","last_active":"2023-07-18T03:40:42.175670+0000","last_peered":"2023-07-18T03:40:42.175670+0000","last_clean":"2023-07-18T03:40:42.175670+0000","last_became_active":"2023-07-18T03:40:39.955566+0000","last_became_peered":"2023-07-18T03:40:39.955566+0000","last_unstale":"2023-07-18T03:40:42.175670+0000","last_undegraded":"2023-07-18T03:40:42.175670+0000","last_fullsized":"2023-07-18T03:40:42.175670+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:06:20.912545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176431+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:40:42.176431+0000","last_peered":"2023-07-18T03:40:42.176431+0000","last_clean":"2023-07-18T03:40:42.176431+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:40:42.176431+0000","last_undegraded":"2023-07-18T03:40:42.176431+0000","last_fullsized":"2023-07-18T03:40:42.176431+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:27.768668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176557+0000","last_change":"2023-07-18T03:40:40.970205+0000","last_active":"2023-07-18T03:40:42.176557+0000","last_peered":"2023-07-18T03:40:42.176557+0000","last_clean":"2023-07-18T03:40:42.176557+0000","last_became_active":"2023-07-18T03:40:40.969812+0000","last_became_peered":"2023-07-18T03:40:40.969812+0000","last_unstale":"2023-07-18T03:40:42.176557+0000","last_undegraded":"2023-07-18T03:40:42.176557+0000","last_fullsized":"2023-07-18T03:40:42.176557+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:05:16.859125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176039+0000","last_change":"2023-07-18T03:40:39.956778+0000","last_active":"2023-07-18T03:40:42.176039+0000","last_peered":"2023-07-18T03:40:42.176039+0000","last_clean":"2023-07-18T03:40:42.176039+0000","last_became_active":"2023-07-18T03:40:39.956487+0000","last_became_peered":"2023-07-18T03:40:39.956487+0000","last_unstale":"2023-07-18T03:40:42.176039+0000","last_undegraded":"2023-07-18T03:40:42.176039+0000","last_fullsized":"2023-07-18T03:40:42.176039+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:15:28.593173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176713+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:40:42.176713+0000","last_peered":"2023-07-18T03:40:42.176713+0000","last_clean":"2023-07-18T03:40:42.176713+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:40:42.176713+0000","last_undegraded":"2023-07-18T03:40:42.176713+0000","last_fullsized":"2023-07-18T03:40:42.176713+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:25:03.136061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-18T03:40:40.978002+0000","last_change":"2023-07-18T03:40:40.974072+0000","last_active":"2023-07-18T03:40:40.978002+0000","last_peered":"2023-07-18T03:40:40.978002+0000","last_clean":"2023-07-18T03:40:40.978002+0000","last_became_active":"2023-07-18T03:40:40.973819+0000","last_became_peered":"2023-07-18T03:40:40.973819+0000","last_unstale":"2023-07-18T03:40:40.978002+0000","last_undegraded":"2023-07-18T03:40:40.978002+0000","last_fullsized":"2023-07-18T03:40:40.978002+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:27:14.738764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176668+0000","last_change":"2023-07-18T03:40:39.946448+0000","last_active":"2023-07-18T03:40:42.176668+0000","last_peered":"2023-07-18T03:40:42.176668+0000","last_clean":"2023-07-18T03:40:42.176668+0000","last_became_active":"2023-07-18T03:40:39.946158+0000","last_became_peered":"2023-07-18T03:40:39.946158+0000","last_unstale":"2023-07-18T03:40:42.176668+0000","last_undegraded":"2023-07-18T03:40:42.176668+0000","last_fullsized":"2023-07-18T03:40:42.176668+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:47:14.934558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175956+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:40:42.175956+0000","last_peered":"2023-07-18T03:40:42.175956+0000","last_clean":"2023-07-18T03:40:42.175956+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:40:42.175956+0000","last_undegraded":"2023-07-18T03:40:42.175956+0000","last_fullsized":"2023-07-18T03:40:42.175956+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:09:53.507284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.175435+0000","last_change":"2023-07-18T03:40:40.979329+0000","last_active":"2023-07-18T03:40:42.175435+0000","last_peered":"2023-07-18T03:40:42.175435+0000","last_clean":"2023-07-18T03:40:42.175435+0000","last_became_active":"2023-07-18T03:40:40.979114+0000","last_became_peered":"2023-07-18T03:40:40.979114+0000","last_unstale":"2023-07-18T03:40:42.175435+0000","last_undegraded":"2023-07-18T03:40:42.175435+0000","last_fullsized":"2023-07-18T03:40:42.175435+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:39.923804+0000","last_clean_scrub_stamp":"2023-07-18T03:40:39.923804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:05:36.754409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:41.958573+0000","last_change":"2023-07-18T03:40:39.949191+0000","last_active":"2023-07-18T03:40:41.958573+0000","last_peered":"2023-07-18T03:40:41.958573+0000","last_clean":"2023-07-18T03:40:41.958573+0000","last_became_active":"2023-07-18T03:40:39.948816+0000","last_became_peered":"2023-07-18T03:40:39.948816+0000","last_unstale":"2023-07-18T03:40:41.958573+0000","last_undegraded":"2023-07-18T03:40:41.958573+0000","last_fullsized":"2023-07-18T03:40:41.958573+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:40:38.918113+0000","last_clean_scrub_stamp":"2023-07-18T03:40:38.918113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:09.912282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-18T03:40:42.176170+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:40:42.176170+0000","last_peered":"2023-07-18T03:40:42.176170+0000","last_clean":"2023-07-18T03:40:42.176170+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:40:42.176170+0000","last_undegraded":"2023-07-18T03:40:42.176170+0000","last_fullsized":"2023-07-18T03:40:42.176170+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:30:06.650785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1144,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1171456,"data_stored":837345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85599999999999998,"5min":1.167,"15min":1.167},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.68700000000000006}]},{"osd":1,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71999999999999997,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.71999999999999997}]},{"osd":2,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.62},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.873,"15min":0.89500000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.2570000000000001,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.746,"15min":0.753},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1419999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0249999999999999,"5min":1.365,"15min":1.365},"last":0.67000000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.0010000000000003,"5min":4.0010000000000003,"15min":4.0010000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.495}]},{"osd":5,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.72899999999999998,"15min":0.71199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.2989999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.77700000000000002,"15min":0.76000000000000001},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.64500000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.81699999999999995,"15min":0.80000000000000004},"min":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0609999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.55800000000000005}]}]},{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":577536,"data_stored":247505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.73799999999999999,"15min":0.754},"min":{"1min":0.51400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84699999999999998,"5min":2.427,"15min":2.427},"last":0.83499999999999996}]},{"osd":1,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.84099999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76700000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.873},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95999999999999996,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.72199999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91300000000000003,"5min":1.034,"15min":1.034},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96399999999999997,"5min":1.113,"15min":1.113},"last":0.78700000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90400000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66500000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55900000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81000000000000005,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0820000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.90800000000000003}]}]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":573440,"data_stored":241786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.432,"15min":0.441},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.46300000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.505},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.58899999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60599999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.69999999999999996,"15min":0.72799999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.66100000000000003,"5min":3.956,"15min":3.956},"last":0.433},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.68400000000000005,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.755,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.58099999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.94399999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.86599999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.63200000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.92900000000000005,"5min":1.22,"15min":1.22},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.88200000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625}]},{"osd":6,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91300000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90400000000000003,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.81799999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.94599999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.79900000000000004,"15min":0.81000000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84899999999999998,"5min":1.272,"15min":1.272},"last":0.66200000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1132,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1159168,"data_stored":830728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64500000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.438},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.46100000000000002,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.434,"15min":0.435},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.60099999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40999999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.65700000000000003,"5min":0.748,"15min":0.748},"last":0.47699999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90400000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.59399999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":0.73799999999999999},"min":{"1min":0.57199999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84999999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.747,"15min":0.754},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88400000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.754,"15min":0.755},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.94999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58299999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.61499999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.01,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.626},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92000000000000004,"5min":1.042,"15min":1.042},"last":0.57099999999999995}]}]},{"osd":4,"up_from":14,"seq":60129542175,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":573440,"data_stored":247435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48599999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.47499999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.434}]},{"osd":2,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89400000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.81699999999999995,"5min":1.032,"15min":1.032},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67000000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.70399999999999996,"15min":0.71599999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.78100000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92700000000000005,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92200000000000004,"5min":0.999,"15min":0.999},"last":0.70099999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":557056,"data_stored":240200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.67500000000000004,"15min":0.63700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.68600000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.74099999999999999,"15min":0.68700000000000006},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":8.3780000000000001,"5min":8.3780000000000001,"15min":8.3780000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.74299999999999999,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.82599999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.77200000000000002,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":4.8769999999999998,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.65800000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70499999999999996,"15min":0.69499999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.70199999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.755,"15min":0.73899999999999999},"min":{"1min":0.41199999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.9690000000000003,"5min":4.9690000000000003,"15min":4.9690000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.6829999999999998,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71399999999999997,"15min":0.71899999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90800000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.83899999999999997,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.3140000000000001,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":0.56699999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.80300000000000005,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":3.7389999999999999,"5min":3.7389999999999999,"15min":3.7389999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94799999999999995,"5min":1.472,"15min":1.472},"last":0.59899999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":577536,"data_stored":246909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.47499999999999998,"15min":0.48899999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":1.389,"15min":1.389},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.64500000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.84099999999999997,"15min":0.90300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.74199999999999999,"5min":8.3040000000000003,"15min":8.3040000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.66700000000000004,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.77300000000000002,"15min":0.81899999999999995},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63100000000000001,"5min":8.0229999999999997,"15min":8.0229999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.88,"15min":0.93799999999999994},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.70999999999999996,"5min":8.359,"15min":8.359},"last":0.46600000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97699999999999998,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78400000000000003,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0129999999999999,"5min":1.429,"15min":1.429},"last":0.77900000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.117,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.754,"15min":0.755},"min":{"1min":0.623,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0920000000000001,"5min":1.266,"15min":1.266},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95199999999999996,"5min":1.752,"15min":1.752},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.626,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0449999999999999,"5min":1.454,"15min":1.454},"last":0.76100000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.58499999999999996,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.94299999999999995,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0249999999999999,"5min":1.222,"15min":1.222},"last":0.60099999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":581632,"data_stored":248417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.40300000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.503,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.498,"15min":0.497},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72499999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.46000000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.48299999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.622,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.58799999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78700000000000003,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56999999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0580000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.038,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67000000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.60799999999999998,"5min":0.504,"15min":0.504},"max":{"1min":1.0209999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.61699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:40:45.895 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-18T03:40:45.895 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-18T03:40:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.145+0000 7fb7b55ff700 1 Processor -- start 2023-07-18T03:40:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.145+0000 7fb7b55ff700 1 -- start start 2023-07-18T03:40:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.146+0000 7fb7b55ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7b00b8660 0x7fb7b00cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.146+0000 7fb7b55ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7b00cd970 0x7fb7b00b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.147+0000 7fb7b55ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b00d02a0 con 0x7fb7b00b4da0 2023-07-18T03:40:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.147+0000 7fb7b55ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b00b51c0 con 0x7fb7b00cd970 2023-07-18T03:40:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.147+0000 7fb7b55ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b00cfce0 con 0x7fb7b00b8660 2023-07-18T03:40:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.147+0000 7fb7a7fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7b00cd970 0x7fb7b00b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.147+0000 7fb7a7fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7b00cd970 0x7fb7b00b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35612/0 (socket says 172.21.15.49:35612) 2023-07-18T03:40:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.147+0000 7fb7a7fff700 1 -- 172.21.15.49:0/416065190 learned_addr learned my addr 172.21.15.49:0/416065190 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.147+0000 7fb7aeffd700 1 --2- 172.21.15.49:0/416065190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7b00b8660 0x7fb7b00cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.147+0000 7fb7a7fff700 1 -- 172.21.15.49:0/416065190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7b00b8660 msgr2=0x7fb7b00cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.147+0000 7fb7a7fff700 1 --2- 172.21.15.49:0/416065190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7b00b8660 0x7fb7b00cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.147+0000 7fb7a7fff700 1 -- 172.21.15.49:0/416065190 >> v1:172.21.15.107:6789/0 conn(0x7fb7b00b4da0 legacy=0x7fb7b01601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.147+0000 7fb7a7fff700 1 -- 172.21.15.49:0/416065190 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b00cffc0 con 0x7fb7b00cd970 2023-07-18T03:40:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.148+0000 7fb7a7fff700 1 --2- 172.21.15.49:0/416065190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7b00cd970 0x7fb7b00b4860 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb7980098f0 tx=0x7fb798009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6130972bf568a102 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.148+0000 7fb7ae7fc700 1 -- 172.21.15.49:0/416065190 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79800b8a0 con 0x7fb7b00cd970 2023-07-18T03:40:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.148+0000 7fb7ae7fc700 1 -- 172.21.15.49:0/416065190 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79800ba00 con 0x7fb7b00cd970 2023-07-18T03:40:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.148+0000 7fb7ae7fc700 1 -- 172.21.15.49:0/416065190 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79800bd90 con 0x7fb7b00cd970 2023-07-18T03:40:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.148+0000 7fb7b55ff700 1 -- 172.21.15.49:0/416065190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7b00cd970 msgr2=0x7fb7b00b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.148+0000 7fb7b55ff700 1 --2- 172.21.15.49:0/416065190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7b00cd970 0x7fb7b00b4860 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb7980098f0 tx=0x7fb798009c00 comp rx=0 tx=0).stop 2023-07-18T03:40:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.149+0000 7fb7b55ff700 1 -- 172.21.15.49:0/416065190 shutdown_connections 2023-07-18T03:40:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.149+0000 7fb7b55ff700 1 --2- 172.21.15.49:0/416065190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7b00b8660 0x7fb7b00cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.149+0000 7fb7b55ff700 1 --2- 172.21.15.49:0/416065190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7b00cd970 0x7fb7b00b4860 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.149+0000 7fb7b55ff700 1 -- 172.21.15.49:0/416065190 >> 172.21.15.49:0/416065190 conn(0x7fb7b0066860 msgr2=0x7fb7b006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.149+0000 7fb7b55ff700 1 -- 172.21.15.49:0/416065190 shutdown_connections 2023-07-18T03:40:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.149+0000 7fb7b55ff700 1 -- 172.21.15.49:0/416065190 wait complete. 2023-07-18T03:40:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.150+0000 7fb7b55ff700 1 Processor -- start 2023-07-18T03:40:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.150+0000 7fb7b55ff700 1 -- start start 2023-07-18T03:40:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.151+0000 7fb7b55ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7b00b4da0 0x7fb7b0154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.151+0000 7fb7b55ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b00b8660 0x7fb7b0159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.151+0000 7fb7b55ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7b00cd970 0x7fb7b01596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.151+0000 7fb7b55ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b0150e10 con 0x7fb7b00cd970 2023-07-18T03:40:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.151+0000 7fb7b55ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b015d970 con 0x7fb7b00b8660 2023-07-18T03:40:46.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.151+0000 7fb7b55ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b0154c80 con 0x7fb7b00b4da0 2023-07-18T03:40:46.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.152+0000 7fb7af7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7b00cd970 0x7fb7b01596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:46.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.152+0000 7fb7af7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7b00cd970 0x7fb7b01596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35622/0 (socket says 172.21.15.49:35622) 2023-07-18T03:40:46.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.152+0000 7fb7af7fe700 1 -- 172.21.15.49:0/2932632035 learned_addr learned my addr 172.21.15.49:0/2932632035 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:46.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.152+0000 7fb7a7fff700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b00b8660 0x7fb7b0159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:46.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.152+0000 7fb7aeffd700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7b00b4da0 0x7fb7b0154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:46.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.152+0000 7fb7af7fe700 1 -- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7b00b4da0 msgr2=0x7fb7b0154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:46.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.152+0000 7fb7af7fe700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7b00b4da0 0x7fb7b0154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.152+0000 7fb7af7fe700 1 -- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b00b8660 msgr2=0x7fb7b0159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.152+0000 7fb7af7fe700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b00b8660 0x7fb7b0159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.152+0000 7fb7af7fe700 1 -- 172.21.15.49:0/2932632035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798009550 con 0x7fb7b00cd970 2023-07-18T03:40:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.152+0000 7fb7af7fe700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7b00cd970 0x7fb7b01596b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800cb80 tx=0x7fb7a800ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.153+0000 7fb7a77fe700 1 -- 172.21.15.49:0/2932632035 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8003c00 con 0x7fb7b00cd970 2023-07-18T03:40:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.153+0000 7fb7a77fe700 1 -- 172.21.15.49:0/2932632035 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a8003d60 con 0x7fb7b00cd970 2023-07-18T03:40:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.153+0000 7fb7a77fe700 1 -- 172.21.15.49:0/2932632035 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800aaa0 con 0x7fb7b00cd970 2023-07-18T03:40:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.153+0000 7fb7b55ff700 1 -- 172.21.15.49:0/2932632035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b015cd60 con 0x7fb7b00cd970 2023-07-18T03:40:46.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.153+0000 7fb7b55ff700 1 -- 172.21.15.49:0/2932632035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b015d320 con 0x7fb7b00cd970 2023-07-18T03:40:46.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.154+0000 7fb7b55ff700 1 -- 172.21.15.49:0/2932632035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b015db20 con 0x7fb7b00cd970 2023-07-18T03:40:46.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.155+0000 7fb7a77fe700 1 -- 172.21.15.49:0/2932632035 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fb7a800ac00 con 0x7fb7b00cd970 2023-07-18T03:40:46.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.156+0000 7fb7a77fe700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb79006ad70 0x7fb79006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:46.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.156+0000 7fb7a77fe700 1 -- 172.21.15.49:0/2932632035 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7a8089400 con 0x7fb7b00cd970 2023-07-18T03:40:46.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.157+0000 7fb7aeffd700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb79006ad70 0x7fb79006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:46.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.157+0000 7fb7aeffd700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb79006ad70 0x7fb79006d220 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0006b20 tx=0x7fb7a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:46.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.160+0000 7fb7a77fe700 1 -- 172.21.15.49:0/2932632035 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb7b015db20 con 0x7fb7b00cd970 2023-07-18T03:40:46.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.459+0000 7fb7b55ff700 1 -- 172.21.15.49:0/2932632035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb7b015db20 con 0x7fb7b00cd970 2023-07-18T03:40:46.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.463+0000 7fb7a77fe700 1 -- 172.21.15.49:0/2932632035 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb7b015db20 con 0x7fb7b00cd970 2023-07-18T03:40:46.459 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2023-07-18T03:40:46.459 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQD+CLZkI5fMMhAAisfdJO7kEPCDAngS1ywP+g== 2023-07-18T03:40:46.459 INFO:teuthology.orchestra.run.smithi049.stdout: caps mds = "allow" 2023-07-18T03:40:46.459 INFO:teuthology.orchestra.run.smithi049.stdout: caps mon = "allow r" 2023-07-18T03:40:46.459 INFO:teuthology.orchestra.run.smithi049.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-18T03:40:46.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.465+0000 7fb7b55ff700 1 -- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb79006ad70 msgr2=0x7fb79006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.465+0000 7fb7b55ff700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb79006ad70 0x7fb79006d220 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0006b20 tx=0x7fb7a0008040 comp rx=0 tx=0).stop 2023-07-18T03:40:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.465+0000 7fb7b55ff700 1 -- 172.21.15.49:0/2932632035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7b00cd970 msgr2=0x7fb7b01596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.465+0000 7fb7b55ff700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7b00cd970 0x7fb7b01596b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800cb80 tx=0x7fb7a800ce90 comp rx=0 tx=0).stop 2023-07-18T03:40:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.466+0000 7fb7b55ff700 1 -- 172.21.15.49:0/2932632035 shutdown_connections 2023-07-18T03:40:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.466+0000 7fb7b55ff700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb79006ad70 0x7fb79006d220 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.466+0000 7fb7b55ff700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7b00cd970 0x7fb7b01596b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.466+0000 7fb7b55ff700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7b00b4da0 0x7fb7b0154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.466+0000 7fb7b55ff700 1 --2- 172.21.15.49:0/2932632035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b00b8660 0x7fb7b0159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.466+0000 7fb7b55ff700 1 -- 172.21.15.49:0/2932632035 >> 172.21.15.49:0/2932632035 conn(0x7fb7b0066860 msgr2=0x7fb7b00b7870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:46.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.466+0000 7fb7b55ff700 1 -- 172.21.15.49:0/2932632035 shutdown_connections 2023-07-18T03:40:46.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.466+0000 7fb7b55ff700 1 -- 172.21.15.49:0/2932632035 wait complete. 2023-07-18T03:40:46.462 INFO:teuthology.orchestra.run.smithi049.stderr:updated caps for client.0 2023-07-18T03:40:46.479 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:40:46.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.688+0000 7fd9413ff700 1 Processor -- start 2023-07-18T03:40:46.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.688+0000 7fd9413ff700 1 -- start start 2023-07-18T03:40:46.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.689+0000 7fd9413ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd93c0b4510 0x7fd93c0b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:46.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.689+0000 7fd9413ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd93c0d0b10 0x7fd93c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:46.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.689+0000 7fd9413ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93c0b7f50 con 0x7fd93c0b5d90 2023-07-18T03:40:46.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.689+0000 7fd9413ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd93c0b48e0 con 0x7fd93c0b4510 2023-07-18T03:40:46.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.689+0000 7fd9413ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c0d14d0 con 0x7fd93c0d0b10 2023-07-18T03:40:46.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.689+0000 7fd93a7fc700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd93c0b5d90 0x7fd93c0cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50464/0 (socket says 172.21.15.49:50464) 2023-07-18T03:40:46.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.689+0000 7fd93a7fc700 1 -- 172.21.15.49:0/3515290076 learned_addr learned my addr 172.21.15.49:0/3515290076 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:46.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.690+0000 7fd93affd700 1 --2- 172.21.15.49:0/3515290076 >> v2:172.21.15.107:3300/0 conn(0x7fd93c0b4510 0x7fd93c0b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:46.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.690+0000 7fd93b7fe700 1 --2- 172.21.15.49:0/3515290076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd93c0d0b10 0x7fd93c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:46.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.690+0000 7fd939ffb700 1 -- 172.21.15.49:0/3515290076 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030404561 0 0) 0x7fd93c0b7f50 con 0x7fd93c0b5d90 2023-07-18T03:40:46.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.690+0000 7fd939ffb700 1 -- 172.21.15.49:0/3515290076 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9240035b0 con 0x7fd93c0b5d90 2023-07-18T03:40:46.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.690+0000 7fd939ffb700 1 -- 172.21.15.49:0/3515290076 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4091025450 0 0) 0x7fd9240035b0 con 0x7fd93c0b5d90 2023-07-18T03:40:46.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.690+0000 7fd93affd700 1 -- 172.21.15.49:0/3515290076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd93c0d0b10 msgr2=0x7fd93c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:46.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.690+0000 7fd93affd700 1 --2- 172.21.15.49:0/3515290076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd93c0d0b10 0x7fd93c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.690+0000 7fd93affd700 1 -- 172.21.15.49:0/3515290076 >> v1:172.21.15.49:6789/0 conn(0x7fd93c0b5d90 legacy=0x7fd93c0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:46.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.690+0000 7fd93affd700 1 -- 172.21.15.49:0/3515290076 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c0cdca0 con 0x7fd93c0b4510 2023-07-18T03:40:46.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.690+0000 7fd93b7fe700 1 --2- 172.21.15.49:0/3515290076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd93c0d0b10 0x7fd93c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:46.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.691+0000 7fd93affd700 1 --2- 172.21.15.49:0/3515290076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd93c0b4510 0x7fd93c0b6c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009820 tx=0x7fd92c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6963d6c80dc62229 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:46.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.691+0000 7fd939ffb700 1 -- 172.21.15.49:0/3515290076 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930001670 con 0x7fd93c0b4510 2023-07-18T03:40:46.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.691+0000 7fd939ffb700 1 -- 172.21.15.49:0/3515290076 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c005c30 con 0x7fd93c0b4510 2023-07-18T03:40:46.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.691+0000 7fd939ffb700 1 -- 172.21.15.49:0/3515290076 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c004030 con 0x7fd93c0b4510 2023-07-18T03:40:46.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.691+0000 7fd9413ff700 1 -- 172.21.15.49:0/3515290076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd93c0b4510 msgr2=0x7fd93c0b6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:46.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.691+0000 7fd9413ff700 1 --2- 172.21.15.49:0/3515290076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd93c0b4510 0x7fd93c0b6c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009820 tx=0x7fd92c009b30 comp rx=0 tx=0).stop 2023-07-18T03:40:46.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.691+0000 7fd9413ff700 1 -- 172.21.15.49:0/3515290076 shutdown_connections 2023-07-18T03:40:46.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.691+0000 7fd9413ff700 1 --2- 172.21.15.49:0/3515290076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd93c0d0b10 0x7fd93c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.691+0000 7fd9413ff700 1 --2- 172.21.15.49:0/3515290076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd93c0b4510 0x7fd93c0b6c80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.692+0000 7fd9413ff700 1 -- 172.21.15.49:0/3515290076 >> 172.21.15.49:0/3515290076 conn(0x7fd93c09c1d0 msgr2=0x7fd93c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:46.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.692+0000 7fd9413ff700 1 -- 172.21.15.49:0/3515290076 shutdown_connections 2023-07-18T03:40:46.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.692+0000 7fd9413ff700 1 -- 172.21.15.49:0/3515290076 wait complete. 2023-07-18T03:40:46.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.693+0000 7fd9413ff700 1 Processor -- start 2023-07-18T03:40:46.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.694+0000 7fd9413ff700 1 -- start start 2023-07-18T03:40:46.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.694+0000 7fd9413ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd93c0b4510 0x7fd93c0cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:46.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd9413ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd93c0b5d90 0x7fd93c0a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:46.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd9413ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd93c0d0b10 0x7fd93c0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:46.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd9413ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c09cc40 con 0x7fd93c0b4510 2023-07-18T03:40:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd9413ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c1f26a0 con 0x7fd93c0d0b10 2023-07-18T03:40:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd9413ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c1f2980 con 0x7fd93c0b5d90 2023-07-18T03:40:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd93affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd93c0b4510 0x7fd93c0cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd93affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd93c0b4510 0x7fd93c0cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35630/0 (socket says 172.21.15.49:35630) 2023-07-18T03:40:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd93affd700 1 -- 172.21.15.49:0/2471545402 learned_addr learned my addr 172.21.15.49:0/2471545402 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd93a7fc700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd93c0b5d90 0x7fd93c0a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd93b7fe700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd93c0d0b10 0x7fd93c0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd93affd700 1 -- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd93c0b5d90 msgr2=0x7fd93c0a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:46.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd93affd700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd93c0b5d90 0x7fd93c0a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd93affd700 1 -- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd93c0d0b10 msgr2=0x7fd93c0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:46.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd93affd700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd93c0d0b10 0x7fd93c0cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd93affd700 1 -- 172.21.15.49:0/2471545402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd934009a70 con 0x7fd93c0b4510 2023-07-18T03:40:46.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd93b7fe700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd93c0d0b10 0x7fd93c0cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:46.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.695+0000 7fd93a7fc700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd93c0b5d90 0x7fd93c0a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:46.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.696+0000 7fd93affd700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd93c0b4510 0x7fd93c0cc3a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009b70 tx=0x7fd92c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:46.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.696+0000 7fd9237fe700 1 -- 172.21.15.49:0/2471545402 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c003bb0 con 0x7fd93c0b4510 2023-07-18T03:40:46.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.696+0000 7fd9237fe700 1 -- 172.21.15.49:0/2471545402 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c003d10 con 0x7fd93c0b4510 2023-07-18T03:40:46.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.696+0000 7fd9237fe700 1 -- 172.21.15.49:0/2471545402 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c01d670 con 0x7fd93c0b4510 2023-07-18T03:40:46.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.696+0000 7fd9413ff700 1 -- 172.21.15.49:0/2471545402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92c009430 con 0x7fd93c0b4510 2023-07-18T03:40:46.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.696+0000 7fd9413ff700 1 -- 172.21.15.49:0/2471545402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93c1f2f40 con 0x7fd93c0b4510 2023-07-18T03:40:46.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.697+0000 7fd9413ff700 1 -- 172.21.15.49:0/2471545402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93c0c6810 con 0x7fd93c0b4510 2023-07-18T03:40:46.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.698+0000 7fd9237fe700 1 -- 172.21.15.49:0/2471545402 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fd92c01d7d0 con 0x7fd93c0b4510 2023-07-18T03:40:46.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.699+0000 7fd9237fe700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd92806ad70 0x7fd92806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:46.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.699+0000 7fd9237fe700 1 -- 172.21.15.49:0/2471545402 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd92c08a730 con 0x7fd93c0b4510 2023-07-18T03:40:46.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.699+0000 7fd93a7fc700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd92806ad70 0x7fd92806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:46.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.700+0000 7fd93a7fc700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd92806ad70 0x7fd92806d220 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd930000f60 tx=0x7fd930009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:46.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.703+0000 7fd9237fe700 1 -- 172.21.15.49:0/2471545402 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd92c0558a0 con 0x7fd93c0b4510 2023-07-18T03:40:46.918 INFO:tasks.ceph.mds.c.smithi049.stderr:2023-07-18T03:40:46.924+0000 7f09e41f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-18T03:40:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.955+0000 7fd9413ff700 1 -- 172.21.15.49:0/2471545402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd93c04e260 con 0x7fd93c0b4510 2023-07-18T03:40:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.956+0000 7fd9237fe700 1 -- 172.21.15.49:0/2471545402 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fd92c058ec0 con 0x7fd93c0b4510 2023-07-18T03:40:46.952 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:46.952 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/777524677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":777524677},{"type":"v1","addr":"172.21.15.107:6837","nonce":777524677}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/2382452558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2382452558},{"type":"v1","addr":"172.21.15.49:6834","nonce":2382452558}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/2843143219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2843143219},{"type":"v1","addr":"172.21.15.107:6836","nonce":2843143219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:40:40.451705+0000","modified":"2023-07-18T03:40:43.957663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5479":{"gid":5479,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1453078087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":1453078087},{"type":"v1","addr":"172.21.15.49:6835","nonce":1453078087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-18T03:40:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.958+0000 7fd9413ff700 1 -- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd92806ad70 msgr2=0x7fd92806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.958+0000 7fd9413ff700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd92806ad70 0x7fd92806d220 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd930000f60 tx=0x7fd930009040 comp rx=0 tx=0).stop 2023-07-18T03:40:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.958+0000 7fd9413ff700 1 -- 172.21.15.49:0/2471545402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd93c0b4510 msgr2=0x7fd93c0cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.958+0000 7fd9413ff700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd93c0b4510 0x7fd93c0cc3a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009b70 tx=0x7fd92c004750 comp rx=0 tx=0).stop 2023-07-18T03:40:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.958+0000 7fd9413ff700 1 -- 172.21.15.49:0/2471545402 shutdown_connections 2023-07-18T03:40:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.958+0000 7fd9413ff700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd93c0d0b10 0x7fd93c0cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.958+0000 7fd9413ff700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd92806ad70 0x7fd92806d220 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.958+0000 7fd9413ff700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd93c0b4510 0x7fd93c0cc3a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.958+0000 7fd9413ff700 1 --2- 172.21.15.49:0/2471545402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd93c0b5d90 0x7fd93c0a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.958+0000 7fd9413ff700 1 -- 172.21.15.49:0/2471545402 >> 172.21.15.49:0/2471545402 conn(0x7fd93c09c1d0 msgr2=0x7fd93c15c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.958+0000 7fd9413ff700 1 -- 172.21.15.49:0/2471545402 shutdown_connections 2023-07-18T03:40:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:46.959+0000 7fd9413ff700 1 -- 172.21.15.49:0/2471545402 wait complete. 2023-07-18T03:40:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2023-07-18T03:40:47.374 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-18T03:40:40.451705+0000', 'modified': '2023-07-18T03:40:43.957663+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5479}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5479': {'gid': 5479, 'name': 'c', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.49:6835/1453078087', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6833', 'nonce': 1453078087}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 1453078087}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-18T03:40:47.374 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-18T03:40:47.375 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-18T03:40:47.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.586+0000 7f60933ff700 1 Processor -- start 2023-07-18T03:40:47.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.586+0000 7f60933ff700 1 -- start start 2023-07-18T03:40:47.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.587+0000 7f60933ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f608c0d0b10 0x7f608c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.587+0000 7f60933ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f608c0b0e20 0x7f608c0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.587+0000 7f60933ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c0b8e30 con 0x7f608c0b8a60 2023-07-18T03:40:47.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.587+0000 7f60933ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c0d14d0 con 0x7f608c0d0b10 2023-07-18T03:40:47.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.587+0000 7f60933ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0afaa0 con 0x7f608c0b0e20 2023-07-18T03:40:47.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.588+0000 7f609119b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f608c0b8a60 0x7f608c0b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50470/0 (socket says 172.21.15.49:50470) 2023-07-18T03:40:47.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.588+0000 7f609119b700 1 -- 172.21.15.49:0/1341226035 learned_addr learned my addr 172.21.15.49:0/1341226035 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:47.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.588+0000 7f609099a700 1 --2- 172.21.15.49:0/1341226035 >> v2:172.21.15.107:3300/0 conn(0x7f608c0d0b10 0x7f608c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.588+0000 7f609199c700 1 --2- 172.21.15.49:0/1341226035 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f608c0b0e20 0x7f608c0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.588+0000 7f6083fff700 1 -- 172.21.15.49:0/1341226035 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655083089 0 0) 0x7f608c0b8e30 con 0x7f608c0b8a60 2023-07-18T03:40:47.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.588+0000 7f6083fff700 1 -- 172.21.15.49:0/1341226035 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60780035b0 con 0x7f608c0b8a60 2023-07-18T03:40:47.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.588+0000 7f609099a700 1 -- 172.21.15.49:0/1341226035 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f608c0b0e20 msgr2=0x7f608c0b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:47.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.588+0000 7f609099a700 1 --2- 172.21.15.49:0/1341226035 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f608c0b0e20 0x7f608c0b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.588+0000 7f609099a700 1 -- 172.21.15.49:0/1341226035 >> v1:172.21.15.49:6789/0 conn(0x7f608c0b8a60 legacy=0x7f608c0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.588+0000 7f609099a700 1 -- 172.21.15.49:0/1341226035 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c0afd80 con 0x7f608c0d0b10 2023-07-18T03:40:47.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.589+0000 7f609099a700 1 --2- 172.21.15.49:0/1341226035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f608c0d0b10 0x7f608c0d0f90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f607c009840 tx=0x7f607c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f022f0734b286458 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:47.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.589+0000 7f6083fff700 1 -- 172.21.15.49:0/1341226035 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c011830 con 0x7f608c0d0b10 2023-07-18T03:40:47.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.589+0000 7f6083fff700 1 -- 172.21.15.49:0/1341226035 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c011990 con 0x7f608c0d0b10 2023-07-18T03:40:47.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.589+0000 7f6083fff700 1 -- 172.21.15.49:0/1341226035 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c011d20 con 0x7f608c0d0b10 2023-07-18T03:40:47.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.589+0000 7f60933ff700 1 -- 172.21.15.49:0/1341226035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f608c0d0b10 msgr2=0x7f608c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.589+0000 7f60933ff700 1 --2- 172.21.15.49:0/1341226035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f608c0d0b10 0x7f608c0d0f90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f607c009840 tx=0x7f607c009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:47.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.590+0000 7f60933ff700 1 -- 172.21.15.49:0/1341226035 shutdown_connections 2023-07-18T03:40:47.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.590+0000 7f60933ff700 1 --2- 172.21.15.49:0/1341226035 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f608c0b0e20 0x7f608c0b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.590+0000 7f60933ff700 1 --2- 172.21.15.49:0/1341226035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f608c0d0b10 0x7f608c0d0f90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.590+0000 7f60933ff700 1 -- 172.21.15.49:0/1341226035 >> 172.21.15.49:0/1341226035 conn(0x7f608c09c1b0 msgr2=0x7f608c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:47.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.590+0000 7f60933ff700 1 -- 172.21.15.49:0/1341226035 shutdown_connections 2023-07-18T03:40:47.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.590+0000 7f60933ff700 1 -- 172.21.15.49:0/1341226035 wait complete. 2023-07-18T03:40:47.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.591+0000 7f60933ff700 1 Processor -- start 2023-07-18T03:40:47.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.592+0000 7f60933ff700 1 -- start start 2023-07-18T03:40:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f60933ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f608c0b0e20 0x7f608c0cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f60933ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c0b8a60 0x7f608c15b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f60933ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f608c0d0b10 0x7f608c15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f60933ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0cadd0 con 0x7f608c0b8a60 2023-07-18T03:40:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f60933ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0cb0b0 con 0x7f608c0d0b10 2023-07-18T03:40:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f60933ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0cb390 con 0x7f608c0b0e20 2023-07-18T03:40:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f609099a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c0b8a60 0x7f608c15b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f609099a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c0b8a60 0x7f608c15b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35636/0 (socket says 172.21.15.49:35636) 2023-07-18T03:40:47.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f609099a700 1 -- 172.21.15.49:0/1624547097 learned_addr learned my addr 172.21.15.49:0/1624547097 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:47.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f609119b700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f608c0b0e20 0x7f608c0cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f609199c700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f608c0d0b10 0x7f608c15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f609099a700 1 -- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f608c0b0e20 msgr2=0x7f608c0cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f609099a700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f608c0b0e20 0x7f608c0cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f609099a700 1 -- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f608c0d0b10 msgr2=0x7f608c15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f609099a700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f608c0d0b10 0x7f608c15b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.593+0000 7f609099a700 1 -- 172.21.15.49:0/1624547097 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c0094a0 con 0x7f608c0b8a60 2023-07-18T03:40:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.594+0000 7f609099a700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c0b8a60 0x7f608c15b360 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f607c014040 tx=0x7f607c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.594+0000 7f6081ffb700 1 -- 172.21.15.49:0/1624547097 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c004590 con 0x7f608c0b8a60 2023-07-18T03:40:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.594+0000 7f6081ffb700 1 -- 172.21.15.49:0/1624547097 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c01f070 con 0x7f608c0b8a60 2023-07-18T03:40:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.594+0000 7f60933ff700 1 -- 172.21.15.49:0/1624547097 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c0c7100 con 0x7f608c0b8a60 2023-07-18T03:40:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.594+0000 7f6081ffb700 1 -- 172.21.15.49:0/1624547097 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c0038a0 con 0x7f608c0b8a60 2023-07-18T03:40:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.594+0000 7f60933ff700 1 -- 172.21.15.49:0/1624547097 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c0cfef0 con 0x7f608c0b8a60 2023-07-18T03:40:47.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.596+0000 7f6081ffb700 1 -- 172.21.15.49:0/1624547097 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f607c003a00 con 0x7f608c0b8a60 2023-07-18T03:40:47.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.597+0000 7f6081ffb700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f607006ad70 0x7f607006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.597+0000 7f6081ffb700 1 -- 172.21.15.49:0/1624547097 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f607c021070 con 0x7f608c0b8a60 2023-07-18T03:40:47.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.598+0000 7f609119b700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f607006ad70 0x7f607006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.599+0000 7f609119b700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f607006ad70 0x7f607006d220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6088007fd0 tx=0x7f6088007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:47.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.599+0000 7f60a6ca9b80 1 Processor -- start 2023-07-18T03:40:47.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.599+0000 7f60a6ca9b80 1 -- start start 2023-07-18T03:40:47.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.600+0000 7f60a6ca9b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f691f00 0x55f71f6926f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.600+0000 7f60a6ca9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f696960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.600+0000 7f60a6ca9b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f696ea0 0x55f71f6b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.600+0000 7f60a6ca9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f71f570c60 con 0x55f71f6922d0 2023-07-18T03:40:47.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.600+0000 7f60a6ca9b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f71f5c39f0 con 0x55f71f696ea0 2023-07-18T03:40:47.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.600+0000 7f60a6ca9b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f71f5bce80 con 0x55f71f691f00 2023-07-18T03:40:47.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.600+0000 7f609099a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f696960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.600+0000 7f609099a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f696960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35644/0 (socket says 172.21.15.49:35644) 2023-07-18T03:40:47.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.600+0000 7f609099a700 1 -- 172.21.15.49:0/3139948292 learned_addr learned my addr 172.21.15.49:0/3139948292 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:47.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f609119b700 1 --2- 172.21.15.49:0/3139948292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f691f00 0x55f71f6926f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f609199c700 1 --2- 172.21.15.49:0/3139948292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f696ea0 0x55f71f6b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f609099a700 1 -- 172.21.15.49:0/3139948292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f691f00 msgr2=0x55f71f6926f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:47.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f609099a700 1 --2- 172.21.15.49:0/3139948292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f691f00 0x55f71f6926f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f609099a700 1 -- 172.21.15.49:0/3139948292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f696ea0 msgr2=0x55f71f6b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:47.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f609099a700 1 --2- 172.21.15.49:0/3139948292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f696ea0 0x55f71f6b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f609099a700 1 -- 172.21.15.49:0/3139948292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f71f3ead90 con 0x55f71f6922d0 2023-07-18T03:40:47.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f609199c700 1 --2- 172.21.15.49:0/3139948292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f696ea0 0x55f71f6b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:47.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f609119b700 1 --2- 172.21.15.49:0/3139948292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f691f00 0x55f71f6926f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:47.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f609099a700 1 --2- 172.21.15.49:0/3139948292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f696960 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f607c061770 tx=0x7f607c0595c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f45db38296164b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:47.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f6076ffd700 1 -- 172.21.15.49:0/3139948292 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c061ca0 con 0x55f71f6922d0 2023-07-18T03:40:47.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f6076ffd700 1 -- 172.21.15.49:0/3139948292 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c0890f0 con 0x55f71f6922d0 2023-07-18T03:40:47.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.601+0000 7f6076ffd700 1 -- 172.21.15.49:0/3139948292 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c061a30 con 0x55f71f6922d0 2023-07-18T03:40:47.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.602+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3139948292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 msgr2=0x55f71f696960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.602+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/3139948292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f696960 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f607c061770 tx=0x7f607c0595c0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.602+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3139948292 shutdown_connections 2023-07-18T03:40:47.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.602+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/3139948292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f696ea0 0x55f71f6b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.602+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/3139948292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f691f00 0x55f71f6926f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.602+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/3139948292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f696960 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.602+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3139948292 >> 172.21.15.49:0/3139948292 conn(0x55f71f660b90 msgr2=0x55f71f666130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:47.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.602+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3139948292 shutdown_connections 2023-07-18T03:40:47.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.602+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3139948292 wait complete. 2023-07-18T03:40:47.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.603+0000 7f60a6ca9b80 1 -- start start 2023-07-18T03:40:47.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.603+0000 7f60a6ca9b80 1 Processor -- start 2023-07-18T03:40:47.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.604+0000 7f60a6ca9b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 0x55f71f669680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.604+0000 7f60a6ca9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f669bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.604+0000 7f60a6ca9b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f696ea0 0x55f71f6b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.604+0000 7f60a6ca9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f71f55ca70 con 0x55f71f6922d0 2023-07-18T03:40:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.604+0000 7f60a6ca9b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f71f55c770 con 0x55f71f691f00 2023-07-18T03:40:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.604+0000 7f60a6ca9b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f71f4408d0 con 0x55f71f696ea0 2023-07-18T03:40:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.604+0000 7f60a6ca9b80 10 client.0 _ll_register_callbacks cb 0x55f71f55a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.604+0000 7f609099a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f669bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.604+0000 7f609099a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f669bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35648/0 (socket says 172.21.15.49:35648) 2023-07-18T03:40:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.604+0000 7f609099a700 1 -- 172.21.15.49:0/588259698 learned_addr learned my addr 172.21.15.49:0/588259698 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f609199c700 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f696ea0 0x55f71f6b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f609119b700 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 0x55f71f669680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f609099a700 1 -- 172.21.15.49:0/588259698 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f696ea0 msgr2=0x55f71f6b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f609099a700 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f696ea0 0x55f71f6b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f609099a700 1 -- 172.21.15.49:0/588259698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 msgr2=0x55f71f669680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f609099a700 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 0x55f71f669680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f609099a700 1 -- 172.21.15.49:0/588259698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c0cfef0 con 0x55f71f6922d0 2023-07-18T03:40:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f609119b700 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 0x55f71f669680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f609099a700 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f669bc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f607c000c00 tx=0x7f607c05c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3007e598ecc65dcb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f606bfff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-07-18T03:40:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f606bfff700 1 -- 172.21.15.49:0/588259698 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c061ca0 con 0x55f71f6922d0 2023-07-18T03:40:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/588259698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f71f65e1a0 con 0x55f71f6922d0 2023-07-18T03:40:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f606bfff700 1 -- 172.21.15.49:0/588259698 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f607c05db70 con 0x55f71f6922d0 2023-07-18T03:40:47.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f606bfff700 1 -- 172.21.15.49:0/588259698 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f607c05dd30 con 0x55f71f6922d0 2023-07-18T03:40:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.605+0000 7f60a6ca9b80 10 client.5832 fetch_fsmap learned FSMap version 28 2023-07-18T03:40:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.606+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/588259698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55f71f3d3c10 con 0x55f71f6922d0 2023-07-18T03:40:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.606+0000 7f606bfff700 1 -- 172.21.15.49:0/588259698 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f71f65e1a0 con 0x55f71f6922d0 2023-07-18T03:40:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.606+0000 7f606bfff700 1 -- 172.21.15.49:0/588259698 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f607c086030 con 0x55f71f6922d0 2023-07-18T03:40:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.606+0000 7f60a6ca9b80 10 client.5832 fetch_fsmap finished waiting for FSMap version 28 2023-07-18T03:40:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.606+0000 7f60a6ca9b80 10 client.5832 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-18T03:40:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.606+0000 7f60a6ca9b80 20 client.5832 populate_metadata read hostname 'smithi049' 2023-07-18T03:40:47.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.606+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.49:6833/1453078087,v1:172.21.15.49:6835/1453078087] conn(0x55f71f68eac0 0x55f71f667fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.606+0000 7f60a6ca9b80 4 client.5832 mds_command: new command op to 5479 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:40:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.606+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/588259698 --> [v2:172.21.15.49:6833/1453078087,v1:172.21.15.49:6835/1453078087] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f71f5b5e60 con 0x55f71f68eac0 2023-07-18T03:40:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.606+0000 7f609119b700 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.49:6833/1453078087,v1:172.21.15.49:6835/1453078087] conn(0x55f71f68eac0 0x55f71f667fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.607+0000 7f609119b700 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.49:6833/1453078087,v1:172.21.15.49:6835/1453078087] conn(0x55f71f68eac0 0x55f71f667fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bc8750dfbd95461 server_cookie=7b983efca23f7328 in_seq=0 out_seq=0 2023-07-18T03:40:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.607+0000 7f606bfff700 10 client.5832 ms_handle_connect on v2:172.21.15.49:6833/1453078087 2023-07-18T03:40:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.607+0000 7f606bfff700 1 -- 172.21.15.49:0/588259698 <== mds.0 v2:172.21.15.49:6833/1453078087 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f71f5b5e60 con 0x55f71f68eac0 2023-07-18T03:40:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.607+0000 7f606bfff700 10 client.5832 handle_command_reply: tid=0 2023-07-18T03:40:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f60a6ca9b80 1 client.5832 shutdown 2023-07-18T03:40:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/588259698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 msgr2=0x55f71f669bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f669bc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f607c000c00 tx=0x7f607c05c070 comp rx=0 tx=0).stop 2023-07-18T03:40:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/588259698 shutdown_connections 2023-07-18T03:40:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.49:6833/1453078087,v1:172.21.15.49:6835/1453078087] conn(0x55f71f68eac0 0x55f71f667fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f696ea0 0x55f71f6b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 0x55f71f669680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/588259698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f669bc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/588259698 >> 172.21.15.49:0/588259698 conn(0x55f71f660b90 msgr2=0x55f71f66a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:47.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f606bfff700 0 client.5832 ms_handle_reset on v2:172.21.15.49:6833/1453078087 2023-07-18T03:40:47.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/588259698 shutdown_connections 2023-07-18T03:40:47.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/588259698 wait complete. 2023-07-18T03:40:47.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.608+0000 7f60a6ca9b80 20 client.5832 trim_cache size 0 max 16384 2023-07-18T03:40:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.627+0000 7f60a6ca9b80 1 Processor -- start 2023-07-18T03:40:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.627+0000 7f60a6ca9b80 1 -- start start 2023-07-18T03:40:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.627+0000 7f60a6ca9b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 0x55f71f695650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.627+0000 7f60a6ca9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f691110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.627+0000 7f60a6ca9b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f696ea0 0x55f71f68cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.627+0000 7f60a6ca9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f71f698060 con 0x55f71f6922d0 2023-07-18T03:40:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.627+0000 7f60a6ca9b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f71f66ce40 con 0x55f71f691f00 2023-07-18T03:40:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.628+0000 7f60a6ca9b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f71f660760 con 0x55f71f696ea0 2023-07-18T03:40:47.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.628+0000 7f609099a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f691110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.628+0000 7f609099a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f691110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35652/0 (socket says 172.21.15.49:35652) 2023-07-18T03:40:47.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.628+0000 7f609099a700 1 -- 172.21.15.49:0/2283136825 learned_addr learned my addr 172.21.15.49:0/2283136825 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:47.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.628+0000 7f609099a700 1 -- 172.21.15.49:0/2283136825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f696ea0 msgr2=0x55f71f68cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:47.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.628+0000 7f609099a700 1 --2- 172.21.15.49:0/2283136825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f696ea0 0x55f71f68cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.628+0000 7f609099a700 1 -- 172.21.15.49:0/2283136825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 msgr2=0x55f71f695650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:47.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.628+0000 7f609099a700 1 --2- 172.21.15.49:0/2283136825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 0x55f71f695650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.628+0000 7f609099a700 1 -- 172.21.15.49:0/2283136825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f71f411200 con 0x55f71f6922d0 2023-07-18T03:40:47.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.628+0000 7f609099a700 1 --2- 172.21.15.49:0/2283136825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f691110 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f607c05dfd0 tx=0x7f607c074000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4578eeecf7f4d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:47.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.629+0000 7f60777fe700 1 -- 172.21.15.49:0/2283136825 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c061ca0 con 0x55f71f6922d0 2023-07-18T03:40:47.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.629+0000 7f60777fe700 1 -- 172.21.15.49:0/2283136825 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c055880 con 0x55f71f6922d0 2023-07-18T03:40:47.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.629+0000 7f60777fe700 1 -- 172.21.15.49:0/2283136825 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c05db70 con 0x55f71f6922d0 2023-07-18T03:40:47.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.629+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/2283136825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 msgr2=0x55f71f691110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.629+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/2283136825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f691110 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f607c05dfd0 tx=0x7f607c074000 comp rx=0 tx=0).stop 2023-07-18T03:40:47.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.629+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/2283136825 shutdown_connections 2023-07-18T03:40:47.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.629+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/2283136825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f696ea0 0x55f71f68cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.629+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/2283136825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 0x55f71f695650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.629+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/2283136825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f6922d0 0x55f71f691110 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.629+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/2283136825 >> 172.21.15.49:0/2283136825 conn(0x55f71f660b90 msgr2=0x55f71f697510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:47.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.629+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/2283136825 shutdown_connections 2023-07-18T03:40:47.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.629+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/2283136825 wait complete. 2023-07-18T03:40:47.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.630+0000 7f60a6ca9b80 1 -- start start 2023-07-18T03:40:47.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.630+0000 7f60a6ca9b80 1 Processor -- start 2023-07-18T03:40:47.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.632+0000 7f60a6ca9b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 0x55f71f714ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.632+0000 7f60a6ca9b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f6922d0 0x55f71f68c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.632+0000 7f60a6ca9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f696ea0 0x55f71f6c9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.632+0000 7f60a6ca9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f71f71bdf0 con 0x55f71f696ea0 2023-07-18T03:40:47.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.632+0000 7f60a6ca9b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f71f71b9d0 con 0x55f71f691f00 2023-07-18T03:40:47.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.632+0000 7f60a6ca9b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f71f667fd0 con 0x55f71f6922d0 2023-07-18T03:40:47.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.632+0000 7f60a6ca9b80 10 client.0 _ll_register_callbacks cb 0x7f607c0592b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:47.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.632+0000 7f609199c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f696ea0 0x55f71f6c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.632+0000 7f609199c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f696ea0 0x55f71f6c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35658/0 (socket says 172.21.15.49:35658) 2023-07-18T03:40:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.632+0000 7f609199c700 1 -- 172.21.15.49:0/3775833632 learned_addr learned my addr 172.21.15.49:0/3775833632 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.633+0000 7f609119b700 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 0x55f71f714ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.633+0000 7f609099a700 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f6922d0 0x55f71f68c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.633+0000 7f609199c700 1 -- 172.21.15.49:0/3775833632 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f6922d0 msgr2=0x55f71f68c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.633+0000 7f609199c700 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f6922d0 0x55f71f68c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.633+0000 7f609199c700 1 -- 172.21.15.49:0/3775833632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 msgr2=0x55f71f714ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.633+0000 7f609199c700 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 0x55f71f714ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.633+0000 7f609199c700 1 -- 172.21.15.49:0/3775833632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f71f3d3c10 con 0x55f71f696ea0 2023-07-18T03:40:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.633+0000 7f609199c700 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f696ea0 0x55f71f6c9730 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6084004660 tx=0x7f6084004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668f471cf6d016d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.633+0000 7f606bfff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-07-18T03:40:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.633+0000 7f606bfff700 1 -- 172.21.15.49:0/3775833632 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608400e040 con 0x55f71f696ea0 2023-07-18T03:40:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3775833632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f607c0890f0 con 0x55f71f696ea0 2023-07-18T03:40:47.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f606bfff700 1 -- 172.21.15.49:0/3775833632 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f60840178e0 con 0x55f71f696ea0 2023-07-18T03:40:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f606bfff700 1 -- 172.21.15.49:0/3775833632 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6084017aa0 con 0x55f71f696ea0 2023-07-18T03:40:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f60a6ca9b80 10 client.5838 fetch_fsmap learned FSMap version 28 2023-07-18T03:40:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3775833632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55f71f3e9260 con 0x55f71f696ea0 2023-07-18T03:40:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f606bfff700 1 -- 172.21.15.49:0/3775833632 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c0890f0 con 0x55f71f696ea0 2023-07-18T03:40:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f606bfff700 1 -- 172.21.15.49:0/3775833632 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f608401d020 con 0x55f71f696ea0 2023-07-18T03:40:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f60a6ca9b80 10 client.5838 fetch_fsmap finished waiting for FSMap version 28 2023-07-18T03:40:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f60a6ca9b80 10 client.5838 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-18T03:40:47.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f60a6ca9b80 20 client.5838 populate_metadata read hostname 'smithi049' 2023-07-18T03:40:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.49:6833/1453078087,v1:172.21.15.49:6835/1453078087] conn(0x55f71f717bb0 0x55f71f717f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f60a6ca9b80 4 client.5838 mds_command: new command op to 5479 tid=0[{"prefix": "status"}] 2023-07-18T03:40:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3775833632 --> [v2:172.21.15.49:6833/1453078087,v1:172.21.15.49:6835/1453078087] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f71f578000 con 0x55f71f717bb0 2023-07-18T03:40:47.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.634+0000 7f609119b700 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.49:6833/1453078087,v1:172.21.15.49:6835/1453078087] conn(0x55f71f717bb0 0x55f71f717f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f609119b700 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.49:6833/1453078087,v1:172.21.15.49:6835/1453078087] conn(0x55f71f717bb0 0x55f71f717f80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7742fd0540d8b6ab server_cookie=4691a638ee2d170 in_seq=0 out_seq=0 2023-07-18T03:40:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f606bfff700 10 client.5838 ms_handle_connect on v2:172.21.15.49:6833/1453078087 2023-07-18T03:40:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f606bfff700 1 -- 172.21.15.49:0/3775833632 <== mds.0 v2:172.21.15.49:6833/1453078087 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55f71f578000 con 0x55f71f717bb0 2023-07-18T03:40:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f606bfff700 10 client.5838 handle_command_reply: tid=0 2023-07-18T03:40:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f60a6ca9b80 1 client.5838 shutdown 2023-07-18T03:40:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3775833632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f696ea0 msgr2=0x55f71f6c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f696ea0 0x55f71f6c9730 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6084004660 tx=0x7f6084004690 comp rx=0 tx=0).stop 2023-07-18T03:40:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3775833632 shutdown_connections 2023-07-18T03:40:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.49:6833/1453078087,v1:172.21.15.49:6835/1453078087] conn(0x55f71f717bb0 0x55f71f717f80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f71f696ea0 0x55f71f6c9730 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f71f691f00 0x55f71f714ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f606bfff700 0 client.5838 ms_handle_reset on v2:172.21.15.49:6833/1453078087 2023-07-18T03:40:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f60a6ca9b80 1 --2- 172.21.15.49:0/3775833632 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f71f6922d0 0x55f71f68c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.635+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3775833632 >> 172.21.15.49:0/3775833632 conn(0x55f71f660b90 msgr2=0x55f71f6b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:47.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3775833632 shutdown_connections 2023-07-18T03:40:47.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60a6ca9b80 1 -- 172.21.15.49:0/3775833632 wait complete. 2023-07-18T03:40:47.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60a6ca9b80 20 client.5838 trim_cache size 0 max 16384 2023-07-18T03:40:47.638 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:40:47.638 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "e859a751-7613-40d7-973c-3e47a5e301cb", 2023-07-18T03:40:47.638 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-07-18T03:40:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 5479, 2023-07-18T03:40:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-07-18T03:40:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-07-18T03:40:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-07-18T03:40:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 6.6950435539999997, 2023-07-18T03:40:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2023-07-18T03:40:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 49, 2023-07-18T03:40:47.640 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 49, 2023-07-18T03:40:47.640 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 12.435925842 2023-07-18T03:40:47.640 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:40:47.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60933ff700 1 -- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f607006ad70 msgr2=0x7f607006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60933ff700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f607006ad70 0x7f607006d220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6088007fd0 tx=0x7f6088007dc0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60933ff700 1 -- 172.21.15.49:0/1624547097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c0b8a60 msgr2=0x7f608c15b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60933ff700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c0b8a60 0x7f608c15b360 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f607c014040 tx=0x7f607c004750 comp rx=0 tx=0).stop 2023-07-18T03:40:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60933ff700 1 -- 172.21.15.49:0/1624547097 shutdown_connections 2023-07-18T03:40:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60933ff700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f608c0d0b10 0x7f608c15b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60933ff700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f608c0b0e20 0x7f608c0cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60933ff700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f607006ad70 0x7f607006d220 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60933ff700 1 --2- 172.21.15.49:0/1624547097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f608c0b8a60 0x7f608c15b360 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.636+0000 7f60933ff700 1 -- 172.21.15.49:0/1624547097 >> 172.21.15.49:0/1624547097 conn(0x7f608c09c1b0 msgr2=0x7f608c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:47.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.637+0000 7f60933ff700 1 -- 172.21.15.49:0/1624547097 shutdown_connections 2023-07-18T03:40:47.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.637+0000 7f60933ff700 1 -- 172.21.15.49:0/1624547097 wait complete. 2023-07-18T03:40:47.650 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-18T03:40:47.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.860+0000 7fa651dff700 1 Processor -- start 2023-07-18T03:40:47.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.860+0000 7fa651dff700 1 -- start start 2023-07-18T03:40:47.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.862+0000 7fa651dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa64c15c950 0x7fa64c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.862+0000 7fa651dff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa64c0d1220 0x7fa64c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.862+0000 7fa651dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c0d2ce0 con 0x7fa64c0b1e00 2023-07-18T03:40:47.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.862+0000 7fa651dff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c0b3350 con 0x7fa64c0d1220 2023-07-18T03:40:47.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.862+0000 7fa651dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c0b3630 con 0x7fa64c15c950 2023-07-18T03:40:47.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.862+0000 7fa64bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa64c0d1220 0x7fa64c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.862+0000 7fa643fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa64c0b1e00 0x7fa64c0d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50482/0 (socket says 172.21.15.49:50482) 2023-07-18T03:40:47.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.862+0000 7fa643fff700 1 -- 172.21.15.49:0/762340733 learned_addr learned my addr 172.21.15.49:0/762340733 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:47.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.862+0000 7fa64b7fe700 1 --2- 172.21.15.49:0/762340733 >> v2:172.21.15.107:3300/0 conn(0x7fa64c15c950 0x7fa64c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.863+0000 7fa64bfff700 1 -- 172.21.15.49:0/762340733 >> v2:172.21.15.107:3300/0 conn(0x7fa64c15c950 msgr2=0x7fa64c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:47.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.863+0000 7fa64bfff700 1 --2- 172.21.15.49:0/762340733 >> v2:172.21.15.107:3300/0 conn(0x7fa64c15c950 0x7fa64c0b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.863+0000 7fa64bfff700 1 -- 172.21.15.49:0/762340733 >> v1:172.21.15.49:6789/0 conn(0x7fa64c0b1e00 legacy=0x7fa64c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.863+0000 7fa64bfff700 1 -- 172.21.15.49:0/762340733 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c0b3910 con 0x7fa64c0d1220 2023-07-18T03:40:47.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.863+0000 7fa64affd700 1 -- 172.21.15.49:0/762340733 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363618737 0 0) 0x7fa64c0d2ce0 con 0x7fa64c0b1e00 2023-07-18T03:40:47.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.863+0000 7fa64bfff700 1 --2- 172.21.15.49:0/762340733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa64c0d1220 0x7fa64c0afac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa64c08d170 tx=0x7fa64400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00f15c5fc830cf5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:47.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.863+0000 7fa64affd700 1 -- 172.21.15.49:0/762340733 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400e070 con 0x7fa64c0d1220 2023-07-18T03:40:47.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.863+0000 7fa64affd700 1 -- 172.21.15.49:0/762340733 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64400bb20 con 0x7fa64c0d1220 2023-07-18T03:40:47.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.863+0000 7fa64affd700 1 -- 172.21.15.49:0/762340733 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644012430 con 0x7fa64c0d1220 2023-07-18T03:40:47.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.864+0000 7fa651dff700 1 -- 172.21.15.49:0/762340733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa64c0d1220 msgr2=0x7fa64c0afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.864+0000 7fa651dff700 1 --2- 172.21.15.49:0/762340733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa64c0d1220 0x7fa64c0afac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa64c08d170 tx=0x7fa64400b330 comp rx=0 tx=0).stop 2023-07-18T03:40:47.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.864+0000 7fa651dff700 1 -- 172.21.15.49:0/762340733 shutdown_connections 2023-07-18T03:40:47.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.864+0000 7fa651dff700 1 --2- 172.21.15.49:0/762340733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa64c0d1220 0x7fa64c0afac0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.864+0000 7fa651dff700 1 --2- 172.21.15.49:0/762340733 >> v2:172.21.15.107:3300/0 conn(0x7fa64c15c950 0x7fa64c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.864+0000 7fa651dff700 1 -- 172.21.15.49:0/762340733 >> 172.21.15.49:0/762340733 conn(0x7fa64c09c1d0 msgr2=0x7fa64c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:47.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.864+0000 7fa651dff700 1 -- 172.21.15.49:0/762340733 shutdown_connections 2023-07-18T03:40:47.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.864+0000 7fa651dff700 1 -- 172.21.15.49:0/762340733 wait complete. 2023-07-18T03:40:47.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.865+0000 7fa651dff700 1 Processor -- start 2023-07-18T03:40:47.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.866+0000 7fa651dff700 1 -- start start 2023-07-18T03:40:47.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa651dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa64c0b1e00 0x7fa64c0c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa651dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa64c0d1220 0x7fa64c0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa651dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa64c15c950 0x7fa64c0c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa651dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0c98f0 con 0x7fa64c15c950 2023-07-18T03:40:47.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa651dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0cf740 con 0x7fa64c0d1220 2023-07-18T03:40:47.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa651dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c151290 con 0x7fa64c0b1e00 2023-07-18T03:40:47.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa64bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa64c15c950 0x7fa64c0c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa64bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa64c15c950 0x7fa64c0c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35672/0 (socket says 172.21.15.49:35672) 2023-07-18T03:40:47.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa64bfff700 1 -- 172.21.15.49:0/3537401363 learned_addr learned my addr 172.21.15.49:0/3537401363 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:47.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa64b7fe700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa64c0b1e00 0x7fa64c0c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa643fff700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa64c0d1220 0x7fa64c0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa64bfff700 1 -- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa64c0b1e00 msgr2=0x7fa64c0c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa64bfff700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa64c0b1e00 0x7fa64c0c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa64bfff700 1 -- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa64c0d1220 msgr2=0x7fa64c0c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:47.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa64bfff700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa64c0d1220 0x7fa64c0c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:47.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.867+0000 7fa64bfff700 1 -- 172.21.15.49:0/3537401363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa644009a70 con 0x7fa64c15c950 2023-07-18T03:40:47.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.868+0000 7fa64bfff700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa64c15c950 0x7fa64c0c74f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa644013000 tx=0x7fa644004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:47.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.868+0000 7fa648ff9700 1 -- 172.21.15.49:0/3537401363 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644012430 con 0x7fa64c15c950 2023-07-18T03:40:47.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.868+0000 7fa648ff9700 1 -- 172.21.15.49:0/3537401363 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa644020070 con 0x7fa64c15c950 2023-07-18T03:40:47.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.868+0000 7fa651dff700 1 -- 172.21.15.49:0/3537401363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c0cc700 con 0x7fa64c15c950 2023-07-18T03:40:47.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.868+0000 7fa648ff9700 1 -- 172.21.15.49:0/3537401363 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644004280 con 0x7fa64c15c950 2023-07-18T03:40:47.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.868+0000 7fa651dff700 1 -- 172.21.15.49:0/3537401363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c0cf950 con 0x7fa64c15c950 2023-07-18T03:40:47.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.869+0000 7fa651dff700 1 -- 172.21.15.49:0/3537401363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c04e280 con 0x7fa64c15c950 2023-07-18T03:40:47.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.870+0000 7fa648ff9700 1 -- 172.21.15.49:0/3537401363 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fa6440043e0 con 0x7fa64c15c950 2023-07-18T03:40:47.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.871+0000 7fa648ff9700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa62c06ad70 0x7fa62c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.871+0000 7fa648ff9700 1 -- 172.21.15.49:0/3537401363 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa644022070 con 0x7fa64c15c950 2023-07-18T03:40:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.872+0000 7fa64b7fe700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa62c06ad70 0x7fa62c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.872+0000 7fa64b7fe700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa62c06ad70 0x7fa62c06d220 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa64c151dc0 tx=0x7fa63c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:47.875+0000 7fa648ff9700 1 -- 172.21.15.49:0/3537401363 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa6440562d0 con 0x7fa64c15c950 2023-07-18T03:40:48.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.130+0000 7fa651dff700 1 -- 172.21.15.49:0/3537401363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa64c1e7120 con 0x7fa64c15c950 2023-07-18T03:40:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.131+0000 7fa648ff9700 1 -- 172.21.15.49:0/3537401363 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa6440598f0 con 0x7fa64c15c950 2023-07-18T03:40:48.126 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-18T03:40:48.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.133+0000 7fa651dff700 1 -- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa62c06ad70 msgr2=0x7fa62c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:48.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.133+0000 7fa651dff700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa62c06ad70 0x7fa62c06d220 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa64c151dc0 tx=0x7fa63c008040 comp rx=0 tx=0).stop 2023-07-18T03:40:48.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.133+0000 7fa651dff700 1 -- 172.21.15.49:0/3537401363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa64c15c950 msgr2=0x7fa64c0c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:48.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.133+0000 7fa651dff700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa64c15c950 0x7fa64c0c74f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa644013000 tx=0x7fa644004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:48.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.133+0000 7fa651dff700 1 -- 172.21.15.49:0/3537401363 shutdown_connections 2023-07-18T03:40:48.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.133+0000 7fa651dff700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa62c06ad70 0x7fa62c06d220 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:48.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.133+0000 7fa651dff700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa64c15c950 0x7fa64c0c74f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:48.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.133+0000 7fa651dff700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa64c0b1e00 0x7fa64c0c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:48.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.133+0000 7fa651dff700 1 --2- 172.21.15.49:0/3537401363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa64c0d1220 0x7fa64c0c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:48.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.133+0000 7fa651dff700 1 -- 172.21.15.49:0/3537401363 >> 172.21.15.49:0/3537401363 conn(0x7fa64c09c1d0 msgr2=0x7fa64c0d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:48.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.134+0000 7fa651dff700 1 -- 172.21.15.49:0/3537401363 shutdown_connections 2023-07-18T03:40:48.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:48.134+0000 7fa651dff700 1 -- 172.21.15.49:0/3537401363 wait complete. 2023-07-18T03:40:48.147 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-18T03:40:48.147 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-18T03:40:48.148 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-07-18T03:40:48.148 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-18T03:40:48.149 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-18T03:40:48.149 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-18T03:40:48.149 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-07-18T03:40:48.149 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-18T03:40:48.150 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-18T03:40:48.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:48.151 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-07-18T03:40:48.196 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:40:48.197 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:40:48.197 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:40:48.197 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:48.197 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-07-18T03:40:48.197 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:48.197 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:40:48.198 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:48.198 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-07-18T03:40:48.198 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:40:48.198 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:48.198 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-07-18T03:40:48.198 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-18T03:40:48.198 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:48.199 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-07-18T03:40:48.199 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-18T03:40:48.199 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 41860sec preferred_lft 41860sec 2023-07-18T03:40:48.199 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-07-18T03:40:48.199 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:48.200 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:40:48.200 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:48.200 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-07-18T03:40:48.200 INFO:teuthology.orchestra.run.smithi049.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-18T03:40:48.200 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 16:94:d3:1e:3e:91 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:48.201 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-18T03:40:48.201 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:48.201 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::463:84ff:fe79:c51d/64 scope link 2023-07-18T03:40:48.201 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:48.201 INFO:teuthology.orchestra.run.smithi049.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-18T03:40:48.201 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 16:94:d3:1e:3e:91 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-18T03:40:48.201 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::1494:d3ff:fe1e:3e91/64 scope link 2023-07-18T03:40:48.202 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:48.202 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-18T03:40:48.202 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether ca:3a:27:0d:fe:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-18T03:40:48.202 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c83a:27ff:fe0d:fe50/64 scope link 2023-07-18T03:40:48.202 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:48.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:48.203 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-18T03:40:48.306 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-07-18T03:40:48.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:48.310 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-07-18T03:40:48.358 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-07-18T03:40:48.358 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-18T03:40:48.358 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-18T03:40:48.359 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-18T03:40:48.359 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-18T03:40:48.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:48.360 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-07-18T03:40:48.361 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-07-18T03:40:48.361 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-18T03:40:48.361 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-18T03:40:48.362 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-18T03:40:48.363 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-07-18T03:40:48.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:48.540 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-07-18T03:40:48.585 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-18T03:40:48.585 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-18T03:40:48.585 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-18T03:40:48.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:48.587 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-07-18T03:40:48.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:40:48.697 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-18T03:40:48.744 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-18T03:40:48.745 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:40:48.746 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-18T03:40:48.828 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-07-18T03:40:48.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:48.830 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:40:48.980 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:40:48.980 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:40:49.010 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:40:49.011 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:40:49.011 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2023-07-18T03:40:49.011 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with new device syntax: 0@e859a751-7613-40d7-973c-3e47a5e301cb.cephfs=/ 2023-07-18T03:40:49.011 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.49:3300/172.21.15.107:3300/172.21.15.107:3301" will pass to kernel 2023-07-18T03:40:49.015 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-18T03:40:49.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:49.016 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-18T03:40:49.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:49.089 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:40:49.190 INFO:teuthology.orchestra.run.smithi049.stdout:client5195 2023-07-18T03:40:49.190 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:40:49.191 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:49.236 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-18T03:40:49.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.486+0000 7f4c3e1ff700 1 Processor -- start 2023-07-18T03:40:49.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.486+0000 7f4c3e1ff700 1 -- start start 2023-07-18T03:40:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.487+0000 7f4c3e1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4c3815c710 0x7f4c380cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c3e1ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4c3806db30 0x7f4c3806a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:49.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c3e1ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c380cd320 con 0x7f4c38069f30 2023-07-18T03:40:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c3e1ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c380ce750 con 0x7f4c3806db30 2023-07-18T03:40:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c3e1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c380cea30 con 0x7f4c3815c710 2023-07-18T03:40:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c37fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4c3806db30 0x7f4c3806a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c36ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4c38069f30 0x7f4c3806d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33310/0 (socket says 172.21.15.49:33310) 2023-07-18T03:40:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c36ffd700 1 -- 172.21.15.49:0/3146419770 learned_addr learned my addr 172.21.15.49:0/3146419770 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c377fe700 1 --2- 172.21.15.49:0/3146419770 >> v2:172.21.15.107:3300/0 conn(0x7f4c3815c710 0x7f4c380cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c37fff700 1 -- 172.21.15.49:0/3146419770 >> v2:172.21.15.107:3300/0 conn(0x7f4c3815c710 msgr2=0x7f4c380cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:49.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c37fff700 1 --2- 172.21.15.49:0/3146419770 >> v2:172.21.15.107:3300/0 conn(0x7f4c3815c710 0x7f4c380cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:49.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c37fff700 1 -- 172.21.15.49:0/3146419770 >> v1:172.21.15.49:6789/0 conn(0x7f4c38069f30 legacy=0x7f4c3806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:49.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c37fff700 1 -- 172.21.15.49:0/3146419770 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c380ced10 con 0x7f4c3806db30 2023-07-18T03:40:49.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.488+0000 7f4c367fc700 1 -- 172.21.15.49:0/3146419770 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278807964 0 0) 0x7f4c380cd320 con 0x7f4c38069f30 2023-07-18T03:40:49.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.489+0000 7f4c37fff700 1 --2- 172.21.15.49:0/3146419770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c3806db30 0x7f4c3806a3a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4c38072690 tx=0x7f4c3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d365fa4dca93ade1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:49.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.489+0000 7f4c367fc700 1 -- 172.21.15.49:0/3146419770 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3000e070 con 0x7f4c3806db30 2023-07-18T03:40:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.489+0000 7f4c367fc700 1 -- 172.21.15.49:0/3146419770 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3000bbd0 con 0x7f4c3806db30 2023-07-18T03:40:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.489+0000 7f4c367fc700 1 -- 172.21.15.49:0/3146419770 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30012430 con 0x7f4c3806db30 2023-07-18T03:40:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.489+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/3146419770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c3806db30 msgr2=0x7f4c3806a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.489+0000 7f4c3e1ff700 1 --2- 172.21.15.49:0/3146419770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c3806db30 0x7f4c3806a3a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4c38072690 tx=0x7f4c3000b330 comp rx=0 tx=0).stop 2023-07-18T03:40:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.490+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/3146419770 shutdown_connections 2023-07-18T03:40:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.490+0000 7f4c3e1ff700 1 --2- 172.21.15.49:0/3146419770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c3806db30 0x7f4c3806a3a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.490+0000 7f4c3e1ff700 1 --2- 172.21.15.49:0/3146419770 >> v2:172.21.15.107:3300/0 conn(0x7f4c3815c710 0x7f4c380cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.490+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/3146419770 >> 172.21.15.49:0/3146419770 conn(0x7f4c380b16c0 msgr2=0x7f4c380b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:49.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.490+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/3146419770 shutdown_connections 2023-07-18T03:40:49.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.490+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/3146419770 wait complete. 2023-07-18T03:40:49.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.492+0000 7f4c3e1ff700 1 Processor -- start 2023-07-18T03:40:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.492+0000 7f4c3e1ff700 1 -- start start 2023-07-18T03:40:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c3e1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c38069f30 0x7f4c380b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c3e1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c3806db30 0x7f4c381599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c3e1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c38159f00 0x7f4c380b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c3e1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3815e3f0 con 0x7f4c38069f30 2023-07-18T03:40:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c3e1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c380b5aa0 con 0x7f4c3806db30 2023-07-18T03:40:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c3e1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3815d770 con 0x7f4c38159f00 2023-07-18T03:40:49.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c377fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c38069f30 0x7f4c380b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c377fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c38069f30 0x7f4c380b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53966/0 (socket says 172.21.15.49:53966) 2023-07-18T03:40:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c377fe700 1 -- 172.21.15.49:0/1583967866 learned_addr learned my addr 172.21.15.49:0/1583967866 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c37fff700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c38159f00 0x7f4c380b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c36ffd700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c3806db30 0x7f4c381599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c377fe700 1 -- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c38159f00 msgr2=0x7f4c380b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c377fe700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c38159f00 0x7f4c380b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:49.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c377fe700 1 -- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c3806db30 msgr2=0x7f4c381599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c377fe700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c3806db30 0x7f4c381599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.493+0000 7f4c377fe700 1 -- 172.21.15.49:0/1583967866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c30009a70 con 0x7f4c38069f30 2023-07-18T03:40:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.494+0000 7f4c377fe700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c38069f30 0x7f4c380b5560 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800d800 tx=0x7f4c2800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.494+0000 7f4c1ffff700 1 -- 172.21.15.49:0/1583967866 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c280125f0 con 0x7f4c38069f30 2023-07-18T03:40:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.494+0000 7f4c1ffff700 1 -- 172.21.15.49:0/1583967866 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c28012750 con 0x7f4c38069f30 2023-07-18T03:40:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.494+0000 7f4c1ffff700 1 -- 172.21.15.49:0/1583967866 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c28012b00 con 0x7f4c38069f30 2023-07-18T03:40:49.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.494+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/1583967866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3806ac20 con 0x7f4c38069f30 2023-07-18T03:40:49.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.494+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/1583967866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3815dd30 con 0x7f4c38069f30 2023-07-18T03:40:49.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.495+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/1583967866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3804e280 con 0x7f4c38069f30 2023-07-18T03:40:49.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.496+0000 7f4c1ffff700 1 -- 172.21.15.49:0/1583967866 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f4c28012c60 con 0x7f4c38069f30 2023-07-18T03:40:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.497+0000 7f4c1ffff700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4c2406ada0 0x7f4c2406d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.497+0000 7f4c1ffff700 1 -- 172.21.15.49:0/1583967866 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c280897e0 con 0x7f4c38069f30 2023-07-18T03:40:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.498+0000 7f4c36ffd700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4c2406ada0 0x7f4c2406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:49.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.498+0000 7f4c36ffd700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4c2406ada0 0x7f4c2406d250 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4c3815a2f0 tx=0x7f4c2c007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:49.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.501+0000 7f4c1ffff700 1 -- 172.21.15.49:0/1583967866 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4c280548a0 con 0x7f4c38069f30 2023-07-18T03:40:49.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.755+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/1583967866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4c3806a3a0 con 0x7f4c38069f30 2023-07-18T03:40:49.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.756+0000 7f4c1ffff700 1 -- 172.21.15.49:0/1583967866 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4c28057ec0 con 0x7f4c38069f30 2023-07-18T03:40:49.751 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-18T03:40:49.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.758+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4c2406ada0 msgr2=0x7f4c2406d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:49.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.758+0000 7f4c3e1ff700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4c2406ada0 0x7f4c2406d250 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4c3815a2f0 tx=0x7f4c2c007be0 comp rx=0 tx=0).stop 2023-07-18T03:40:49.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.758+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/1583967866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c38069f30 msgr2=0x7f4c380b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.758+0000 7f4c3e1ff700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c38069f30 0x7f4c380b5560 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800d800 tx=0x7f4c2800dbc0 comp rx=0 tx=0).stop 2023-07-18T03:40:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.758+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/1583967866 shutdown_connections 2023-07-18T03:40:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.758+0000 7f4c3e1ff700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c38159f00 0x7f4c380b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.758+0000 7f4c3e1ff700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4c2406ada0 0x7f4c2406d250 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.758+0000 7f4c3e1ff700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c38069f30 0x7f4c380b5560 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:49.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.758+0000 7f4c3e1ff700 1 --2- 172.21.15.49:0/1583967866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c3806db30 0x7f4c381599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.758+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/1583967866 >> 172.21.15.49:0/1583967866 conn(0x7f4c380b16c0 msgr2=0x7f4c380cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.758+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/1583967866 shutdown_connections 2023-07-18T03:40:49.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:49.758+0000 7f4c3e1ff700 1 -- 172.21.15.49:0/1583967866 wait complete. 2023-07-18T03:40:49.772 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-18T03:40:49.772 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-18T03:40:49.773 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2023-07-18T03:40:49.773 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-18T03:40:49.774 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-18T03:40:49.774 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-18T03:40:49.775 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-07-18T03:40:49.775 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-18T03:40:49.776 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-18T03:40:49.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:49.777 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2023-07-18T03:40:49.827 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:40:49.828 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:40:49.828 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:40:49.828 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:49.828 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2023-07-18T03:40:49.828 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:49.829 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:40:49.829 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:49.829 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2023-07-18T03:40:49.829 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:40:49.829 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:49.829 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2023-07-18T03:40:49.829 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:40:49.830 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:49.830 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-18T03:40:49.830 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:49.830 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-18T03:40:49.830 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 41843sec preferred_lft 41843sec 2023-07-18T03:40:49.830 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::9059:9611:e3b2:9f28/64 scope link noprefixroute 2023-07-18T03:40:49.830 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:49.830 INFO:teuthology.orchestra.run.smithi107.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-18T03:40:49.831 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 3a:61:9e:d0:ce:0b brd ff:ff:ff:ff:ff:ff 2023-07-18T03:40:49.831 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-18T03:40:49.831 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:49.831 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::60b5:e0ff:fe5c:981b/64 scope link 2023-07-18T03:40:49.831 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:49.831 INFO:teuthology.orchestra.run.smithi107.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-18T03:40:49.831 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether d2:89:6b:8e:5e:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-18T03:40:49.831 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::d089:6bff:fe8e:5eaf/64 scope link 2023-07-18T03:40:49.832 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:49.832 INFO:teuthology.orchestra.run.smithi107.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-18T03:40:49.832 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 3a:61:9e:d0:ce:0b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-18T03:40:49.832 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::3861:9eff:fed0:ce0b/64 scope link 2023-07-18T03:40:49.832 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:40:49.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:49.834 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-18T03:40:49.902 INFO:teuthology.orchestra.run.smithi107.stdout:1 2023-07-18T03:40:49.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:49.906 DEBUG:teuthology.orchestra.run.smithi107:> route 2023-07-18T03:40:49.954 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2023-07-18T03:40:49.954 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-18T03:40:49.955 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-18T03:40:49.955 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-18T03:40:49.955 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-18T03:40:49.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:49.956 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-18T03:40:49.956 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-18T03:40:49.957 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-18T03:40:49.957 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-18T03:40:49.957 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-18T03:40:49.958 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-18T03:40:50.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:50.132 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-18T03:40:50.179 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-18T03:40:50.180 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-18T03:40:50.181 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-18T03:40:50.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:50.181 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2023-07-18T03:40:50.295 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:40:50.296 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-18T03:40:50.340 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-18T03:40:50.341 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:40:50.342 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-18T03:40:50.423 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-07-18T03:40:50.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:50.425 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:40:50.567 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:40:50.567 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:40:50.597 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-18T03:40:50.597 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-18T03:40:50.597 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.107:3300,172.21.15.107:3301" 2023-07-18T03:40:50.598 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with new device syntax: 1@e859a751-7613-40d7-973c-3e47a5e301cb.cephfs=/ 2023-07-18T03:40:50.598 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.49:3300/172.21.15.107:3300/172.21.15.107:3301" will pass to kernel 2023-07-18T03:40:50.602 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-18T03:40:50.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:50.604 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-18T03:40:50.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:50.673 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:40:50.758 INFO:teuthology.orchestra.run.smithi107.stdout:client5201 2023-07-18T03:40:50.759 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:40:50.759 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:50.803 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-07-18T03:40:51.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.029+0000 7fd02cdff700 1 Processor -- start 2023-07-18T03:40:51.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.029+0000 7fd02cdff700 1 -- start start 2023-07-18T03:40:51.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.030+0000 7fd02cdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02806b750 0x7fd0280cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:51.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.030+0000 7fd02cdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd0280cc800 0x7fd0280b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:51.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.030+0000 7fd02cdff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0280ad830 con 0x7fd02815c8f0 2023-07-18T03:40:51.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.030+0000 7fd02cdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0280b0f70 con 0x7fd0280cc800 2023-07-18T03:40:51.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.030+0000 7fd02cdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0280cd090 con 0x7fd02806b750 2023-07-18T03:40:51.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.030+0000 7fd02659c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd02815c8f0 0x7fd02806b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33326/0 (socket says 172.21.15.49:33326) 2023-07-18T03:40:51.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.030+0000 7fd02659c700 1 -- 172.21.15.49:0/1697293142 learned_addr learned my addr 172.21.15.49:0/1697293142 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:51.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd026d9d700 1 --2- 172.21.15.49:0/1697293142 >> v2:172.21.15.107:3300/0 conn(0x7fd0280cc800 0x7fd0280b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:51.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd01ffff700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02806b750 0x7fd0280cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:51.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023041143 0 0) 0x7fd0280ad830 con 0x7fd02815c8f0 2023-07-18T03:40:51.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0140035b0 con 0x7fd02815c8f0 2023-07-18T03:40:51.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3893639992 0 0) 0x7fd0140035b0 con 0x7fd02815c8f0 2023-07-18T03:40:51.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02806b750 msgr2=0x7fd0280cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:51.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02806b750 0x7fd0280cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:51.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 >> v2:172.21.15.107:3300/0 conn(0x7fd0280cc800 msgr2=0x7fd0280b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:51.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 --2- 172.21.15.49:0/1697293142 >> v2:172.21.15.107:3300/0 conn(0x7fd0280cc800 0x7fd0280b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:51.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0280b0640 con 0x7fd02815c8f0 2023-07-18T03:40:51.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7fd018001670 con 0x7fd02815c8f0 2023-07-18T03:40:51.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 >> v1:172.21.15.49:6789/0 conn(0x7fd02815c8f0 legacy=0x7fd02806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:51.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd014003ac0 0x7fd014003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:51.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02806b750 0x7fd014004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:51.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd014004d90 0x7fd014005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:51.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0280b0f70 con 0x7fd014004d90 2023-07-18T03:40:51.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0280cd090 con 0x7fd014003ac0 2023-07-18T03:40:51.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.031+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01400ba10 con 0x7fd02806b750 2023-07-18T03:40:51.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.032+0000 7fd02659c700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd014003ac0 0x7fd014003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:51.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.032+0000 7fd01ffff700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02806b750 0x7fd014004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:51.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.032+0000 7fd026d9d700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd014004d90 0x7fd014005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:51.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.032+0000 7fd02659c700 1 -- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02806b750 msgr2=0x7fd014004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:51.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.032+0000 7fd02659c700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02806b750 0x7fd014004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:51.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.032+0000 7fd02659c700 1 -- 172.21.15.49:0/1697293142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd014004d90 msgr2=0x7fd014005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:51.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.032+0000 7fd02659c700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd014004d90 0x7fd014005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:51.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.032+0000 7fd02659c700 1 -- 172.21.15.49:0/1697293142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0140035b0 con 0x7fd014003ac0 2023-07-18T03:40:51.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.032+0000 7fd01ffff700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02806b750 0x7fd014004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:51.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.032+0000 7fd02659c700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd014003ac0 0x7fd014003ed0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd01800c370 tx=0x7fd014005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=667cf91113956349 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:51.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.033+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018007c60 con 0x7fd014003ac0 2023-07-18T03:40:51.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.033+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd018005960 con 0x7fd014003ac0 2023-07-18T03:40:51.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.033+0000 7fd025d9b700 1 -- 172.21.15.49:0/1697293142 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018005d10 con 0x7fd014003ac0 2023-07-18T03:40:51.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.033+0000 7fd02cdff700 1 -- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd014003ac0 msgr2=0x7fd014003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:51.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.033+0000 7fd02cdff700 1 --2- 172.21.15.49:0/1697293142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd014003ac0 0x7fd014003ed0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd01800c370 tx=0x7fd014005880 comp rx=0 tx=0).stop 2023-07-18T03:40:51.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.033+0000 7fd026d9d700 1 -- 172.21.15.49:0/1697293142 reap_dead start 2023-07-18T03:40:51.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.033+0000 7fd02cdff700 1 -- 172.21.15.49:0/1697293142 shutdown_connections 2023-07-18T03:40:51.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.033+0000 7fd02cdff700 1 -- 172.21.15.49:0/1697293142 >> 172.21.15.49:0/1697293142 conn(0x7fd0280664f0 msgr2=0x7fd028099f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:51.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.033+0000 7fd02cdff700 1 -- 172.21.15.49:0/1697293142 shutdown_connections 2023-07-18T03:40:51.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.033+0000 7fd02cdff700 1 -- 172.21.15.49:0/1697293142 wait complete. 2023-07-18T03:40:51.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.035+0000 7fd02cdff700 1 Processor -- start 2023-07-18T03:40:51.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.035+0000 7fd02cdff700 1 -- start start 2023-07-18T03:40:51.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.036+0000 7fd02cdff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd014003ac0 0x7fd028158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:51.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.036+0000 7fd02cdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02815bbc0 0x7fd02815b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:51.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.036+0000 7fd02cdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02815e790 0x7fd02815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:51.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.036+0000 7fd02cdff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02815f140 con 0x7fd014003ac0 2023-07-18T03:40:51.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.036+0000 7fd02cdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02815cf00 con 0x7fd02815bbc0 2023-07-18T03:40:51.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.036+0000 7fd02cdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02815d1e0 con 0x7fd02815e790 2023-07-18T03:40:51.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.037+0000 7fd02659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd014003ac0 0x7fd028158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:51.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.037+0000 7fd02659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd014003ac0 0x7fd028158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53982/0 (socket says 172.21.15.49:53982) 2023-07-18T03:40:51.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.037+0000 7fd02659c700 1 -- 172.21.15.49:0/1807871506 learned_addr learned my addr 172.21.15.49:0/1807871506 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:51.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.037+0000 7fd026d9d700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02815e790 0x7fd02815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:51.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.037+0000 7fd01ffff700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02815bbc0 0x7fd02815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:51.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.037+0000 7fd02659c700 1 -- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02815e790 msgr2=0x7fd02815ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:51.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.037+0000 7fd02659c700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02815e790 0x7fd02815ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:51.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.037+0000 7fd02659c700 1 -- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02815bbc0 msgr2=0x7fd02815b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:51.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.037+0000 7fd02659c700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02815bbc0 0x7fd02815b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:51.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.037+0000 7fd02659c700 1 -- 172.21.15.49:0/1807871506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02815d4c0 con 0x7fd014003ac0 2023-07-18T03:40:51.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.037+0000 7fd01ffff700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02815bbc0 0x7fd02815b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:51.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.037+0000 7fd02659c700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd014003ac0 0x7fd028158370 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd01800c960 tx=0x7fd018000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:51.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.038+0000 7fd01effd700 1 -- 172.21.15.49:0/1807871506 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018007c60 con 0x7fd014003ac0 2023-07-18T03:40:51.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.038+0000 7fd01effd700 1 -- 172.21.15.49:0/1807871506 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd018004ca0 con 0x7fd014003ac0 2023-07-18T03:40:51.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.038+0000 7fd01effd700 1 -- 172.21.15.49:0/1807871506 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018001800 con 0x7fd014003ac0 2023-07-18T03:40:51.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.038+0000 7fd02cdff700 1 -- 172.21.15.49:0/1807871506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02815d7a0 con 0x7fd014003ac0 2023-07-18T03:40:51.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.038+0000 7fd02cdff700 1 -- 172.21.15.49:0/1807871506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02815b770 con 0x7fd014003ac0 2023-07-18T03:40:51.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.039+0000 7fd02cdff700 1 -- 172.21.15.49:0/1807871506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd028074350 con 0x7fd014003ac0 2023-07-18T03:40:51.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.040+0000 7fd01effd700 1 -- 172.21.15.49:0/1807871506 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fd018007310 con 0x7fd014003ac0 2023-07-18T03:40:51.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.041+0000 7fd01effd700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd00806ad70 0x7fd00806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:51.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.041+0000 7fd01effd700 1 -- 172.21.15.49:0/1807871506 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd01808acd0 con 0x7fd014003ac0 2023-07-18T03:40:51.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.041+0000 7fd01ffff700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd00806ad70 0x7fd00806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:51.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.042+0000 7fd01ffff700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd00806ad70 0x7fd00806d220 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd0281f5680 tx=0x7fd010008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:51.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.045+0000 7fd01effd700 1 -- 172.21.15.49:0/1807871506 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd018055d20 con 0x7fd014003ac0 2023-07-18T03:40:51.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.301+0000 7fd02cdff700 1 -- 172.21.15.49:0/1807871506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd02806b2f0 con 0x7fd014003ac0 2023-07-18T03:40:51.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.924+0000 7fd01effd700 1 -- 172.21.15.49:0/1807871506 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 136+0+0 (secure 0 0 0) 0x7fd018059340 con 0x7fd014003ac0 2023-07-18T03:40:51.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.927+0000 7fd02cdff700 1 -- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd00806ad70 msgr2=0x7fd00806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:51.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.927+0000 7fd02cdff700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd00806ad70 0x7fd00806d220 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd0281f5680 tx=0x7fd010008040 comp rx=0 tx=0).stop 2023-07-18T03:40:51.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.927+0000 7fd02cdff700 1 -- 172.21.15.49:0/1807871506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd014003ac0 msgr2=0x7fd028158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:51.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.927+0000 7fd02cdff700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd014003ac0 0x7fd028158370 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd01800c960 tx=0x7fd018000fe0 comp rx=0 tx=0).stop 2023-07-18T03:40:51.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.927+0000 7fd02cdff700 1 -- 172.21.15.49:0/1807871506 shutdown_connections 2023-07-18T03:40:51.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.927+0000 7fd02cdff700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02815e790 0x7fd02815ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:51.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.927+0000 7fd02cdff700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd00806ad70 0x7fd00806d220 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:51.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.927+0000 7fd02cdff700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd014003ac0 0x7fd028158370 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:51.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.927+0000 7fd02cdff700 1 --2- 172.21.15.49:0/1807871506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd02815bbc0 0x7fd02815b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:51.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.927+0000 7fd02cdff700 1 -- 172.21.15.49:0/1807871506 >> 172.21.15.49:0/1807871506 conn(0x7fd0280664f0 msgr2=0x7fd0280b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:51.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.927+0000 7fd02cdff700 1 -- 172.21.15.49:0/1807871506 shutdown_connections 2023-07-18T03:40:51.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:51.927+0000 7fd02cdff700 1 -- 172.21.15.49:0/1807871506 wait complete. 2023-07-18T03:40:51.943 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-07-18T03:40:52.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.159+0000 7f85195ff700 1 Processor -- start 2023-07-18T03:40:52.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.159+0000 7f85195ff700 1 -- start start 2023-07-18T03:40:52.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.160+0000 7f85195ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85140ce030 0x7f8514069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:52.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.160+0000 7f85195ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f851406a470 0x7f85141601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:52.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.160+0000 7f85195ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85140cdba0 con 0x7f851406f010 2023-07-18T03:40:52.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.160+0000 7f85195ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851406f3e0 con 0x7f851406a470 2023-07-18T03:40:52.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.160+0000 7f85195ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851406a8f0 con 0x7f85140ce030 2023-07-18T03:40:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.161+0000 7f8512ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f851406f010 0x7f8514071650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33340/0 (socket says 172.21.15.49:33340) 2023-07-18T03:40:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.161+0000 7f8512ffd700 1 -- 172.21.15.49:0/3779532933 learned_addr learned my addr 172.21.15.49:0/3779532933 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.161+0000 7f85137fe700 1 --2- 172.21.15.49:0/3779532933 >> v2:172.21.15.49:3300/0 conn(0x7f851406a470 0x7f85141601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:52.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.161+0000 7f85127fc700 1 --2- 172.21.15.49:0/3779532933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85140ce030 0x7f8514069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:52.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.161+0000 7f8511ffb700 1 -- 172.21.15.49:0/3779532933 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349234147 0 0) 0x7f85140cdba0 con 0x7f851406f010 2023-07-18T03:40:52.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.161+0000 7f8511ffb700 1 -- 172.21.15.49:0/3779532933 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84fc0035b0 con 0x7f851406f010 2023-07-18T03:40:52.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.161+0000 7f85137fe700 1 -- 172.21.15.49:0/3779532933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85140ce030 msgr2=0x7f8514069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:52.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.161+0000 7f85137fe700 1 --2- 172.21.15.49:0/3779532933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85140ce030 0x7f8514069f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.161+0000 7f85137fe700 1 -- 172.21.15.49:0/3779532933 >> v1:172.21.15.49:6789/0 conn(0x7f851406f010 legacy=0x7f8514071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.161+0000 7f85137fe700 1 -- 172.21.15.49:0/3779532933 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85140cf370 con 0x7f851406a470 2023-07-18T03:40:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.162+0000 7f85137fe700 1 --2- 172.21.15.49:0/3779532933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f851406a470 0x7f85141601d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f850c009d90 tx=0x7f850c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927b67757d9f6a43 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.162+0000 7f8511ffb700 1 -- 172.21.15.49:0/3779532933 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c00e070 con 0x7f851406a470 2023-07-18T03:40:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.162+0000 7f8511ffb700 1 -- 172.21.15.49:0/3779532933 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c00bb20 con 0x7f851406a470 2023-07-18T03:40:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.162+0000 7f8511ffb700 1 -- 172.21.15.49:0/3779532933 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c012430 con 0x7f851406a470 2023-07-18T03:40:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.162+0000 7f85195ff700 1 -- 172.21.15.49:0/3779532933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f851406a470 msgr2=0x7f85141601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.162+0000 7f85195ff700 1 --2- 172.21.15.49:0/3779532933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f851406a470 0x7f85141601d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f850c009d90 tx=0x7f850c00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:52.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.162+0000 7f85195ff700 1 -- 172.21.15.49:0/3779532933 shutdown_connections 2023-07-18T03:40:52.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.162+0000 7f85195ff700 1 --2- 172.21.15.49:0/3779532933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f851406a470 0x7f85141601d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.162+0000 7f85195ff700 1 --2- 172.21.15.49:0/3779532933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85140ce030 0x7f8514069f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.162+0000 7f85195ff700 1 -- 172.21.15.49:0/3779532933 >> 172.21.15.49:0/3779532933 conn(0x7f85140cc640 msgr2=0x7f85140cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:52.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.163+0000 7f85195ff700 1 -- 172.21.15.49:0/3779532933 shutdown_connections 2023-07-18T03:40:52.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.163+0000 7f85195ff700 1 -- 172.21.15.49:0/3779532933 wait complete. 2023-07-18T03:40:52.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.164+0000 7f85195ff700 1 Processor -- start 2023-07-18T03:40:52.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.164+0000 7f85195ff700 1 -- start start 2023-07-18T03:40:52.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.165+0000 7f85195ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f851406a470 0x7f85140b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:52.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.165+0000 7f85195ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f851406f010 0x7f851414aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:52.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.165+0000 7f85195ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85140ce030 0x7f851415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:52.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.165+0000 7f85195ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851414da60 con 0x7f85140ce030 2023-07-18T03:40:52.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.165+0000 7f85195ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85140c3770 con 0x7f851406f010 2023-07-18T03:40:52.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.165+0000 7f85195ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85140c3a50 con 0x7f851406a470 2023-07-18T03:40:52.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.166+0000 7f85137fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85140ce030 0x7f851415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:52.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.166+0000 7f85137fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85140ce030 0x7f851415dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53994/0 (socket says 172.21.15.49:53994) 2023-07-18T03:40:52.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.166+0000 7f85137fe700 1 -- 172.21.15.49:0/2737065390 learned_addr learned my addr 172.21.15.49:0/2737065390 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:52.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.166+0000 7f8512ffd700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f851406a470 0x7f85140b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:52.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.166+0000 7f85127fc700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f851406f010 0x7f851414aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:52.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.166+0000 7f85137fe700 1 -- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f851406a470 msgr2=0x7f85140b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.166+0000 7f85137fe700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f851406a470 0x7f85140b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.166+0000 7f85137fe700 1 -- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f851406f010 msgr2=0x7f851414aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.166+0000 7f85137fe700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f851406f010 0x7f851414aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.166+0000 7f85137fe700 1 -- 172.21.15.49:0/2737065390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c009a70 con 0x7f85140ce030 2023-07-18T03:40:52.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.166+0000 7f85137fe700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85140ce030 0x7f851415dc00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f850c00b370 tx=0x7f850c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:52.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.167+0000 7f84fb7fe700 1 -- 172.21.15.49:0/2737065390 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c012430 con 0x7f85140ce030 2023-07-18T03:40:52.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.167+0000 7f84fb7fe700 1 -- 172.21.15.49:0/2737065390 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c020070 con 0x7f85140ce030 2023-07-18T03:40:52.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.167+0000 7f84fb7fe700 1 -- 172.21.15.49:0/2737065390 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c0098f0 con 0x7f85140ce030 2023-07-18T03:40:52.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.167+0000 7f85195ff700 1 -- 172.21.15.49:0/2737065390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85140c3d30 con 0x7f85140ce030 2023-07-18T03:40:52.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.167+0000 7f85195ff700 1 -- 172.21.15.49:0/2737065390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851415bd10 con 0x7f85140ce030 2023-07-18T03:40:52.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.168+0000 7f85195ff700 1 -- 172.21.15.49:0/2737065390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851415bf30 con 0x7f85140ce030 2023-07-18T03:40:52.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.169+0000 7f84fb7fe700 1 -- 172.21.15.49:0/2737065390 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f850c004040 con 0x7f85140ce030 2023-07-18T03:40:52.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.170+0000 7f84fb7fe700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f850006ad70 0x7f850006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:52.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.170+0000 7f84fb7fe700 1 -- 172.21.15.49:0/2737065390 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f850c022070 con 0x7f85140ce030 2023-07-18T03:40:52.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.170+0000 7f8512ffd700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f850006ad70 0x7f850006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:52.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.171+0000 7f8512ffd700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f850006ad70 0x7f850006d220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8504007fd0 tx=0x7f8504007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:52.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.174+0000 7f84fb7fe700 1 -- 172.21.15.49:0/2737065390 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f850c059b40 con 0x7f85140ce030 2023-07-18T03:40:52.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.409+0000 7f85195ff700 1 -- 172.21.15.49:0/2737065390 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f85140cf930 con 0x7f850006ad70 2023-07-18T03:40:52.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.412+0000 7f84fb7fe700 1 -- 172.21.15.49:0/2737065390 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f8504001370 con 0x7f850006ad70 2023-07-18T03:40:52.406 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2023-07-18T03:40:52.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-07-18T03:40:52.407 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephfs" 2023-07-18T03:40:52.407 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-07-18T03:40:52.407 INFO:teuthology.orchestra.run.smithi049.stdout:] 2023-07-18T03:40:52.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.413+0000 7f85195ff700 1 -- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f850006ad70 msgr2=0x7f850006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.413+0000 7f85195ff700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f850006ad70 0x7f850006d220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8504007fd0 tx=0x7f8504007dc0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.413+0000 7f85195ff700 1 -- 172.21.15.49:0/2737065390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85140ce030 msgr2=0x7f851415dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.413+0000 7f85195ff700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85140ce030 0x7f851415dc00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f850c00b370 tx=0x7f850c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.414+0000 7f85195ff700 1 -- 172.21.15.49:0/2737065390 shutdown_connections 2023-07-18T03:40:52.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.414+0000 7f85195ff700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f850006ad70 0x7f850006d220 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.414+0000 7f85195ff700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85140ce030 0x7f851415dc00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.414+0000 7f85195ff700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f851406a470 0x7f85140b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.414+0000 7f85195ff700 1 --2- 172.21.15.49:0/2737065390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f851406f010 0x7f851414aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.414+0000 7f85195ff700 1 -- 172.21.15.49:0/2737065390 >> 172.21.15.49:0/2737065390 conn(0x7f85140cc640 msgr2=0x7f85140b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:52.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.414+0000 7f85195ff700 1 -- 172.21.15.49:0/2737065390 shutdown_connections 2023-07-18T03:40:52.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.414+0000 7f85195ff700 1 -- 172.21.15.49:0/2737065390 wait complete. 2023-07-18T03:40:52.427 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-18T03:40:52.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.641+0000 7f99493ff700 1 Processor -- start 2023-07-18T03:40:52.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.641+0000 7f99493ff700 1 -- start start 2023-07-18T03:40:52.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.642+0000 7f99493ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f99440b4d00 0x7f99440d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:52.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.642+0000 7f99493ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99440d02a0 con 0x7f99440b8660 2023-07-18T03:40:52.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.642+0000 7f99493ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99440cfce0 con 0x7f99440b4860 2023-07-18T03:40:52.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.642+0000 7f99493ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99440cffc0 con 0x7f99440b4d00 2023-07-18T03:40:52.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.642+0000 7f9942ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f99440b8660 0x7f99440cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33350/0 (socket says 172.21.15.49:33350) 2023-07-18T03:40:52.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.642+0000 7f9942ffd700 1 -- 172.21.15.49:0/2052755044 learned_addr learned my addr 172.21.15.49:0/2052755044 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:52.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.642+0000 7f99437fe700 1 --2- 172.21.15.49:0/2052755044 >> v2:172.21.15.49:3300/0 conn(0x7f99440b4d00 0x7f99440d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:52.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.643+0000 7f9941ffb700 1 -- 172.21.15.49:0/2052755044 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270711459 0 0) 0x7f99440d02a0 con 0x7f99440b8660 2023-07-18T03:40:52.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.643+0000 7f9941ffb700 1 -- 172.21.15.49:0/2052755044 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f992c0035b0 con 0x7f99440b8660 2023-07-18T03:40:52.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.643+0000 7f99437fe700 1 -- 172.21.15.49:0/2052755044 >> v1:172.21.15.107:6789/0 conn(0x7f99440b4860 legacy=0x7f99440d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.643+0000 7f99437fe700 1 -- 172.21.15.49:0/2052755044 >> v1:172.21.15.49:6789/0 conn(0x7f99440b8660 legacy=0x7f99440cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.643+0000 7f99437fe700 1 -- 172.21.15.49:0/2052755044 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99440d0ac0 con 0x7f99440b4d00 2023-07-18T03:40:52.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.643+0000 7f99437fe700 1 --2- 172.21.15.49:0/2052755044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99440b4d00 0x7f99440d19d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f993c009d90 tx=0x7f993c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1d147ab5e55ab2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:52.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.643+0000 7f9941ffb700 1 -- 172.21.15.49:0/2052755044 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c00e070 con 0x7f99440b4d00 2023-07-18T03:40:52.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.643+0000 7f9941ffb700 1 -- 172.21.15.49:0/2052755044 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993c00bb20 con 0x7f99440b4d00 2023-07-18T03:40:52.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.643+0000 7f9941ffb700 1 -- 172.21.15.49:0/2052755044 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c012430 con 0x7f99440b4d00 2023-07-18T03:40:52.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.644+0000 7f99493ff700 1 -- 172.21.15.49:0/2052755044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99440b4d00 msgr2=0x7f99440d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.644+0000 7f99493ff700 1 --2- 172.21.15.49:0/2052755044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99440b4d00 0x7f99440d19d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f993c009d90 tx=0x7f993c00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:52.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.644+0000 7f99493ff700 1 -- 172.21.15.49:0/2052755044 shutdown_connections 2023-07-18T03:40:52.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.644+0000 7f99493ff700 1 --2- 172.21.15.49:0/2052755044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99440b4d00 0x7f99440d19d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.644+0000 7f99493ff700 1 -- 172.21.15.49:0/2052755044 >> 172.21.15.49:0/2052755044 conn(0x7f9944066860 msgr2=0x7f994406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:52.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.644+0000 7f99493ff700 1 -- 172.21.15.49:0/2052755044 shutdown_connections 2023-07-18T03:40:52.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.644+0000 7f99493ff700 1 -- 172.21.15.49:0/2052755044 wait complete. 2023-07-18T03:40:52.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.646+0000 7f99493ff700 1 Processor -- start 2023-07-18T03:40:52.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.646+0000 7f99493ff700 1 -- start start 2023-07-18T03:40:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.647+0000 7f99493ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99440b4860 0x7f99440c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.647+0000 7f99493ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99440b4d00 0x7f99441e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.647+0000 7f99493ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99440b8660 0x7f99440c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:52.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.647+0000 7f99493ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99441e7b80 con 0x7f99440b4860 2023-07-18T03:40:52.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.647+0000 7f99493ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99440c8060 con 0x7f99440b8660 2023-07-18T03:40:52.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.647+0000 7f99493ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9944154060 con 0x7f99440b4d00 2023-07-18T03:40:52.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.647+0000 7f9942ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99440b4860 0x7f99440c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:52.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.647+0000 7f9942ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99440b4860 0x7f99440c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53998/0 (socket says 172.21.15.49:53998) 2023-07-18T03:40:52.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.647+0000 7f9942ffd700 1 -- 172.21.15.49:0/3931237470 learned_addr learned my addr 172.21.15.49:0/3931237470 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:52.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.647+0000 7f99437fe700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99440b8660 0x7f99440c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:52.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.647+0000 7f99427fc700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99440b4d00 0x7f99441e7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.648+0000 7f9942ffd700 1 -- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99440b4d00 msgr2=0x7f99441e7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.648+0000 7f9942ffd700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99440b4d00 0x7f99441e7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.648+0000 7f9942ffd700 1 -- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99440b8660 msgr2=0x7f99440c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.648+0000 7f9942ffd700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99440b8660 0x7f99440c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.648+0000 7f9942ffd700 1 -- 172.21.15.49:0/3931237470 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c009a70 con 0x7f99440b4860 2023-07-18T03:40:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.648+0000 7f9942ffd700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99440b4860 0x7f99440c7b20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9934001eb0 tx=0x7f993400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.648+0000 7f992b7fe700 1 -- 172.21.15.49:0/3931237470 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934005d70 con 0x7f99440b4860 2023-07-18T03:40:52.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.648+0000 7f992b7fe700 1 -- 172.21.15.49:0/3931237470 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99340070d0 con 0x7f99440b4860 2023-07-18T03:40:52.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.648+0000 7f99493ff700 1 -- 172.21.15.49:0/3931237470 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99440c6a40 con 0x7f99440b4860 2023-07-18T03:40:52.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.648+0000 7f992b7fe700 1 -- 172.21.15.49:0/3931237470 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934007460 con 0x7f99440b4860 2023-07-18T03:40:52.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.649+0000 7f99493ff700 1 -- 172.21.15.49:0/3931237470 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99441e77a0 con 0x7f99440b4860 2023-07-18T03:40:52.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.649+0000 7f99493ff700 1 -- 172.21.15.49:0/3931237470 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99440c6f60 con 0x7f99440b4860 2023-07-18T03:40:52.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.651+0000 7f992b7fe700 1 -- 172.21.15.49:0/3931237470 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f99340075c0 con 0x7f99440b4860 2023-07-18T03:40:52.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.651+0000 7f992b7fe700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f993006ad70 0x7f993006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:52.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.652+0000 7f992b7fe700 1 -- 172.21.15.49:0/3931237470 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9934088950 con 0x7f99440b4860 2023-07-18T03:40:52.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.652+0000 7f99427fc700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f993006ad70 0x7f993006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:52.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.653+0000 7f99427fc700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f993006ad70 0x7f993006d220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9944154790 tx=0x7f9938009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:52.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.656+0000 7f992b7fe700 1 -- 172.21.15.49:0/3931237470 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f99340539a0 con 0x7f99440b4860 2023-07-18T03:40:52.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.863+0000 7f99493ff700 1 -- 172.21.15.49:0/3931237470 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9944074350 con 0x7f99440b4860 2023-07-18T03:40:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.863+0000 7f992b7fe700 1 -- 172.21.15.49:0/3931237470 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9934056fc0 con 0x7f99440b4860 2023-07-18T03:40:52.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.865+0000 7f99493ff700 1 -- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f993006ad70 msgr2=0x7f993006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.865+0000 7f99493ff700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f993006ad70 0x7f993006d220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9944154790 tx=0x7f9938009040 comp rx=0 tx=0).stop 2023-07-18T03:40:52.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.865+0000 7f99493ff700 1 -- 172.21.15.49:0/3931237470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99440b4860 msgr2=0x7f99440c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:52.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.865+0000 7f99493ff700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99440b4860 0x7f99440c7b20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9934001eb0 tx=0x7f993400c7b0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.866+0000 7f99493ff700 1 -- 172.21.15.49:0/3931237470 shutdown_connections 2023-07-18T03:40:52.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.866+0000 7f99493ff700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99440b8660 0x7f99440c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.866+0000 7f99493ff700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f993006ad70 0x7f993006d220 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.866+0000 7f99493ff700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99440b4860 0x7f99440c7b20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.866+0000 7f99493ff700 1 --2- 172.21.15.49:0/3931237470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99440b4d00 0x7f99441e7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:52.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.866+0000 7f99493ff700 1 -- 172.21.15.49:0/3931237470 >> 172.21.15.49:0/3931237470 conn(0x7f9944066860 msgr2=0x7f99440cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:52.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.866+0000 7f99493ff700 1 -- 172.21.15.49:0/3931237470 shutdown_connections 2023-07-18T03:40:52.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:52.866+0000 7f99493ff700 1 -- 172.21.15.49:0/3931237470 wait complete. 2023-07-18T03:40:52.879 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000030717 2023-07-18T03:40:53.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.093+0000 7f04219ff700 1 Processor -- start 2023-07-18T03:40:53.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.094+0000 7f04219ff700 1 -- start start 2023-07-18T03:40:53.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.095+0000 7f04219ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f041c15d9d0 0x7f041c06b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.095+0000 7f04219ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f041c06c9f0 0x7f041c0b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.095+0000 7f04219ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f041c0b1e50 0x7f041c071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.095+0000 7f04219ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c06c4e0 con 0x7f041c15d9d0 2023-07-18T03:40:53.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.095+0000 7f04219ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c15dda0 con 0x7f041c06c9f0 2023-07-18T03:40:53.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.095+0000 7f04219ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c06bf20 con 0x7f041c0b1e50 2023-07-18T03:40:53.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.095+0000 7f041affd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f041c15d9d0 0x7f041c06b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.095+0000 7f041affd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f041c15d9d0 0x7f041c06b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54014/0 (socket says 172.21.15.49:54014) 2023-07-18T03:40:53.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.095+0000 7f041affd700 1 -- 172.21.15.49:0/441257104 learned_addr learned my addr 172.21.15.49:0/441257104 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:53.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.095+0000 7f041a7fc700 1 --2- 172.21.15.49:0/441257104 >> v2:172.21.15.107:3300/0 conn(0x7f041c06c9f0 0x7f041c0b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.095+0000 7f041b7fe700 1 --2- 172.21.15.49:0/441257104 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f041c0b1e50 0x7f041c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.096+0000 7f041affd700 1 -- 172.21.15.49:0/441257104 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f041c0b1e50 msgr2=0x7f041c071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.096+0000 7f041affd700 1 --2- 172.21.15.49:0/441257104 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f041c0b1e50 0x7f041c071170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.096+0000 7f041affd700 1 -- 172.21.15.49:0/441257104 >> v2:172.21.15.107:3300/0 conn(0x7f041c06c9f0 msgr2=0x7f041c0b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.096+0000 7f041affd700 1 --2- 172.21.15.49:0/441257104 >> v2:172.21.15.107:3300/0 conn(0x7f041c06c9f0 0x7f041c0b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.096+0000 7f041affd700 1 -- 172.21.15.49:0/441257104 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c06c200 con 0x7f041c15d9d0 2023-07-18T03:40:53.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.096+0000 7f041a7fc700 1 --2- 172.21.15.49:0/441257104 >> v2:172.21.15.107:3300/0 conn(0x7f041c06c9f0 0x7f041c0b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.096+0000 7f041b7fe700 1 --2- 172.21.15.49:0/441257104 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f041c0b1e50 0x7f041c071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.096+0000 7f041affd700 1 --2- 172.21.15.49:0/441257104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f041c15d9d0 0x7f041c06b5b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f041c0720b0 tx=0x7f040c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c587947c7a2ffc98 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.096+0000 7f0419ffb700 1 -- 172.21.15.49:0/441257104 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c00b870 con 0x7f041c15d9d0 2023-07-18T03:40:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.096+0000 7f0419ffb700 1 -- 172.21.15.49:0/441257104 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c00b9d0 con 0x7f041c15d9d0 2023-07-18T03:40:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.096+0000 7f0419ffb700 1 -- 172.21.15.49:0/441257104 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c00bd60 con 0x7f041c15d9d0 2023-07-18T03:40:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.097+0000 7f04219ff700 1 -- 172.21.15.49:0/441257104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f041c15d9d0 msgr2=0x7f041c06b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.097+0000 7f04219ff700 1 --2- 172.21.15.49:0/441257104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f041c15d9d0 0x7f041c06b5b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f041c0720b0 tx=0x7f040c009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.097+0000 7f04219ff700 1 -- 172.21.15.49:0/441257104 shutdown_connections 2023-07-18T03:40:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.097+0000 7f04219ff700 1 --2- 172.21.15.49:0/441257104 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f041c0b1e50 0x7f041c071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.097+0000 7f04219ff700 1 --2- 172.21.15.49:0/441257104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f041c15d9d0 0x7f041c06b5b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.097+0000 7f04219ff700 1 --2- 172.21.15.49:0/441257104 >> v2:172.21.15.107:3300/0 conn(0x7f041c06c9f0 0x7f041c0b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.097+0000 7f04219ff700 1 -- 172.21.15.49:0/441257104 >> 172.21.15.49:0/441257104 conn(0x7f041c0b05d0 msgr2=0x7f041c0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.097+0000 7f04219ff700 1 -- 172.21.15.49:0/441257104 shutdown_connections 2023-07-18T03:40:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.097+0000 7f04219ff700 1 -- 172.21.15.49:0/441257104 wait complete. 2023-07-18T03:40:53.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.099+0000 7f04219ff700 1 Processor -- start 2023-07-18T03:40:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.099+0000 7f04219ff700 1 -- start start 2023-07-18T03:40:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.100+0000 7f04219ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f041c06c9f0 0x7f041c0cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.100+0000 7f04219ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f041c0b1e50 0x7f041c0acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.100+0000 7f04219ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f041c15d9d0 0x7f041c14fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.100+0000 7f04219ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c0ce270 con 0x7f041c15d9d0 2023-07-18T03:40:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.100+0000 7f04219ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c0ad2c0 con 0x7f041c06c9f0 2023-07-18T03:40:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.100+0000 7f04219ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c0cd6f0 con 0x7f041c0b1e50 2023-07-18T03:40:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.100+0000 7f041b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f041c15d9d0 0x7f041c14fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.100+0000 7f041affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f041c06c9f0 0x7f041c0cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.100+0000 7f041affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f041c06c9f0 0x7f041c0cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:35166/0 (socket says 172.21.15.49:35166) 2023-07-18T03:40:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.100+0000 7f041affd700 1 -- 172.21.15.49:0/1618788232 learned_addr learned my addr 172.21.15.49:0/1618788232 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:53.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.100+0000 7f041b7fe700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f041c15d9d0 0x7f041c14fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54016/0 (socket says 172.21.15.49:54016) 2023-07-18T03:40:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.100+0000 7f041a7fc700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f041c0b1e50 0x7f041c0acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.101+0000 7f041b7fe700 1 -- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f041c0b1e50 msgr2=0x7f041c0acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.101+0000 7f041b7fe700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f041c0b1e50 0x7f041c0acd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.101+0000 7f041b7fe700 1 -- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f041c06c9f0 msgr2=0x7f041c0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.101+0000 7f041b7fe700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f041c06c9f0 0x7f041c0cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.101+0000 7f041b7fe700 1 -- 172.21.15.49:0/1618788232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c009550 con 0x7f041c15d9d0 2023-07-18T03:40:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.101+0000 7f041affd700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f041c06c9f0 0x7f041c0cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.101+0000 7f041b7fe700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f041c15d9d0 0x7f041c14fe40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0414007e60 tx=0x7f041400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.101+0000 7f0403fff700 1 -- 172.21.15.49:0/1618788232 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400dd20 con 0x7f041c15d9d0 2023-07-18T03:40:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.101+0000 7f0403fff700 1 -- 172.21.15.49:0/1618788232 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041400f040 con 0x7f041c15d9d0 2023-07-18T03:40:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.101+0000 7f04219ff700 1 -- 172.21.15.49:0/1618788232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c0cd9d0 con 0x7f041c15d9d0 2023-07-18T03:40:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.101+0000 7f0403fff700 1 -- 172.21.15.49:0/1618788232 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414003c60 con 0x7f041c15d9d0 2023-07-18T03:40:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.101+0000 7f04219ff700 1 -- 172.21.15.49:0/1618788232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c0cdf90 con 0x7f041c15d9d0 2023-07-18T03:40:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.102+0000 7f04219ff700 1 -- 172.21.15.49:0/1618788232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c0a84e0 con 0x7f041c15d9d0 2023-07-18T03:40:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.103+0000 7f0403fff700 1 -- 172.21.15.49:0/1618788232 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f0414004030 con 0x7f041c15d9d0 2023-07-18T03:40:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.104+0000 7f0403fff700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f040806ad70 0x7f040806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.104+0000 7f0403fff700 1 -- 172.21.15.49:0/1618788232 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0414089540 con 0x7f041c15d9d0 2023-07-18T03:40:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.105+0000 7f041affd700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f040806ad70 0x7f040806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.105+0000 7f041affd700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f040806ad70 0x7f040806d220 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f040c009c10 tx=0x7f040c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.108+0000 7f0403fff700 1 -- 172.21.15.49:0/1618788232 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0414054480 con 0x7f041c15d9d0 2023-07-18T03:40:53.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.353+0000 7f04219ff700 1 -- 172.21.15.49:0/1618788232 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000000030717", "target": ["mon-mgr", ""]}) v1 -- 0x7f041c070c50 con 0x7f040806ad70 2023-07-18T03:40:53.357 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:40:53.362+0000 7f45ce8fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:40:53.357 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:40:53.362+0000 7f45ce8fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:40:53.357 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:40:53.362+0000 7f45ce8fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:40:53.358 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:40:53.362+0000 7f45ce8fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:40:53.358 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:40:53.362+0000 7f45ce8fe700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:40:53.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.378+0000 7f0403fff700 1 -- 172.21.15.49:0/1618788232 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f040c006c60 con 0x7f040806ad70 2023-07-18T03:40:53.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.381+0000 7f04219ff700 1 -- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f040806ad70 msgr2=0x7f040806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.381+0000 7f04219ff700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f040806ad70 0x7f040806d220 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f040c009c10 tx=0x7f040c008030 comp rx=0 tx=0).stop 2023-07-18T03:40:53.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.381+0000 7f04219ff700 1 -- 172.21.15.49:0/1618788232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f041c15d9d0 msgr2=0x7f041c14fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.381+0000 7f04219ff700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f041c15d9d0 0x7f041c14fe40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0414007e60 tx=0x7f041400d3b0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.381+0000 7f04219ff700 1 -- 172.21.15.49:0/1618788232 shutdown_connections 2023-07-18T03:40:53.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.381+0000 7f04219ff700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f040806ad70 0x7f040806d220 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.381+0000 7f04219ff700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f041c15d9d0 0x7f041c14fe40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.381+0000 7f04219ff700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f041c06c9f0 0x7f041c0cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.381+0000 7f04219ff700 1 --2- 172.21.15.49:0/1618788232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f041c0b1e50 0x7f041c0acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.381+0000 7f04219ff700 1 -- 172.21.15.49:0/1618788232 >> 172.21.15.49:0/1618788232 conn(0x7f041c0b05d0 msgr2=0x7f041c0b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:53.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.381+0000 7f04219ff700 1 -- 172.21.15.49:0/1618788232 shutdown_connections 2023-07-18T03:40:53.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.381+0000 7f04219ff700 1 -- 172.21.15.49:0/1618788232 wait complete. 2023-07-18T03:40:53.394 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000974124 group_0000000000030717 2023-07-18T03:40:53.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.608+0000 7fe3c01ff700 1 Processor -- start 2023-07-18T03:40:53.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.608+0000 7fe3c01ff700 1 -- start start 2023-07-18T03:40:53.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.609+0000 7fe3c01ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe3b80d1140 0x7fe3b80d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.609+0000 7fe3c01ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3b80b1e50 0x7fe3b80ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.609+0000 7fe3c01ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b80d0190 con 0x7fe3b815c7b0 2023-07-18T03:40:53.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.609+0000 7fe3c01ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b80d3de0 con 0x7fe3b80d1140 2023-07-18T03:40:53.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.609+0000 7fe3c01ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80d0470 con 0x7fe3b80b1e50 2023-07-18T03:40:53.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.610+0000 7fe3bdf9b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe3b815c7b0 0x7fe3b80d0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33356/0 (socket says 172.21.15.49:33356) 2023-07-18T03:40:53.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.610+0000 7fe3bdf9b700 1 -- 172.21.15.49:0/2147495983 learned_addr learned my addr 172.21.15.49:0/2147495983 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:53.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.610+0000 7fe3bd79a700 1 --2- 172.21.15.49:0/2147495983 >> v2:172.21.15.107:3300/0 conn(0x7fe3b80d1140 0x7fe3b80d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.610+0000 7fe3be79c700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3b80b1e50 0x7fe3b80ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.610+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498014287 0 0) 0x7fe3b80d0190 con 0x7fe3b815c7b0 2023-07-18T03:40:53.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.610+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3a40035b0 con 0x7fe3b815c7b0 2023-07-18T03:40:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.610+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 776961163 0 0) 0x7fe3a40035b0 con 0x7fe3b815c7b0 2023-07-18T03:40:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.610+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3b80b1e50 msgr2=0x7fe3b80ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.610+0000 7fe3bcf99700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3b80b1e50 0x7fe3b80ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.610+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 >> v2:172.21.15.107:3300/0 conn(0x7fe3b80d1140 msgr2=0x7fe3b80d28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.610+0000 7fe3bcf99700 1 --2- 172.21.15.49:0/2147495983 >> v2:172.21.15.107:3300/0 conn(0x7fe3b80d1140 0x7fe3b80d28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.610+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80d0750 con 0x7fe3b815c7b0 2023-07-18T03:40:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7fe3b4001670 con 0x7fe3b815c7b0 2023-07-18T03:40:53.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3be79c700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3b80b1e50 0x7fe3b80ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bd79a700 1 --2- 172.21.15.49:0/2147495983 >> v2:172.21.15.107:3300/0 conn(0x7fe3b80d1140 0x7fe3b80d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 >> v1:172.21.15.49:6789/0 conn(0x7fe3b815c7b0 legacy=0x7fe3b80d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bcf99700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4003ac0 0x7fe3a4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bcf99700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3b80b1e50 0x7fe3a4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bcf99700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a4004d90 0x7fe3a4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80d3de0 con 0x7fe3a4003ac0 2023-07-18T03:40:53.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80d0470 con 0x7fe3a4004d90 2023-07-18T03:40:53.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a400ba10 con 0x7fe3b80b1e50 2023-07-18T03:40:53.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bdf9b700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4003ac0 0x7fe3a4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3be79c700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a4004d90 0x7fe3a4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bd79a700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3b80b1e50 0x7fe3a4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bdf9b700 1 -- 172.21.15.49:0/2147495983 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3b80b1e50 msgr2=0x7fe3a4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bdf9b700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3b80b1e50 0x7fe3a4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bdf9b700 1 -- 172.21.15.49:0/2147495983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a4004d90 msgr2=0x7fe3a4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bdf9b700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3a4004d90 0x7fe3a4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.611+0000 7fe3bdf9b700 1 -- 172.21.15.49:0/2147495983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b0009a70 con 0x7fe3a4003ac0 2023-07-18T03:40:53.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.612+0000 7fe3bdf9b700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4003ac0 0x7fe3a4003ed0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400c320 tx=0x7fe3a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b942d606c0f35fd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:53.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.612+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b400ce00 con 0x7fe3a4003ac0 2023-07-18T03:40:53.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.612+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b4005960 con 0x7fe3a4003ac0 2023-07-18T03:40:53.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.612+0000 7fe3bcf99700 1 -- 172.21.15.49:0/2147495983 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4005d10 con 0x7fe3a4003ac0 2023-07-18T03:40:53.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.612+0000 7fe3c01ff700 1 -- 172.21.15.49:0/2147495983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4003ac0 msgr2=0x7fe3a4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.612+0000 7fe3c01ff700 1 --2- 172.21.15.49:0/2147495983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3a4003ac0 0x7fe3a4003ed0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400c320 tx=0x7fe3a4005880 comp rx=0 tx=0).stop 2023-07-18T03:40:53.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.613+0000 7fe3be79c700 1 -- 172.21.15.49:0/2147495983 reap_dead start 2023-07-18T03:40:53.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.613+0000 7fe3c01ff700 1 -- 172.21.15.49:0/2147495983 shutdown_connections 2023-07-18T03:40:53.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.613+0000 7fe3c01ff700 1 -- 172.21.15.49:0/2147495983 >> 172.21.15.49:0/2147495983 conn(0x7fe3b809c1b0 msgr2=0x7fe3b80d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:53.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.613+0000 7fe3c01ff700 1 -- 172.21.15.49:0/2147495983 shutdown_connections 2023-07-18T03:40:53.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.613+0000 7fe3c01ff700 1 -- 172.21.15.49:0/2147495983 wait complete. 2023-07-18T03:40:53.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.615+0000 7fe3c01ff700 1 Processor -- start 2023-07-18T03:40:53.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.615+0000 7fe3c01ff700 1 -- start start 2023-07-18T03:40:53.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3c01ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3a4003ac0 0x7fe3b80b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3c01ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3b815cb80 0x7fe3b8157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3c01ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b80c83a0 0x7fe3b80c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3c01ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80c8d60 con 0x7fe3b80c83a0 2023-07-18T03:40:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3c01ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b815dc70 con 0x7fe3b815cb80 2023-07-18T03:40:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3c01ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b815d060 con 0x7fe3a4003ac0 2023-07-18T03:40:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3be79c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b80c83a0 0x7fe3b80c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3be79c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b80c83a0 0x7fe3b80c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54036/0 (socket says 172.21.15.49:54036) 2023-07-18T03:40:53.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3be79c700 1 -- 172.21.15.49:0/3893763173 learned_addr learned my addr 172.21.15.49:0/3893763173 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:53.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3bdf9b700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3a4003ac0 0x7fe3b80b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3be79c700 1 -- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3a4003ac0 msgr2=0x7fe3b80b85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3bd79a700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3b815cb80 0x7fe3b8157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3be79c700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3a4003ac0 0x7fe3b80b85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3be79c700 1 -- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3b815cb80 msgr2=0x7fe3b8157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3be79c700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3b815cb80 0x7fe3b8157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3be79c700 1 -- 172.21.15.49:0/3893763173 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b815d340 con 0x7fe3b80c83a0 2023-07-18T03:40:53.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.616+0000 7fe3bd79a700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3b815cb80 0x7fe3b8157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:53.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.617+0000 7fe3be79c700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b80c83a0 0x7fe3b80c8820 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80720c0 tx=0x7fe3b000f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:53.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.617+0000 7fe3ae7fc700 1 -- 172.21.15.49:0/3893763173 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000e040 con 0x7fe3b80c83a0 2023-07-18T03:40:53.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.617+0000 7fe3ae7fc700 1 -- 172.21.15.49:0/3893763173 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b000f040 con 0x7fe3b80c83a0 2023-07-18T03:40:53.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.617+0000 7fe3ae7fc700 1 -- 172.21.15.49:0/3893763173 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0003a20 con 0x7fe3b80c83a0 2023-07-18T03:40:53.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.617+0000 7fe3c01ff700 1 -- 172.21.15.49:0/3893763173 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b815d620 con 0x7fe3b80c83a0 2023-07-18T03:40:53.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.617+0000 7fe3c01ff700 1 -- 172.21.15.49:0/3893763173 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b80b8b10 con 0x7fe3b80c83a0 2023-07-18T03:40:53.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.618+0000 7fe3c01ff700 1 -- 172.21.15.49:0/3893763173 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b8074350 con 0x7fe3b80c83a0 2023-07-18T03:40:53.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.619+0000 7fe3ae7fc700 1 -- 172.21.15.49:0/3893763173 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7fe3b0015430 con 0x7fe3b80c83a0 2023-07-18T03:40:53.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.620+0000 7fe3ae7fc700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe39c06ad70 0x7fe39c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.620+0000 7fe3ae7fc700 1 -- 172.21.15.49:0/3893763173 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3b0089dc0 con 0x7fe3b80c83a0 2023-07-18T03:40:53.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.620+0000 7fe3bdf9b700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe39c06ad70 0x7fe39c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.621+0000 7fe3bdf9b700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe39c06ad70 0x7fe39c06d220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4015040 tx=0x7fe3b400a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:53.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.624+0000 7fe3ae7fc700 1 -- 172.21.15.49:0/3893763173 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe3b0058790 con 0x7fe3b80c83a0 2023-07-18T03:40:53.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.866+0000 7fe3c01ff700 1 -- 172.21.15.49:0/3893763173 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000974124", "group_name": "group_0000000000030717", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3b80cd830 con 0x7fe39c06ad70 2023-07-18T03:40:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.894+0000 7fe3ae7fc700 1 -- 172.21.15.49:0/3893763173 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe3b40014f0 con 0x7fe39c06ad70 2023-07-18T03:40:53.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.897+0000 7fe3c01ff700 1 -- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe39c06ad70 msgr2=0x7fe39c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.897+0000 7fe3c01ff700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe39c06ad70 0x7fe39c06d220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4015040 tx=0x7fe3b400a000 comp rx=0 tx=0).stop 2023-07-18T03:40:53.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.897+0000 7fe3c01ff700 1 -- 172.21.15.49:0/3893763173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b80c83a0 msgr2=0x7fe3b80c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.897+0000 7fe3c01ff700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b80c83a0 0x7fe3b80c8820 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80720c0 tx=0x7fe3b000f490 comp rx=0 tx=0).stop 2023-07-18T03:40:53.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.897+0000 7fe3c01ff700 1 -- 172.21.15.49:0/3893763173 shutdown_connections 2023-07-18T03:40:53.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.897+0000 7fe3c01ff700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe39c06ad70 0x7fe39c06d220 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.897+0000 7fe3c01ff700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3b80c83a0 0x7fe3b80c8820 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.897+0000 7fe3c01ff700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3a4003ac0 0x7fe3b80b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.897+0000 7fe3c01ff700 1 --2- 172.21.15.49:0/3893763173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3b815cb80 0x7fe3b8157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.897+0000 7fe3c01ff700 1 -- 172.21.15.49:0/3893763173 >> 172.21.15.49:0/3893763173 conn(0x7fe3b809c1b0 msgr2=0x7fe3b80cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:53.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.897+0000 7fe3c01ff700 1 -- 172.21.15.49:0/3893763173 shutdown_connections 2023-07-18T03:40:53.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:53.897+0000 7fe3c01ff700 1 -- 172.21.15.49:0/3893763173 wait complete. 2023-07-18T03:40:53.911 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvolume_0000000000974124 snapshot_0000000000227127 group_0000000000030717 2023-07-18T03:40:54.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.128+0000 7f26c2fff700 1 Processor -- start 2023-07-18T03:40:54.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.128+0000 7f26c2fff700 1 -- start start 2023-07-18T03:40:54.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.129+0000 7f26c2fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f26bc0b12f0 0x7f26bc0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:54.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.129+0000 7f26c2fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26bc0adea0 0x7f26bc0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:54.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.130+0000 7f26c2fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26bc15ce70 con 0x7f26bc0c92d0 2023-07-18T03:40:54.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.130+0000 7f26c2fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26bc15da60 con 0x7f26bc0adea0 2023-07-18T03:40:54.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.130+0000 7f26c2fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26bc15d0c0 con 0x7f26bc0b12f0 2023-07-18T03:40:54.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.130+0000 7f26c159c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26bc0adea0 0x7f26bc0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:54.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.130+0000 7f26c159c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26bc0adea0 0x7f26bc0cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54052/0 (socket says 172.21.15.49:54052) 2023-07-18T03:40:54.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.130+0000 7f26c159c700 1 -- 172.21.15.49:0/1790449258 learned_addr learned my addr 172.21.15.49:0/1790449258 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:54.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.130+0000 7f26bbfff700 1 --2- 172.21.15.49:0/1790449258 >> v2:172.21.15.107:3300/0 conn(0x7f26bc0b12f0 0x7f26bc0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:54.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.130+0000 7f26c159c700 1 -- 172.21.15.49:0/1790449258 >> v2:172.21.15.107:3300/0 conn(0x7f26bc0b12f0 msgr2=0x7f26bc0ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:54.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.130+0000 7f26c159c700 1 --2- 172.21.15.49:0/1790449258 >> v2:172.21.15.107:3300/0 conn(0x7f26bc0b12f0 0x7f26bc0ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.130+0000 7f26c159c700 1 -- 172.21.15.49:0/1790449258 >> v1:172.21.15.107:6789/0 conn(0x7f26bc0c92d0 legacy=0x7f26bc0c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:54.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.130+0000 7f26c159c700 1 -- 172.21.15.49:0/1790449258 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc15d3a0 con 0x7f26bc0adea0 2023-07-18T03:40:54.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.131+0000 7f26c159c700 1 --2- 172.21.15.49:0/1790449258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26bc0adea0 0x7f26bc0cc690 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f26bc06d890 tx=0x7f26b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad540f281e2936d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:54.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.131+0000 7f26bb7fe700 1 -- 172.21.15.49:0/1790449258 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b400bb10 con 0x7f26bc0adea0 2023-07-18T03:40:54.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.131+0000 7f26bb7fe700 1 -- 172.21.15.49:0/1790449258 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b400bc70 con 0x7f26bc0adea0 2023-07-18T03:40:54.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.131+0000 7f26bb7fe700 1 -- 172.21.15.49:0/1790449258 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4011460 con 0x7f26bc0adea0 2023-07-18T03:40:54.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.131+0000 7f26c2fff700 1 -- 172.21.15.49:0/1790449258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26bc0adea0 msgr2=0x7f26bc0cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:54.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.131+0000 7f26c2fff700 1 --2- 172.21.15.49:0/1790449258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26bc0adea0 0x7f26bc0cc690 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f26bc06d890 tx=0x7f26b400b330 comp rx=0 tx=0).stop 2023-07-18T03:40:54.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.132+0000 7f26c2fff700 1 -- 172.21.15.49:0/1790449258 shutdown_connections 2023-07-18T03:40:54.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.132+0000 7f26c2fff700 1 --2- 172.21.15.49:0/1790449258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26bc0adea0 0x7f26bc0cc690 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.132+0000 7f26c2fff700 1 --2- 172.21.15.49:0/1790449258 >> v2:172.21.15.107:3300/0 conn(0x7f26bc0b12f0 0x7f26bc0ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.132+0000 7f26c2fff700 1 -- 172.21.15.49:0/1790449258 >> 172.21.15.49:0/1790449258 conn(0x7f26bc0b05d0 msgr2=0x7f26bc0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:54.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.132+0000 7f26c2fff700 1 -- 172.21.15.49:0/1790449258 shutdown_connections 2023-07-18T03:40:54.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.132+0000 7f26c2fff700 1 -- 172.21.15.49:0/1790449258 wait complete. 2023-07-18T03:40:54.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.134+0000 7f26c2fff700 1 Processor -- start 2023-07-18T03:40:54.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.134+0000 7f26c2fff700 1 -- start start 2023-07-18T03:40:54.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c2fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26bc0adea0 0x7f26bc070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:54.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c2fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26bc0b12f0 0x7f26bc0acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:54.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c2fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26bc0c92d0 0x7f26bc1f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:54.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c2fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc06fc40 con 0x7f26bc0c92d0 2023-07-18T03:40:54.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c2fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc1f30f0 con 0x7f26bc0b12f0 2023-07-18T03:40:54.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c2fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc1f2b30 con 0x7f26bc0adea0 2023-07-18T03:40:54.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c159c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26bc0c92d0 0x7f26bc1f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:54.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c159c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26bc0c92d0 0x7f26bc1f2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54064/0 (socket says 172.21.15.49:54064) 2023-07-18T03:40:54.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c159c700 1 -- 172.21.15.49:0/425485679 learned_addr learned my addr 172.21.15.49:0/425485679 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:54.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26bbfff700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26bc0b12f0 0x7f26bc0acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:54.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c0d9b700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26bc0adea0 0x7f26bc070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c159c700 1 -- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26bc0adea0 msgr2=0x7f26bc070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c159c700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26bc0adea0 0x7f26bc070060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c159c700 1 -- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26bc0b12f0 msgr2=0x7f26bc0acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c159c700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26bc0b12f0 0x7f26bc0acad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.136+0000 7f26c159c700 1 -- 172.21.15.49:0/425485679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b4009b30 con 0x7f26bc0c92d0 2023-07-18T03:40:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.137+0000 7f26c159c700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26bc0c92d0 0x7f26bc1f2120 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f26b400b370 tx=0x7f26b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.137+0000 7f26b97fa700 1 -- 172.21.15.49:0/425485679 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4011a20 con 0x7f26bc0c92d0 2023-07-18T03:40:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.137+0000 7f26b97fa700 1 -- 172.21.15.49:0/425485679 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b4011ba0 con 0x7f26bc0c92d0 2023-07-18T03:40:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.137+0000 7f26b97fa700 1 -- 172.21.15.49:0/425485679 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b40099b0 con 0x7f26bc0c92d0 2023-07-18T03:40:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.137+0000 7f26c2fff700 1 -- 172.21.15.49:0/425485679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26bc1f2e10 con 0x7f26bc0c92d0 2023-07-18T03:40:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.137+0000 7f26c2fff700 1 -- 172.21.15.49:0/425485679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26bc0705a0 con 0x7f26bc0c92d0 2023-07-18T03:40:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.138+0000 7f26c2fff700 1 -- 172.21.15.49:0/425485679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26bc06fd80 con 0x7f26bc0c92d0 2023-07-18T03:40:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.139+0000 7f26b97fa700 1 -- 172.21.15.49:0/425485679 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f26b4004040 con 0x7f26bc0c92d0 2023-07-18T03:40:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.140+0000 7f26b97fa700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f26a006ad70 0x7f26a006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.141+0000 7f26b97fa700 1 -- 172.21.15.49:0/425485679 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f26b408abc0 con 0x7f26bc0c92d0 2023-07-18T03:40:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.141+0000 7f26c0d9b700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f26a006ad70 0x7f26a006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.142+0000 7f26c0d9b700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f26a006ad70 0x7f26a006d220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0c5b20 tx=0x7f26b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:54.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.144+0000 7f26b97fa700 1 -- 172.21.15.49:0/425485679 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f26b4055c20 con 0x7f26bc0c92d0 2023-07-18T03:40:54.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.390+0000 7f26c2fff700 1 -- 172.21.15.49:0/425485679 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000974124", "snap_name": "snapshot_0000000000227127", "group_name": "group_0000000000030717", "target": ["mon-mgr", ""]}) v1 -- 0x7f26bc15d680 con 0x7f26a006ad70 2023-07-18T03:40:54.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.406+0000 7f26b97fa700 1 -- 172.21.15.49:0/425485679 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f26b0001cc0 con 0x7f26a006ad70 2023-07-18T03:40:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.409+0000 7f26c2fff700 1 -- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f26a006ad70 msgr2=0x7f26a006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.409+0000 7f26c2fff700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f26a006ad70 0x7f26a006d220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0c5b20 tx=0x7f26b0009040 comp rx=0 tx=0).stop 2023-07-18T03:40:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.409+0000 7f26c2fff700 1 -- 172.21.15.49:0/425485679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26bc0c92d0 msgr2=0x7f26bc1f2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.409+0000 7f26c2fff700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26bc0c92d0 0x7f26bc1f2120 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f26b400b370 tx=0x7f26b4004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.409+0000 7f26c2fff700 1 -- 172.21.15.49:0/425485679 shutdown_connections 2023-07-18T03:40:54.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.409+0000 7f26c2fff700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f26a006ad70 0x7f26a006d220 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.409+0000 7f26c2fff700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26bc0c92d0 0x7f26bc1f2120 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.409+0000 7f26c2fff700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26bc0adea0 0x7f26bc070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.409+0000 7f26c2fff700 1 --2- 172.21.15.49:0/425485679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26bc0b12f0 0x7f26bc0acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.409+0000 7f26c2fff700 1 -- 172.21.15.49:0/425485679 >> 172.21.15.49:0/425485679 conn(0x7f26bc0b05d0 msgr2=0x7f26bc0aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:54.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.409+0000 7f26c2fff700 1 -- 172.21.15.49:0/425485679 shutdown_connections 2023-07-18T03:40:54.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.409+0000 7f26c2fff700 1 -- 172.21.15.49:0/425485679 wait complete. 2023-07-18T03:40:54.422 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot metadata set cephfs subvolume_0000000000974124 snapshot_0000000000227127 key value group_0000000000030717 2023-07-18T03:40:54.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.641+0000 7f01dffff700 1 Processor -- start 2023-07-18T03:40:54.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.641+0000 7f01dffff700 1 -- start start 2023-07-18T03:40:54.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.642+0000 7f01dffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f01d815ccd0 0x7f01d815d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:54.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.642+0000 7f01dffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01d80cf750 0x7f01d80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:54.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.642+0000 7f01dffff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d80d31c0 con 0x7f01d80d1b30 2023-07-18T03:40:54.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.643+0000 7f01dffff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d815d690 con 0x7f01d815ccd0 2023-07-18T03:40:54.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.643+0000 7f01dffff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d815d8b0 con 0x7f01d80cf750 2023-07-18T03:40:54.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.643+0000 7f01dd59a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f01d815ccd0 0x7f01d815d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:54.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.643+0000 7f01dd59a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f01d815ccd0 0x7f01d815d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54070/0 (socket says 172.21.15.49:54070) 2023-07-18T03:40:54.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.643+0000 7f01dd59a700 1 -- 172.21.15.49:0/2053858469 learned_addr learned my addr 172.21.15.49:0/2053858469 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:54.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.643+0000 7f01de59c700 1 --2- 172.21.15.49:0/2053858469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01d80cf750 0x7f01d80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:54.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.643+0000 7f01dd59a700 1 -- 172.21.15.49:0/2053858469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01d80cf750 msgr2=0x7f01d80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:54.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.643+0000 7f01dd59a700 1 --2- 172.21.15.49:0/2053858469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01d80cf750 0x7f01d80cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.643+0000 7f01dd59a700 1 -- 172.21.15.49:0/2053858469 >> v1:172.21.15.107:6789/0 conn(0x7f01d80d1b30 legacy=0x7f01d80d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:54.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.643+0000 7f01dd59a700 1 -- 172.21.15.49:0/2053858469 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d815db90 con 0x7f01d815ccd0 2023-07-18T03:40:54.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.644+0000 7f01dd59a700 1 --2- 172.21.15.49:0/2053858469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d815ccd0 0x7f01d815d150 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f01c80098f0 tx=0x7f01c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b4c5138f1cad67 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:54.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.644+0000 7f01dcd99700 1 -- 172.21.15.49:0/2053858469 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8011830 con 0x7f01d815ccd0 2023-07-18T03:40:54.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.644+0000 7f01dcd99700 1 -- 172.21.15.49:0/2053858469 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c8011990 con 0x7f01d815ccd0 2023-07-18T03:40:54.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.644+0000 7f01dcd99700 1 -- 172.21.15.49:0/2053858469 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8011d20 con 0x7f01d815ccd0 2023-07-18T03:40:54.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.644+0000 7f01dffff700 1 -- 172.21.15.49:0/2053858469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d815ccd0 msgr2=0x7f01d815d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:54.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.644+0000 7f01dffff700 1 --2- 172.21.15.49:0/2053858469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d815ccd0 0x7f01d815d150 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f01c80098f0 tx=0x7f01c8009c00 comp rx=0 tx=0).stop 2023-07-18T03:40:54.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.644+0000 7f01dffff700 1 -- 172.21.15.49:0/2053858469 shutdown_connections 2023-07-18T03:40:54.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.644+0000 7f01dffff700 1 --2- 172.21.15.49:0/2053858469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01d80cf750 0x7f01d80cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.644+0000 7f01dffff700 1 --2- 172.21.15.49:0/2053858469 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d815ccd0 0x7f01d815d150 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.645+0000 7f01dffff700 1 -- 172.21.15.49:0/2053858469 >> 172.21.15.49:0/2053858469 conn(0x7f01d806a660 msgr2=0x7f01d806aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:54.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.645+0000 7f01dffff700 1 -- 172.21.15.49:0/2053858469 shutdown_connections 2023-07-18T03:40:54.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.645+0000 7f01dffff700 1 -- 172.21.15.49:0/2053858469 wait complete. 2023-07-18T03:40:54.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.646+0000 7f01dffff700 1 Processor -- start 2023-07-18T03:40:54.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.647+0000 7f01dffff700 1 -- start start 2023-07-18T03:40:54.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.647+0000 7f01dffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d80cf750 0x7f01d81539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:54.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.647+0000 7f01dffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d80d1b30 0x7f01d80b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:54.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01dffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01d815ccd0 0x7f01d80b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01dffff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8154a70 con 0x7f01d80d1b30 2023-07-18T03:40:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01dffff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8153f20 con 0x7f01d80cf750 2023-07-18T03:40:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01dffff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d81e6230 con 0x7f01d815ccd0 2023-07-18T03:40:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01dd59a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d80d1b30 0x7f01d80b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01dd59a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d80d1b30 0x7f01d80b6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54078/0 (socket says 172.21.15.49:54078) 2023-07-18T03:40:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01dd59a700 1 -- 172.21.15.49:0/1456231719 learned_addr learned my addr 172.21.15.49:0/1456231719 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01dd59a700 1 -- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01d815ccd0 msgr2=0x7f01d80b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:54.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01dd59a700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01d815ccd0 0x7f01d80b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01dd59a700 1 -- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d80cf750 msgr2=0x7f01d81539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:54.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01ddd9b700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d80cf750 0x7f01d81539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:54.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01dd59a700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d80cf750 0x7f01d81539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01dd59a700 1 -- 172.21.15.49:0/1456231719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c8009550 con 0x7f01d80d1b30 2023-07-18T03:40:54.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.648+0000 7f01ddd9b700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d80cf750 0x7f01d81539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:54.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.649+0000 7f01dd59a700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d80d1b30 0x7f01d80b6610 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009c40 tx=0x7f01c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:54.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.649+0000 7f01ce7fc700 1 -- 172.21.15.49:0/1456231719 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c80036a0 con 0x7f01d80d1b30 2023-07-18T03:40:54.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.649+0000 7f01ce7fc700 1 -- 172.21.15.49:0/1456231719 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c801e070 con 0x7f01d80d1b30 2023-07-18T03:40:54.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.649+0000 7f01dffff700 1 -- 172.21.15.49:0/1456231719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d81e6510 con 0x7f01d80d1b30 2023-07-18T03:40:54.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.649+0000 7f01ce7fc700 1 -- 172.21.15.49:0/1456231719 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8003a60 con 0x7f01d80d1b30 2023-07-18T03:40:54.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.649+0000 7f01dffff700 1 -- 172.21.15.49:0/1456231719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d81e6ad0 con 0x7f01d80d1b30 2023-07-18T03:40:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.650+0000 7f01dffff700 1 -- 172.21.15.49:0/1456231719 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d8074350 con 0x7f01d80d1b30 2023-07-18T03:40:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.651+0000 7f01ce7fc700 1 -- 172.21.15.49:0/1456231719 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 6) v1 ==== 89088+0+0 (secure 0 0 0) 0x7f01c8003bc0 con 0x7f01d80d1b30 2023-07-18T03:40:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.652+0000 7f01ce7fc700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f01bc06ad70 0x7f01bc06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.652+0000 7f01ce7fc700 1 -- 172.21.15.49:0/1456231719 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f01c808a480 con 0x7f01d80d1b30 2023-07-18T03:40:54.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.652+0000 7f01ddd9b700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f01bc06ad70 0x7f01bc06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:54.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.653+0000 7f01ddd9b700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f01bc06ad70 0x7f01bc06d220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f01d4000f60 tx=0x7f01d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:54.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.656+0000 7f01ce7fc700 1 -- 172.21.15.49:0/1456231719 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f01c80554d0 con 0x7f01d80d1b30 2023-07-18T03:40:54.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.899+0000 7f01dffff700 1 -- 172.21.15.49:0/1456231719 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot metadata set", "vol_name": "cephfs", "sub_name": "subvolume_0000000000974124", "snap_name": "snapshot_0000000000227127", "key_name": "key", "value": "value", "group_name": "group_0000000000030717", "target": ["mon-mgr", ""]}) v1 -- 0x7f01d80d2e50 con 0x7f01bc06ad70 2023-07-18T03:40:54.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.917+0000 7f01ce7fc700 1 -- 172.21.15.49:0/1456231719 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f01d40013e0 con 0x7f01bc06ad70 2023-07-18T03:40:54.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.919+0000 7f01dffff700 1 -- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f01bc06ad70 msgr2=0x7f01bc06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:54.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.919+0000 7f01dffff700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f01bc06ad70 0x7f01bc06d220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f01d4000f60 tx=0x7f01d4009040 comp rx=0 tx=0).stop 2023-07-18T03:40:54.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.919+0000 7f01dffff700 1 -- 172.21.15.49:0/1456231719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d80d1b30 msgr2=0x7f01d80b6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:54.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.919+0000 7f01dffff700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d80d1b30 0x7f01d80b6610 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009c40 tx=0x7f01c8004750 comp rx=0 tx=0).stop 2023-07-18T03:40:54.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.919+0000 7f01dffff700 1 -- 172.21.15.49:0/1456231719 shutdown_connections 2023-07-18T03:40:54.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.919+0000 7f01dffff700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01d815ccd0 0x7f01d80b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.919+0000 7f01dffff700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01d80cf750 0x7f01d81539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.919+0000 7f01dffff700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f01bc06ad70 0x7f01bc06d220 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.919+0000 7f01dffff700 1 --2- 172.21.15.49:0/1456231719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01d80d1b30 0x7f01d80b6610 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:54.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.919+0000 7f01dffff700 1 -- 172.21.15.49:0/1456231719 >> 172.21.15.49:0/1456231719 conn(0x7f01d806a660 msgr2=0x7f01d8160250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:54.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.919+0000 7f01dffff700 1 -- 172.21.15.49:0/1456231719 shutdown_connections 2023-07-18T03:40:54.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:54.919+0000 7f01dffff700 1 -- 172.21.15.49:0/1456231719 wait complete. 2023-07-18T03:40:54.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:54.935 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/group_0000000000030717/subvolume_0000000000974124/.meta ./volumes/group_0000000000030717/subvolume_0000000000974124/.meta.stale_snap_section) 2023-07-18T03:40:55.213 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvolume_0000000000974124 snapshot_0000000000227127 group_0000000000030717 2023-07-18T03:40:55.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.437+0000 7f3a8b7ff700 1 Processor -- start 2023-07-18T03:40:55.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.437+0000 7f3a8b7ff700 1 -- start start 2023-07-18T03:40:55.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.438+0000 7f3a8b7ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3a840b8a60 0x7f3a840b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:55.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.438+0000 7f3a8b7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3a840d1220 0x7f3a840b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:55.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.438+0000 7f3a8b7ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a840b8e30 con 0x7f3a840b4530 2023-07-18T03:40:55.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.438+0000 7f3a8b7ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a840afaa0 con 0x7f3a840b8a60 2023-07-18T03:40:55.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.438+0000 7f3a8b7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a840afd80 con 0x7f3a840d1220 2023-07-18T03:40:55.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.439+0000 7f3a8959b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3a840b8a60 0x7f3a840b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:55.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.439+0000 7f3a88d9a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3a840b4530 0x7f3a840d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33358/0 (socket says 172.21.15.49:33358) 2023-07-18T03:40:55.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.439+0000 7f3a88d9a700 1 -- 172.21.15.49:0/393782644 learned_addr learned my addr 172.21.15.49:0/393782644 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:55.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.439+0000 7f3a89d9c700 1 --2- 172.21.15.49:0/393782644 >> v2:172.21.15.107:3300/0 conn(0x7f3a840d1220 0x7f3a840b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.439+0000 7f3a8959b700 1 -- 172.21.15.49:0/393782644 >> v2:172.21.15.107:3300/0 conn(0x7f3a840d1220 msgr2=0x7f3a840b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.439+0000 7f3a8959b700 1 --2- 172.21.15.49:0/393782644 >> v2:172.21.15.107:3300/0 conn(0x7f3a840d1220 0x7f3a840b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.439+0000 7f3a8959b700 1 -- 172.21.15.49:0/393782644 >> v1:172.21.15.49:6789/0 conn(0x7f3a840b4530 legacy=0x7f3a840d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.439+0000 7f3a8959b700 1 -- 172.21.15.49:0/393782644 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a840b0060 con 0x7f3a840b8a60 2023-07-18T03:40:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.439+0000 7f3a7bfff700 1 -- 172.21.15.49:0/393782644 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182092961 0 0) 0x7f3a840b8e30 con 0x7f3a840b4530 2023-07-18T03:40:55.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.440+0000 7f3a8959b700 1 --2- 172.21.15.49:0/393782644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a840b8a60 0x7f3a840b3ff0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3a840674e0 tx=0x7f3a80009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4243b67d4df601b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:55.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.440+0000 7f3a7bfff700 1 -- 172.21.15.49:0/393782644 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80011830 con 0x7f3a840b8a60 2023-07-18T03:40:55.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.440+0000 7f3a7bfff700 1 -- 172.21.15.49:0/393782644 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a80011990 con 0x7f3a840b8a60 2023-07-18T03:40:55.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.440+0000 7f3a7bfff700 1 -- 172.21.15.49:0/393782644 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80011d20 con 0x7f3a840b8a60 2023-07-18T03:40:55.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.440+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/393782644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a840b8a60 msgr2=0x7f3a840b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:55.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.440+0000 7f3a8b7ff700 1 --2- 172.21.15.49:0/393782644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a840b8a60 0x7f3a840b3ff0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3a840674e0 tx=0x7f3a80009b20 comp rx=0 tx=0).stop 2023-07-18T03:40:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.441+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/393782644 shutdown_connections 2023-07-18T03:40:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.441+0000 7f3a8b7ff700 1 --2- 172.21.15.49:0/393782644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a840b8a60 0x7f3a840b3ff0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.441+0000 7f3a8b7ff700 1 --2- 172.21.15.49:0/393782644 >> v2:172.21.15.107:3300/0 conn(0x7f3a840d1220 0x7f3a840b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.441+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/393782644 >> 172.21.15.49:0/393782644 conn(0x7f3a8409c1b0 msgr2=0x7f3a840d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.441+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/393782644 shutdown_connections 2023-07-18T03:40:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.441+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/393782644 wait complete. 2023-07-18T03:40:55.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.442+0000 7f3a8b7ff700 1 Processor -- start 2023-07-18T03:40:55.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.443+0000 7f3a8b7ff700 1 -- start start 2023-07-18T03:40:55.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8b7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a840b4530 0x7f3a84156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:55.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8b7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a840b8a60 0x7f3a84151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:55.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8b7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a840d1220 0x7f3a84157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:55.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8b7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a84156640 con 0x7f3a840b4530 2023-07-18T03:40:55.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8b7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8415ea80 con 0x7f3a840b8a60 2023-07-18T03:40:55.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8b7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a84151c90 con 0x7f3a840d1220 2023-07-18T03:40:55.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8959b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a840b4530 0x7f3a84156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:55.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8959b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a840b4530 0x7f3a84156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54096/0 (socket says 172.21.15.49:54096) 2023-07-18T03:40:55.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8959b700 1 -- 172.21.15.49:0/1656918680 learned_addr learned my addr 172.21.15.49:0/1656918680 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:55.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8959b700 1 -- 172.21.15.49:0/1656918680 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a840d1220 msgr2=0x7f3a84157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:55.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8959b700 1 --2- 172.21.15.49:0/1656918680 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a840d1220 0x7f3a84157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:55.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8959b700 1 -- 172.21.15.49:0/1656918680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a840b8a60 msgr2=0x7f3a84151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:55.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8959b700 1 --2- 172.21.15.49:0/1656918680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a840b8a60 0x7f3a84151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:55.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8959b700 1 -- 172.21.15.49:0/1656918680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a800094a0 con 0x7f3a840b4530 2023-07-18T03:40:55.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.444+0000 7f3a8959b700 1 --2- 172.21.15.49:0/1656918680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a840b4530 0x7f3a84156100 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3a800090b0 tx=0x7f3a80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:55.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.445+0000 7f3a79ffb700 1 -- 172.21.15.49:0/1656918680 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a800036a0 con 0x7f3a840b4530 2023-07-18T03:40:55.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.445+0000 7f3a79ffb700 1 -- 172.21.15.49:0/1656918680 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8001f070 con 0x7f3a840b4530 2023-07-18T03:40:55.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.445+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/1656918680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8415e1e0 con 0x7f3a840b4530 2023-07-18T03:40:55.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.445+0000 7f3a79ffb700 1 -- 172.21.15.49:0/1656918680 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80003a20 con 0x7f3a840b4530 2023-07-18T03:40:55.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.445+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/1656918680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8415e7a0 con 0x7f3a840b4530 2023-07-18T03:40:55.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.446+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/1656918680 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a84074350 con 0x7f3a840b4530 2023-07-18T03:40:55.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.447+0000 7f3a79ffb700 1 -- 172.21.15.49:0/1656918680 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f3a80003b80 con 0x7f3a840b4530 2023-07-18T03:40:55.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.448+0000 7f3a79ffb700 1 --2- 172.21.15.49:0/1656918680 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3a6806ae00 0x7f3a6806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:55.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.448+0000 7f3a79ffb700 1 -- 172.21.15.49:0/1656918680 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a80021070 con 0x7f3a840b4530 2023-07-18T03:40:55.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.448+0000 7f3a88d9a700 1 --2- 172.21.15.49:0/1656918680 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3a6806ae00 0x7f3a6806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:55.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.449+0000 7f3a88d9a700 1 --2- 172.21.15.49:0/1656918680 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3a6806ae00 0x7f3a6806d2b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3a740011e0 tx=0x7f3a740079b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:55.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.453+0000 7f3a79ffb700 1 -- 172.21.15.49:0/1656918680 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3a80055ed0 con 0x7f3a840b4530 2023-07-18T03:40:55.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.701+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/1656918680 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000974124", "snap_name": "snapshot_0000000000227127", "group_name": "group_0000000000030717", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a840b0620 con 0x7f3a6806ae00 2023-07-18T03:40:55.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.739+0000 7f3a79ffb700 1 -- 172.21.15.49:0/1656918680 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3a740070d0 con 0x7f3a6806ae00 2023-07-18T03:40:55.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.742+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/1656918680 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3a6806ae00 msgr2=0x7f3a6806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:55.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.742+0000 7f3a8b7ff700 1 --2- 172.21.15.49:0/1656918680 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3a6806ae00 0x7f3a6806d2b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3a740011e0 tx=0x7f3a740079b0 comp rx=0 tx=0).stop 2023-07-18T03:40:55.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.742+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/1656918680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a840b4530 msgr2=0x7f3a84156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:55.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.742+0000 7f3a8b7ff700 1 --2- 172.21.15.49:0/1656918680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a840b4530 0x7f3a84156100 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3a800090b0 tx=0x7f3a80004750 comp rx=0 tx=0).stop 2023-07-18T03:40:55.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.742+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/1656918680 shutdown_connections 2023-07-18T03:40:55.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.742+0000 7f3a8b7ff700 1 --2- 172.21.15.49:0/1656918680 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a840d1220 0x7f3a84157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:55.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.742+0000 7f3a8b7ff700 1 --2- 172.21.15.49:0/1656918680 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3a6806ae00 0x7f3a6806d2b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:55.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.742+0000 7f3a8b7ff700 1 --2- 172.21.15.49:0/1656918680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a840b4530 0x7f3a84156100 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:55.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.742+0000 7f3a8b7ff700 1 --2- 172.21.15.49:0/1656918680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a840b8a60 0x7f3a84151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:55.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.742+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/1656918680 >> 172.21.15.49:0/1656918680 conn(0x7f3a8409c1b0 msgr2=0x7f3a8415bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:55.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.742+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/1656918680 shutdown_connections 2023-07-18T03:40:55.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:55.742+0000 7f3a8b7ff700 1 -- 172.21.15.49:0/1656918680 wait complete. 2023-07-18T03:40:55.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:55.756 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/group_0000000000030717/subvolume_0000000000974124/.meta.stale_snap_section ./volumes/group_0000000000030717/subvolume_0000000000974124/.meta) 2023-07-18T03:40:55.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:55.839 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_snapshot_0000000000227127 ./volumes/group_0000000000030717/subvolume_0000000000974124/.meta) 2023-07-18T03:40:55.908 INFO:teuthology.orchestra.run.smithi049.stdout:[SNAP_METADATA_snapshot_0000000000227127] 2023-07-18T03:40:55.913 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvolume_0000000000974124 group_0000000000030717 2023-07-18T03:40:56.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.130+0000 7f4a667ff700 1 Processor -- start 2023-07-18T03:40:56.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.130+0000 7f4a667ff700 1 -- start start 2023-07-18T03:40:56.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.131+0000 7f4a667ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a600b0480 0x7f4a600b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:56.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.131+0000 7f4a667ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4a60070520 0x7f4a600709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:56.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.131+0000 7f4a667ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a6015b530 con 0x7f4a600ca1e0 2023-07-18T03:40:56.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.132+0000 7f4a667ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6015aa10 con 0x7f4a60070520 2023-07-18T03:40:56.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.132+0000 7f4a667ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6015eef0 con 0x7f4a600b0480 2023-07-18T03:40:56.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.132+0000 7f4a64d9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4a60070520 0x7f4a600709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:56.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.132+0000 7f4a64d9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4a60070520 0x7f4a600709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54100/0 (socket says 172.21.15.49:54100) 2023-07-18T03:40:56.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.132+0000 7f4a64d9c700 1 -- 172.21.15.49:0/3184210116 learned_addr learned my addr 172.21.15.49:0/3184210116 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:56.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.132+0000 7f4a5f7fe700 1 --2- 172.21.15.49:0/3184210116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a600b0480 0x7f4a600b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:56.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.132+0000 7f4a64d9c700 1 -- 172.21.15.49:0/3184210116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a600b0480 msgr2=0x7f4a600b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:56.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.132+0000 7f4a64d9c700 1 --2- 172.21.15.49:0/3184210116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a600b0480 0x7f4a600b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:56.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.132+0000 7f4a64d9c700 1 -- 172.21.15.49:0/3184210116 >> v1:172.21.15.107:6789/0 conn(0x7f4a600ca1e0 legacy=0x7f4a600ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:56.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.132+0000 7f4a64d9c700 1 -- 172.21.15.49:0/3184210116 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6015ac00 con 0x7f4a60070520 2023-07-18T03:40:56.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.132+0000 7f4a64d9c700 1 --2- 172.21.15.49:0/3184210116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a60070520 0x7f4a600709a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009e80 tx=0x7f4a5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69029ad52ffd7487 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:56.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.133+0000 7f4a5effd700 1 -- 172.21.15.49:0/3184210116 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800e070 con 0x7f4a60070520 2023-07-18T03:40:56.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.133+0000 7f4a5effd700 1 -- 172.21.15.49:0/3184210116 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5800b980 con 0x7f4a60070520 2023-07-18T03:40:56.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.133+0000 7f4a5effd700 1 -- 172.21.15.49:0/3184210116 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800bd30 con 0x7f4a60070520 2023-07-18T03:40:56.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.133+0000 7f4a667ff700 1 -- 172.21.15.49:0/3184210116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a60070520 msgr2=0x7f4a600709a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:56.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.133+0000 7f4a667ff700 1 --2- 172.21.15.49:0/3184210116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a60070520 0x7f4a600709a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009e80 tx=0x7f4a5800b330 comp rx=0 tx=0).stop 2023-07-18T03:40:56.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.133+0000 7f4a667ff700 1 -- 172.21.15.49:0/3184210116 shutdown_connections 2023-07-18T03:40:56.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.133+0000 7f4a667ff700 1 --2- 172.21.15.49:0/3184210116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a60070520 0x7f4a600709a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:56.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.133+0000 7f4a667ff700 1 --2- 172.21.15.49:0/3184210116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a600b0480 0x7f4a600b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:56.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.133+0000 7f4a667ff700 1 -- 172.21.15.49:0/3184210116 >> 172.21.15.49:0/3184210116 conn(0x7f4a600cea00 msgr2=0x7f4a600cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:56.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.134+0000 7f4a667ff700 1 -- 172.21.15.49:0/3184210116 shutdown_connections 2023-07-18T03:40:56.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.134+0000 7f4a667ff700 1 -- 172.21.15.49:0/3184210116 wait complete. 2023-07-18T03:40:56.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.136+0000 7f4a667ff700 1 Processor -- start 2023-07-18T03:40:56.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.136+0000 7f4a667ff700 1 -- start start 2023-07-18T03:40:56.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.137+0000 7f4a667ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a60070520 0x7f4a601f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:56.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.137+0000 7f4a667ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a600b0480 0x7f4a600c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:56.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.137+0000 7f4a667ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a600ca1e0 0x7f4a60154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:56.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a667ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a600c4090 con 0x7f4a600ca1e0 2023-07-18T03:40:56.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a667ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a600c4370 con 0x7f4a600b0480 2023-07-18T03:40:56.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a667ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a600c4650 con 0x7f4a60070520 2023-07-18T03:40:56.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a64d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a600ca1e0 0x7f4a60154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:56.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a64d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a600ca1e0 0x7f4a60154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54108/0 (socket says 172.21.15.49:54108) 2023-07-18T03:40:56.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a64d9c700 1 -- 172.21.15.49:0/3370690907 learned_addr learned my addr 172.21.15.49:0/3370690907 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:56.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a5ffff700 1 --2- 172.21.15.49:0/3370690907 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a60070520 0x7f4a601f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:56.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a5f7fe700 1 --2- 172.21.15.49:0/3370690907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a600b0480 0x7f4a600c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:56.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a64d9c700 1 -- 172.21.15.49:0/3370690907 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a60070520 msgr2=0x7f4a601f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:56.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a64d9c700 1 --2- 172.21.15.49:0/3370690907 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a60070520 0x7f4a601f2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:56.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a64d9c700 1 -- 172.21.15.49:0/3370690907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a600b0480 msgr2=0x7f4a600c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:56.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a64d9c700 1 --2- 172.21.15.49:0/3370690907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a600b0480 0x7f4a600c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:56.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.138+0000 7f4a64d9c700 1 -- 172.21.15.49:0/3370690907 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a58009b30 con 0x7f4a600ca1e0 2023-07-18T03:40:56.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.139+0000 7f4a64d9c700 1 --2- 172.21.15.49:0/3370690907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a600ca1e0 0x7f4a60154cc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4a600673b0 tx=0x7f4a58004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:56.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.139+0000 7f4a5cff9700 1 -- 172.21.15.49:0/3370690907 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a580099b0 con 0x7f4a600ca1e0 2023-07-18T03:40:56.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.139+0000 7f4a5cff9700 1 -- 172.21.15.49:0/3370690907 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a58004250 con 0x7f4a600ca1e0 2023-07-18T03:40:56.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.139+0000 7f4a667ff700 1 -- 172.21.15.49:0/3370690907 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a600c4930 con 0x7f4a600ca1e0 2023-07-18T03:40:56.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.139+0000 7f4a5cff9700 1 -- 172.21.15.49:0/3370690907 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58004600 con 0x7f4a600ca1e0 2023-07-18T03:40:56.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.139+0000 7f4a667ff700 1 -- 172.21.15.49:0/3370690907 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a601f27b0 con 0x7f4a600ca1e0 2023-07-18T03:40:56.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.140+0000 7f4a667ff700 1 -- 172.21.15.49:0/3370690907 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a600c4e50 con 0x7f4a600ca1e0 2023-07-18T03:40:56.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.141+0000 7f4a5cff9700 1 -- 172.21.15.49:0/3370690907 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f4a58004760 con 0x7f4a600ca1e0 2023-07-18T03:40:56.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.142+0000 7f4a5cff9700 1 --2- 172.21.15.49:0/3370690907 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4a4c06ae00 0x7f4a4c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:56.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.142+0000 7f4a5cff9700 1 -- 172.21.15.49:0/3370690907 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4a5808ac80 con 0x7f4a600ca1e0 2023-07-18T03:40:56.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.142+0000 7f4a5ffff700 1 --2- 172.21.15.49:0/3370690907 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4a4c06ae00 0x7f4a4c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:56.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.143+0000 7f4a5ffff700 1 --2- 172.21.15.49:0/3370690907 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4a4c06ae00 0x7f4a4c06d2b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4a50007fd0 tx=0x7f4a50009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:56.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.146+0000 7f4a5cff9700 1 -- 172.21.15.49:0/3370690907 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4a600c4e50 con 0x7f4a600ca1e0 2023-07-18T03:40:56.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.390+0000 7f4a667ff700 1 -- 172.21.15.49:0/3370690907 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvolume_0000000000974124", "group_name": "group_0000000000030717", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a6015b1c0 con 0x7f4a4c06ae00 2023-07-18T03:40:56.414 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-18T03:40:56.420+0000 7f45ce8fe700 -1 mgr.server reply reply (61) No data available error in getxattr 2023-07-18T03:40:56.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:56.419+0000 7f4a5cff9700 1 -- 172.21.15.49:0/3370690907 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: -61 error in getxattr) v1 ==== 25+0+0 (secure 0 0 0) 0x7f4a500014c0 con 0x7f4a4c06ae00 2023-07-18T03:40:56.416 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENODATA: error in getxattr 2023-07-18T03:40:56.424 DEBUG:teuthology.orchestra.run:got remote process result: 61 2023-07-18T03:40:56.425 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-18T03:40:56.426 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-18T03:40:56.519 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-18T03:40:56.633 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-18T03:40:56.634 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:40:56.634 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:56.678 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:40:56.678 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:56.763 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:57.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.016+0000 7faa995ff700 1 Processor -- start 2023-07-18T03:40:57.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.016+0000 7faa995ff700 1 -- start start 2023-07-18T03:40:57.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.018+0000 7faa995ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faa94160030 0x7faa9406c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.018+0000 7faa995ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa940d1160 0x7faa940b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.018+0000 7faa995ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa940d2cc0 con 0x7faa940b5cd0 2023-07-18T03:40:57.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.018+0000 7faa995ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa940afbb0 con 0x7faa94160030 2023-07-18T03:40:57.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.018+0000 7faa995ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa940afe90 con 0x7faa940d1160 2023-07-18T03:40:57.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.018+0000 7faa927fc700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7faa940b5cd0 0x7faa940d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33372/0 (socket says 172.21.15.49:33372) 2023-07-18T03:40:57.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.018+0000 7faa927fc700 1 -- 172.21.15.49:0/4009355081 learned_addr learned my addr 172.21.15.49:0/4009355081 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:57.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.018+0000 7faa92ffd700 1 --2- 172.21.15.49:0/4009355081 >> v2:172.21.15.49:3300/0 conn(0x7faa94160030 0x7faa9406c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.018+0000 7faa937fe700 1 --2- 172.21.15.49:0/4009355081 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa940d1160 0x7faa940b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.018+0000 7faa91ffb700 1 -- 172.21.15.49:0/4009355081 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141906412 0 0) 0x7faa940d2cc0 con 0x7faa940b5cd0 2023-07-18T03:40:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.018+0000 7faa91ffb700 1 -- 172.21.15.49:0/4009355081 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa7c0035b0 con 0x7faa940b5cd0 2023-07-18T03:40:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.019+0000 7faa92ffd700 1 -- 172.21.15.49:0/4009355081 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa940d1160 msgr2=0x7faa940b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.019+0000 7faa92ffd700 1 --2- 172.21.15.49:0/4009355081 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa940d1160 0x7faa940b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.019+0000 7faa92ffd700 1 -- 172.21.15.49:0/4009355081 >> v1:172.21.15.49:6789/0 conn(0x7faa940b5cd0 legacy=0x7faa940d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.019+0000 7faa92ffd700 1 -- 172.21.15.49:0/4009355081 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa940b0170 con 0x7faa94160030 2023-07-18T03:40:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.019+0000 7faa92ffd700 1 --2- 172.21.15.49:0/4009355081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa94160030 0x7faa9406c0f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faa84009840 tx=0x7faa84009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411aa3b47379b768 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.019+0000 7faa91ffb700 1 -- 172.21.15.49:0/4009355081 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84011830 con 0x7faa94160030 2023-07-18T03:40:57.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.019+0000 7faa91ffb700 1 -- 172.21.15.49:0/4009355081 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa84011990 con 0x7faa94160030 2023-07-18T03:40:57.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.019+0000 7faa91ffb700 1 -- 172.21.15.49:0/4009355081 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84011d20 con 0x7faa94160030 2023-07-18T03:40:57.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.020+0000 7faa995ff700 1 -- 172.21.15.49:0/4009355081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa94160030 msgr2=0x7faa9406c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.020+0000 7faa995ff700 1 --2- 172.21.15.49:0/4009355081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa94160030 0x7faa9406c0f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faa84009840 tx=0x7faa84009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:57.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.020+0000 7faa995ff700 1 -- 172.21.15.49:0/4009355081 shutdown_connections 2023-07-18T03:40:57.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.020+0000 7faa995ff700 1 --2- 172.21.15.49:0/4009355081 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa940d1160 0x7faa940b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.020+0000 7faa995ff700 1 --2- 172.21.15.49:0/4009355081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa94160030 0x7faa9406c0f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.020+0000 7faa995ff700 1 -- 172.21.15.49:0/4009355081 >> 172.21.15.49:0/4009355081 conn(0x7faa9409c190 msgr2=0x7faa940d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:57.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.020+0000 7faa995ff700 1 -- 172.21.15.49:0/4009355081 shutdown_connections 2023-07-18T03:40:57.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.020+0000 7faa995ff700 1 -- 172.21.15.49:0/4009355081 wait complete. 2023-07-18T03:40:57.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.022+0000 7faa995ff700 1 Processor -- start 2023-07-18T03:40:57.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.023+0000 7faa995ff700 1 -- start start 2023-07-18T03:40:57.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa995ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa940b5cd0 0x7faa94156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa995ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa940d1160 0x7faa941552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa995ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa94160030 0x7faa94159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa995ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa940b22b0 con 0x7faa940b5cd0 2023-07-18T03:40:57.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa995ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa94157570 con 0x7faa940d1160 2023-07-18T03:40:57.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa995ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa94157850 con 0x7faa94160030 2023-07-18T03:40:57.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa92ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa940b5cd0 0x7faa94156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa92ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa940b5cd0 0x7faa94156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54134/0 (socket says 172.21.15.49:54134) 2023-07-18T03:40:57.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa92ffd700 1 -- 172.21.15.49:0/3235146703 learned_addr learned my addr 172.21.15.49:0/3235146703 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:57.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa927fc700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa940d1160 0x7faa941552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa937fe700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa94160030 0x7faa94159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa92ffd700 1 -- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa94160030 msgr2=0x7faa94159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa92ffd700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa94160030 0x7faa94159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa92ffd700 1 -- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa940d1160 msgr2=0x7faa941552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa92ffd700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa940d1160 0x7faa941552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa92ffd700 1 -- 172.21.15.49:0/3235146703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa840094a0 con 0x7faa940b5cd0 2023-07-18T03:40:57.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.024+0000 7faa927fc700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa940d1160 0x7faa941552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:57.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.025+0000 7faa92ffd700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa940b5cd0 0x7faa94156d40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faa84009fd0 tx=0x7faa84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.025+0000 7faa7b7fe700 1 -- 172.21.15.49:0/3235146703 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa840036a0 con 0x7faa940b5cd0 2023-07-18T03:40:57.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.025+0000 7faa7b7fe700 1 -- 172.21.15.49:0/3235146703 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8401e070 con 0x7faa940b5cd0 2023-07-18T03:40:57.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.025+0000 7faa995ff700 1 -- 172.21.15.49:0/3235146703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa94157b30 con 0x7faa940b5cd0 2023-07-18T03:40:57.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.025+0000 7faa7b7fe700 1 -- 172.21.15.49:0/3235146703 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84003a60 con 0x7faa940b5cd0 2023-07-18T03:40:57.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.025+0000 7faa995ff700 1 -- 172.21.15.49:0/3235146703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa94157280 con 0x7faa940b5cd0 2023-07-18T03:40:57.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.026+0000 7faa995ff700 1 -- 172.21.15.49:0/3235146703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9404e280 con 0x7faa940b5cd0 2023-07-18T03:40:57.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.027+0000 7faa7b7fe700 1 -- 172.21.15.49:0/3235146703 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7faa84003bc0 con 0x7faa940b5cd0 2023-07-18T03:40:57.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.028+0000 7faa7b7fe700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7faa8006ae00 0x7faa8006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.028+0000 7faa7b7fe700 1 -- 172.21.15.49:0/3235146703 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faa8408a550 con 0x7faa940b5cd0 2023-07-18T03:40:57.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.029+0000 7faa927fc700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7faa8006ae00 0x7faa8006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.029+0000 7faa927fc700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7faa8006ae00 0x7faa8006d2b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faa94158cf0 tx=0x7faa88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.032+0000 7faa7b7fe700 1 -- 172.21.15.49:0/3235146703 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7faa840554e0 con 0x7faa940b5cd0 2023-07-18T03:40:57.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.239+0000 7faa995ff700 1 -- 172.21.15.49:0/3235146703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa940d38c0 con 0x7faa940b5cd0 2023-07-18T03:40:57.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.241+0000 7faa7b7fe700 1 -- 172.21.15.49:0/3235146703 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18257 (secure 0 0 0) 0x7faa84058b00 con 0x7faa940b5cd0 2023-07-18T03:40:57.236 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:40:57.237 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":49,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:40:41.947761+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:40:38.792808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-18T03:40:39.438590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:57.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.243+0000 7faa995ff700 1 -- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7faa8006ae00 msgr2=0x7faa8006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.243+0000 7faa995ff700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7faa8006ae00 0x7faa8006d2b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faa94158cf0 tx=0x7faa88009040 comp rx=0 tx=0).stop 2023-07-18T03:40:57.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.243+0000 7faa995ff700 1 -- 172.21.15.49:0/3235146703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa940b5cd0 msgr2=0x7faa94156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.243+0000 7faa995ff700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa940b5cd0 0x7faa94156d40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faa84009fd0 tx=0x7faa84004750 comp rx=0 tx=0).stop 2023-07-18T03:40:57.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.243+0000 7faa995ff700 1 -- 172.21.15.49:0/3235146703 shutdown_connections 2023-07-18T03:40:57.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.243+0000 7faa995ff700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa94160030 0x7faa94159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.243+0000 7faa995ff700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7faa8006ae00 0x7faa8006d2b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.243+0000 7faa995ff700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa940b5cd0 0x7faa94156d40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.243+0000 7faa995ff700 1 --2- 172.21.15.49:0/3235146703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa940d1160 0x7faa941552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.243+0000 7faa995ff700 1 -- 172.21.15.49:0/3235146703 >> 172.21.15.49:0/3235146703 conn(0x7faa9409c190 msgr2=0x7faa940d2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:57.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.243+0000 7faa995ff700 1 -- 172.21.15.49:0/3235146703 shutdown_connections 2023-07-18T03:40:57.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.243+0000 7faa995ff700 1 -- 172.21.15.49:0/3235146703 wait complete. 2023-07-18T03:40:57.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:40:38.792808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:40:39.438590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:40:57.259 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:40:57.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.472+0000 7f2c71bff700 1 Processor -- start 2023-07-18T03:40:57.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.472+0000 7f2c71bff700 1 -- start start 2023-07-18T03:40:57.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.473+0000 7f2c71bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c6c0b4860 0x7f2c6c0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.473+0000 7f2c71bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2c6c0d12c0 0x7f2c6c0d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.473+0000 7f2c71bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6c0d02a0 con 0x7f2c6c0b8660 2023-07-18T03:40:57.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.473+0000 7f2c71bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6c0b5210 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.473+0000 7f2c71bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0cfce0 con 0x7f2c6c0b4860 2023-07-18T03:40:57.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.474+0000 7f2c6bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2c6c0d12c0 0x7f2c6c0d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.474+0000 7f2c6bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2c6c0d12c0 0x7f2c6c0d1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54144/0 (socket says 172.21.15.49:54144) 2023-07-18T03:40:57.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.474+0000 7f2c6bfff700 1 -- 172.21.15.49:0/1414505526 learned_addr learned my addr 172.21.15.49:0/1414505526 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:57.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.474+0000 7f2c63fff700 1 --2- 172.21.15.49:0/1414505526 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c6c0b4860 0x7f2c6c0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.474+0000 7f2c6bfff700 1 -- 172.21.15.49:0/1414505526 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c6c0b4860 msgr2=0x7f2c6c0b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:57.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.474+0000 7f2c6bfff700 1 --2- 172.21.15.49:0/1414505526 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c6c0b4860 0x7f2c6c0b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.474+0000 7f2c6bfff700 1 -- 172.21.15.49:0/1414505526 >> v1:172.21.15.107:6789/0 conn(0x7f2c6c0b8660 legacy=0x7f2c6c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:57.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.474+0000 7f2c6bfff700 1 -- 172.21.15.49:0/1414505526 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6c0cffc0 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.475+0000 7f2c6bfff700 1 --2- 172.21.15.49:0/1414505526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c6c0d12c0 0x7f2c6c0d1700 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009e50 tx=0x7f2c6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcaaa5605c306f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.475+0000 7f2c6affd700 1 -- 172.21.15.49:0/1414505526 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400e070 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.475+0000 7f2c6affd700 1 -- 172.21.15.49:0/1414505526 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6400b9c0 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.475+0000 7f2c6affd700 1 -- 172.21.15.49:0/1414505526 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400bd70 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.475+0000 7f2c71bff700 1 -- 172.21.15.49:0/1414505526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c6c0d12c0 msgr2=0x7f2c6c0d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.475+0000 7f2c71bff700 1 --2- 172.21.15.49:0/1414505526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c6c0d12c0 0x7f2c6c0d1700 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009e50 tx=0x7f2c6400b330 comp rx=0 tx=0).stop 2023-07-18T03:40:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.475+0000 7f2c71bff700 1 -- 172.21.15.49:0/1414505526 shutdown_connections 2023-07-18T03:40:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.476+0000 7f2c71bff700 1 --2- 172.21.15.49:0/1414505526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c6c0d12c0 0x7f2c6c0d1700 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.476+0000 7f2c71bff700 1 --2- 172.21.15.49:0/1414505526 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c6c0b4860 0x7f2c6c0b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.476+0000 7f2c71bff700 1 -- 172.21.15.49:0/1414505526 >> 172.21.15.49:0/1414505526 conn(0x7f2c6c066860 msgr2=0x7f2c6c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.476+0000 7f2c71bff700 1 -- 172.21.15.49:0/1414505526 shutdown_connections 2023-07-18T03:40:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.476+0000 7f2c71bff700 1 -- 172.21.15.49:0/1414505526 wait complete. 2023-07-18T03:40:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.477+0000 7f2c71bff700 1 Processor -- start 2023-07-18T03:40:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.478+0000 7f2c71bff700 1 -- start start 2023-07-18T03:40:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.478+0000 7f2c71bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c6c0b4860 0x7f2c6c144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c71bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c6c0b8660 0x7f2c6c145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c71bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c6c0d12c0 0x7f2c6c1f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c71bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c145750 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c71bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c144b40 con 0x7f2c6c0b4860 2023-07-18T03:40:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c71bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c144e20 con 0x7f2c6c0b8660 2023-07-18T03:40:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c6bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c6c0d12c0 0x7f2c6c1f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c6bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c6c0d12c0 0x7f2c6c1f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54152/0 (socket says 172.21.15.49:54152) 2023-07-18T03:40:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c6bfff700 1 -- 172.21.15.49:0/4121959684 learned_addr learned my addr 172.21.15.49:0/4121959684 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c63fff700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c6c0b8660 0x7f2c6c145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c6b7fe700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c6c0b4860 0x7f2c6c144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c6bfff700 1 -- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c6c0b8660 msgr2=0x7f2c6c145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c6bfff700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c6c0b8660 0x7f2c6c145be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c6bfff700 1 -- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c6c0b4860 msgr2=0x7f2c6c144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c6bfff700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c6c0b4860 0x7f2c6c144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c6bfff700 1 -- 172.21.15.49:0/4121959684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64009b30 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c63fff700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c6c0b8660 0x7f2c6c145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.479+0000 7f2c6b7fe700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c6c0b4860 0x7f2c6c144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.480+0000 7f2c6bfff700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c6c0d12c0 0x7f2c6c1f1e80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0673b0 tx=0x7f2c64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.480+0000 7f2c68ff9700 1 -- 172.21.15.49:0/4121959684 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64004960 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.480+0000 7f2c68ff9700 1 -- 172.21.15.49:0/4121959684 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c640099b0 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.480+0000 7f2c68ff9700 1 -- 172.21.15.49:0/4121959684 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6401c680 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.480+0000 7f2c71bff700 1 -- 172.21.15.49:0/4121959684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6c145100 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.480+0000 7f2c71bff700 1 -- 172.21.15.49:0/4121959684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c146120 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.481+0000 7f2c71bff700 1 -- 172.21.15.49:0/4121959684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c145900 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.482+0000 7f2c68ff9700 1 -- 172.21.15.49:0/4121959684 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f2c6401c7e0 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.483+0000 7f2c68ff9700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2c4c06ae00 0x7f2c4c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.483+0000 7f2c68ff9700 1 -- 172.21.15.49:0/4121959684 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2c6408b450 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.483+0000 7f2c6b7fe700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2c4c06ae00 0x7f2c4c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.484+0000 7f2c6b7fe700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2c4c06ae00 0x7f2c4c06d2b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000f60 tx=0x7f2c5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.487+0000 7f2c68ff9700 1 -- 172.21.15.49:0/4121959684 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2c6c145900 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.694+0000 7f2c71bff700 1 -- 172.21.15.49:0/4121959684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2c6c1462b0 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.694+0000 7f2c68ff9700 1 -- 172.21.15.49:0/4121959684 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2c640562d0 con 0x7f2c6c0d12c0 2023-07-18T03:40:57.690 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:40:57.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.696+0000 7f2c71bff700 1 -- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2c4c06ae00 msgr2=0x7f2c4c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.696+0000 7f2c71bff700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2c4c06ae00 0x7f2c4c06d2b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000f60 tx=0x7f2c5c009040 comp rx=0 tx=0).stop 2023-07-18T03:40:57.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.696+0000 7f2c71bff700 1 -- 172.21.15.49:0/4121959684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c6c0d12c0 msgr2=0x7f2c6c1f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.696+0000 7f2c71bff700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c6c0d12c0 0x7f2c6c1f1e80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0673b0 tx=0x7f2c64004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.697+0000 7f2c71bff700 1 -- 172.21.15.49:0/4121959684 shutdown_connections 2023-07-18T03:40:57.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.697+0000 7f2c71bff700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2c4c06ae00 0x7f2c4c06d2b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.697+0000 7f2c71bff700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c6c0d12c0 0x7f2c6c1f1e80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.697+0000 7f2c71bff700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c6c0b4860 0x7f2c6c144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.697+0000 7f2c71bff700 1 --2- 172.21.15.49:0/4121959684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c6c0b8660 0x7f2c6c145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.697+0000 7f2c71bff700 1 -- 172.21.15.49:0/4121959684 >> 172.21.15.49:0/4121959684 conn(0x7f2c6c066860 msgr2=0x7f2c6c0d16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:57.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.697+0000 7f2c71bff700 1 -- 172.21.15.49:0/4121959684 shutdown_connections 2023-07-18T03:40:57.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.697+0000 7f2c71bff700 1 -- 172.21.15.49:0/4121959684 wait complete. 2023-07-18T03:40:57.710 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:40:57.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.924+0000 7fb9647ff700 1 Processor -- start 2023-07-18T03:40:57.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.924+0000 7fb9647ff700 1 -- start start 2023-07-18T03:40:57.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.925+0000 7fb9647ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb95c0d1e40 0x7fb95c0cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.925+0000 7fb9647ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb95c0c71b0 0x7fb95c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.925+0000 7fb9647ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c0cd5a0 con 0x7fb95c15c860 2023-07-18T03:40:57.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.925+0000 7fb9647ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c146740 con 0x7fb95c0d1e40 2023-07-18T03:40:57.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.925+0000 7fb9647ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0cd7f0 con 0x7fb95c0c71b0 2023-07-18T03:40:57.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.925+0000 7fb962d9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb95c0c71b0 0x7fb95c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.925+0000 7fb961d9a700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb95c0d1e40 0x7fb95c0cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.925+0000 7fb962d9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb95c0c71b0 0x7fb95c0b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:60530/0 (socket says 172.21.15.49:60530) 2023-07-18T03:40:57.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.925+0000 7fb962d9c700 1 -- 172.21.15.49:0/3196197609 learned_addr learned my addr 172.21.15.49:0/3196197609 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:57.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.926+0000 7fb962d9c700 1 -- 172.21.15.49:0/3196197609 >> v2:172.21.15.107:3300/0 conn(0x7fb95c0d1e40 msgr2=0x7fb95c0cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:57.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.926+0000 7fb962d9c700 1 --2- 172.21.15.49:0/3196197609 >> v2:172.21.15.107:3300/0 conn(0x7fb95c0d1e40 0x7fb95c0cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.926+0000 7fb962d9c700 1 -- 172.21.15.49:0/3196197609 >> v1:172.21.15.107:6789/0 conn(0x7fb95c15c860 legacy=0x7fb95c0c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.926+0000 7fb962d9c700 1 -- 172.21.15.49:0/3196197609 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c0cdad0 con 0x7fb95c0c71b0 2023-07-18T03:40:57.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.926+0000 7fb961d9a700 1 --2- 172.21.15.49:0/3196197609 >> v2:172.21.15.107:3300/0 conn(0x7fb95c0d1e40 0x7fb95c0cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:57.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.926+0000 7fb962d9c700 1 --2- 172.21.15.49:0/3196197609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb95c0c71b0 0x7fb95c0b49d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb954009e50 tx=0x7fb95400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6009318076fe79c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.926+0000 7fb961599700 1 -- 172.21.15.49:0/3196197609 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95400e070 con 0x7fb95c0c71b0 2023-07-18T03:40:57.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.927+0000 7fb961599700 1 -- 172.21.15.49:0/3196197609 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95400b910 con 0x7fb95c0c71b0 2023-07-18T03:40:57.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.927+0000 7fb961599700 1 -- 172.21.15.49:0/3196197609 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95400bcc0 con 0x7fb95c0c71b0 2023-07-18T03:40:57.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.927+0000 7fb9647ff700 1 -- 172.21.15.49:0/3196197609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb95c0c71b0 msgr2=0x7fb95c0b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.927+0000 7fb9647ff700 1 --2- 172.21.15.49:0/3196197609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb95c0c71b0 0x7fb95c0b49d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb954009e50 tx=0x7fb95400b330 comp rx=0 tx=0).stop 2023-07-18T03:40:57.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.927+0000 7fb9647ff700 1 -- 172.21.15.49:0/3196197609 shutdown_connections 2023-07-18T03:40:57.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.927+0000 7fb9647ff700 1 --2- 172.21.15.49:0/3196197609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb95c0c71b0 0x7fb95c0b49d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.927+0000 7fb9647ff700 1 --2- 172.21.15.49:0/3196197609 >> v2:172.21.15.107:3300/0 conn(0x7fb95c0d1e40 0x7fb95c0cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.927+0000 7fb9647ff700 1 -- 172.21.15.49:0/3196197609 >> 172.21.15.49:0/3196197609 conn(0x7fb95c065bd0 msgr2=0x7fb95c065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:57.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.927+0000 7fb9647ff700 1 -- 172.21.15.49:0/3196197609 shutdown_connections 2023-07-18T03:40:57.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.927+0000 7fb9647ff700 1 -- 172.21.15.49:0/3196197609 wait complete. 2023-07-18T03:40:57.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.929+0000 7fb9647ff700 1 Processor -- start 2023-07-18T03:40:57.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.929+0000 7fb9647ff700 1 -- start start 2023-07-18T03:40:57.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.930+0000 7fb9647ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb95c0c71b0 0x7fb95c15d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.930+0000 7fb9647ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb95c0d1e40 0x7fb95c0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.930+0000 7fb9647ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb95c15c860 0x7fb95c156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.930+0000 7fb9647ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0b74f0 con 0x7fb95c15c860 2023-07-18T03:40:57.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.930+0000 7fb9647ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0b77d0 con 0x7fb95c0c71b0 2023-07-18T03:40:57.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.930+0000 7fb9647ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0b7ab0 con 0x7fb95c0d1e40 2023-07-18T03:40:57.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.930+0000 7fb962d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb95c15c860 0x7fb95c156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.931+0000 7fb962d9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb95c15c860 0x7fb95c156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54158/0 (socket says 172.21.15.49:54158) 2023-07-18T03:40:57.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.931+0000 7fb962d9c700 1 -- 172.21.15.49:0/813471212 learned_addr learned my addr 172.21.15.49:0/813471212 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:57.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.931+0000 7fb96259b700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb95c0c71b0 0x7fb95c15d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.931+0000 7fb961d9a700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb95c0d1e40 0x7fb95c0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.931+0000 7fb962d9c700 1 -- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb95c0d1e40 msgr2=0x7fb95c0ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.931+0000 7fb962d9c700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb95c0d1e40 0x7fb95c0ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.931+0000 7fb962d9c700 1 -- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb95c0c71b0 msgr2=0x7fb95c15d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.931+0000 7fb962d9c700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb95c0c71b0 0x7fb95c15d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.931+0000 7fb962d9c700 1 -- 172.21.15.49:0/813471212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c0094a0 con 0x7fb95c15c860 2023-07-18T03:40:57.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.931+0000 7fb962d9c700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb95c15c860 0x7fb95c156470 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0cd930 tx=0x7fb954004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.932+0000 7fb952ffd700 1 -- 172.21.15.49:0/813471212 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954004840 con 0x7fb95c15c860 2023-07-18T03:40:57.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.932+0000 7fb952ffd700 1 -- 172.21.15.49:0/813471212 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9540099b0 con 0x7fb95c15c860 2023-07-18T03:40:57.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.932+0000 7fb952ffd700 1 -- 172.21.15.49:0/813471212 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95401c680 con 0x7fb95c15c860 2023-07-18T03:40:57.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.932+0000 7fb9647ff700 1 -- 172.21.15.49:0/813471212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb954009b30 con 0x7fb95c15c860 2023-07-18T03:40:57.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.932+0000 7fb9647ff700 1 -- 172.21.15.49:0/813471212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c0b8070 con 0x7fb95c15c860 2023-07-18T03:40:57.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.933+0000 7fb9647ff700 1 -- 172.21.15.49:0/813471212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95c074350 con 0x7fb95c15c860 2023-07-18T03:40:57.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.934+0000 7fb952ffd700 1 -- 172.21.15.49:0/813471212 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fb95401c7e0 con 0x7fb95c15c860 2023-07-18T03:40:57.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.935+0000 7fb952ffd700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb94006ae00 0x7fb94006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.935+0000 7fb952ffd700 1 -- 172.21.15.49:0/813471212 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb95408b370 con 0x7fb95c15c860 2023-07-18T03:40:57.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.935+0000 7fb96259b700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb94006ae00 0x7fb94006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.936+0000 7fb96259b700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb94006ae00 0x7fb94006d2b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb95c145dc0 tx=0x7fb958009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:57.940+0000 7fb952ffd700 1 -- 172.21.15.49:0/813471212 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb9540561f0 con 0x7fb95c15c860 2023-07-18T03:40:58.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.151+0000 7fb9647ff700 1 -- 172.21.15.49:0/813471212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb95c0cd040 con 0x7fb95c15c860 2023-07-18T03:40:58.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.152+0000 7fb952ffd700 1 -- 172.21.15.49:0/813471212 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb954059810 con 0x7fb95c15c860 2023-07-18T03:40:58.147 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:40:58.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.154+0000 7fb9647ff700 1 -- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb94006ae00 msgr2=0x7fb94006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:58.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.154+0000 7fb9647ff700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb94006ae00 0x7fb94006d2b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb95c145dc0 tx=0x7fb958009040 comp rx=0 tx=0).stop 2023-07-18T03:40:58.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.154+0000 7fb9647ff700 1 -- 172.21.15.49:0/813471212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb95c15c860 msgr2=0x7fb95c156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:58.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.154+0000 7fb9647ff700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb95c15c860 0x7fb95c156470 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0cd930 tx=0x7fb954004ad0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.154+0000 7fb9647ff700 1 -- 172.21.15.49:0/813471212 shutdown_connections 2023-07-18T03:40:58.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.154+0000 7fb9647ff700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb94006ae00 0x7fb94006d2b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.154+0000 7fb9647ff700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb95c15c860 0x7fb95c156470 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.154+0000 7fb9647ff700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb95c0c71b0 0x7fb95c15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.154+0000 7fb9647ff700 1 --2- 172.21.15.49:0/813471212 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb95c0d1e40 0x7fb95c0ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.154+0000 7fb9647ff700 1 -- 172.21.15.49:0/813471212 >> 172.21.15.49:0/813471212 conn(0x7fb95c065bd0 msgr2=0x7fb95c159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:58.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.154+0000 7fb9647ff700 1 -- 172.21.15.49:0/813471212 shutdown_connections 2023-07-18T03:40:58.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.154+0000 7fb9647ff700 1 -- 172.21.15.49:0/813471212 wait complete. 2023-07-18T03:40:58.168 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:40:58.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.381+0000 7fa9191ff700 1 Processor -- start 2023-07-18T03:40:58.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.381+0000 7fa9191ff700 1 -- start start 2023-07-18T03:40:58.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.382+0000 7fa9191ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa9140d0b10 0x7fa9140d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:58.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.382+0000 7fa9191ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9140b0e20 0x7fa9140b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:58.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.382+0000 7fa9191ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9140b8e30 con 0x7fa9140b8a60 2023-07-18T03:40:58.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.382+0000 7fa9191ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9140d14d0 con 0x7fa9140d0b10 2023-07-18T03:40:58.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.382+0000 7fa9191ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140afaa0 con 0x7fa9140b0e20 2023-07-18T03:40:58.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.382+0000 7fa91259c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa9140d0b10 0x7fa9140d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:58.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.382+0000 7fa91259c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa9140d0b10 0x7fa9140d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54172/0 (socket says 172.21.15.49:54172) 2023-07-18T03:40:58.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.383+0000 7fa91259c700 1 -- 172.21.15.49:0/4280497716 learned_addr learned my addr 172.21.15.49:0/4280497716 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:58.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.383+0000 7fa91359e700 1 --2- 172.21.15.49:0/4280497716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9140b0e20 0x7fa9140b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:58.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.383+0000 7fa91259c700 1 -- 172.21.15.49:0/4280497716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9140b0e20 msgr2=0x7fa9140b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:58.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.383+0000 7fa91259c700 1 --2- 172.21.15.49:0/4280497716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9140b0e20 0x7fa9140b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.383+0000 7fa91259c700 1 -- 172.21.15.49:0/4280497716 >> v1:172.21.15.107:6789/0 conn(0x7fa9140b8a60 legacy=0x7fa9140b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:58.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.383+0000 7fa91259c700 1 -- 172.21.15.49:0/4280497716 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140afd80 con 0x7fa9140d0b10 2023-07-18T03:40:58.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.383+0000 7fa91359e700 1 --2- 172.21.15.49:0/4280497716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9140b0e20 0x7fa9140b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:58.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.383+0000 7fa91259c700 1 --2- 172.21.15.49:0/4280497716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9140d0b10 0x7fa9140d0f90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa9080098f0 tx=0x7fa908009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64571461909b77da server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:58.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.383+0000 7fa911d9b700 1 -- 172.21.15.49:0/4280497716 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800b8a0 con 0x7fa9140d0b10 2023-07-18T03:40:58.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.384+0000 7fa911d9b700 1 -- 172.21.15.49:0/4280497716 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90800ba00 con 0x7fa9140d0b10 2023-07-18T03:40:58.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.384+0000 7fa911d9b700 1 -- 172.21.15.49:0/4280497716 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800bd90 con 0x7fa9140d0b10 2023-07-18T03:40:58.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.384+0000 7fa9191ff700 1 -- 172.21.15.49:0/4280497716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9140d0b10 msgr2=0x7fa9140d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:58.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.384+0000 7fa9191ff700 1 --2- 172.21.15.49:0/4280497716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9140d0b10 0x7fa9140d0f90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa9080098f0 tx=0x7fa908009c00 comp rx=0 tx=0).stop 2023-07-18T03:40:58.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.384+0000 7fa9191ff700 1 -- 172.21.15.49:0/4280497716 shutdown_connections 2023-07-18T03:40:58.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.384+0000 7fa9191ff700 1 --2- 172.21.15.49:0/4280497716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9140b0e20 0x7fa9140b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.384+0000 7fa9191ff700 1 --2- 172.21.15.49:0/4280497716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9140d0b10 0x7fa9140d0f90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.384+0000 7fa9191ff700 1 -- 172.21.15.49:0/4280497716 >> 172.21.15.49:0/4280497716 conn(0x7fa91409c1b0 msgr2=0x7fa9140d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:58.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.384+0000 7fa9191ff700 1 -- 172.21.15.49:0/4280497716 shutdown_connections 2023-07-18T03:40:58.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.384+0000 7fa9191ff700 1 -- 172.21.15.49:0/4280497716 wait complete. 2023-07-18T03:40:58.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.386+0000 7fa9191ff700 1 Processor -- start 2023-07-18T03:40:58.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.386+0000 7fa9191ff700 1 -- start start 2023-07-18T03:40:58.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.387+0000 7fa9191ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9140b0e20 0x7fa91415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:58.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.387+0000 7fa9191ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9140b8a60 0x7fa9140ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:58.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.387+0000 7fa9191ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9140d0b10 0x7fa9140ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:58.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.387+0000 7fa9191ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91415e940 con 0x7fa9140d0b10 2023-07-18T03:40:58.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.387+0000 7fa9191ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91415dcc0 con 0x7fa9140b8a60 2023-07-18T03:40:58.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.387+0000 7fa9191ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91415dfa0 con 0x7fa9140b0e20 2023-07-18T03:40:58.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.387+0000 7fa91359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9140d0b10 0x7fa9140ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:58.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.387+0000 7fa91359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9140d0b10 0x7fa9140ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54178/0 (socket says 172.21.15.49:54178) 2023-07-18T03:40:58.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.387+0000 7fa91359e700 1 -- 172.21.15.49:0/1687965039 learned_addr learned my addr 172.21.15.49:0/1687965039 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:58.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.387+0000 7fa91259c700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9140b8a60 0x7fa9140ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:58.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.387+0000 7fa912d9d700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9140b0e20 0x7fa91415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:58.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.388+0000 7fa91359e700 1 -- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9140b0e20 msgr2=0x7fa91415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:58.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.388+0000 7fa91359e700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9140b0e20 0x7fa91415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.388+0000 7fa91359e700 1 -- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9140b8a60 msgr2=0x7fa9140ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:58.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.388+0000 7fa91359e700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9140b8a60 0x7fa9140ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.388+0000 7fa91359e700 1 -- 172.21.15.49:0/1687965039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908009550 con 0x7fa9140d0b10 2023-07-18T03:40:58.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.388+0000 7fa91359e700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9140d0b10 0x7fa9140ca550 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009fa0 tx=0x7fa90c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:58.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.388+0000 7fa8fb7fe700 1 -- 172.21.15.49:0/1687965039 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00ddb0 con 0x7fa9140d0b10 2023-07-18T03:40:58.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.388+0000 7fa8fb7fe700 1 -- 172.21.15.49:0/1687965039 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c00f040 con 0x7fa9140d0b10 2023-07-18T03:40:58.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.388+0000 7fa8fb7fe700 1 -- 172.21.15.49:0/1687965039 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c003c60 con 0x7fa9140d0b10 2023-07-18T03:40:58.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.388+0000 7fa9191ff700 1 -- 172.21.15.49:0/1687965039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91415e280 con 0x7fa9140d0b10 2023-07-18T03:40:58.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.389+0000 7fa9191ff700 1 -- 172.21.15.49:0/1687965039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91415d750 con 0x7fa9140d0b10 2023-07-18T03:40:58.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.390+0000 7fa9191ff700 1 -- 172.21.15.49:0/1687965039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91415ea80 con 0x7fa9140d0b10 2023-07-18T03:40:58.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.391+0000 7fa8fb7fe700 1 -- 172.21.15.49:0/1687965039 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fa90c004030 con 0x7fa9140d0b10 2023-07-18T03:40:58.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.392+0000 7fa8fb7fe700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa90006ae00 0x7fa90006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:58.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.392+0000 7fa8fb7fe700 1 -- 172.21.15.49:0/1687965039 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa90c089730 con 0x7fa9140d0b10 2023-07-18T03:40:58.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.392+0000 7fa912d9d700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa90006ae00 0x7fa90006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:58.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.393+0000 7fa912d9d700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa90006ae00 0x7fa90006d2b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa904007fd0 tx=0x7fa904009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:58.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.396+0000 7fa8fb7fe700 1 -- 172.21.15.49:0/1687965039 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa91415ea80 con 0x7fa9140d0b10 2023-07-18T03:40:58.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.602+0000 7fa9191ff700 1 -- 172.21.15.49:0/1687965039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa91415ea80 con 0x7fa9140d0b10 2023-07-18T03:40:58.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.602+0000 7fa8fb7fe700 1 -- 172.21.15.49:0/1687965039 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa91415ea80 con 0x7fa9140d0b10 2023-07-18T03:40:58.598 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:40:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.605+0000 7fa9191ff700 1 -- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa90006ae00 msgr2=0x7fa90006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:58.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.605+0000 7fa9191ff700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa90006ae00 0x7fa90006d2b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa904007fd0 tx=0x7fa904009040 comp rx=0 tx=0).stop 2023-07-18T03:40:58.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.605+0000 7fa9191ff700 1 -- 172.21.15.49:0/1687965039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9140d0b10 msgr2=0x7fa9140ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:58.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.605+0000 7fa9191ff700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9140d0b10 0x7fa9140ca550 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009fa0 tx=0x7fa90c00d3b0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.605+0000 7fa9191ff700 1 -- 172.21.15.49:0/1687965039 shutdown_connections 2023-07-18T03:40:58.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.605+0000 7fa9191ff700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa90006ae00 0x7fa90006d2b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.605+0000 7fa9191ff700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9140d0b10 0x7fa9140ca550 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.605+0000 7fa9191ff700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9140b0e20 0x7fa91415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.605+0000 7fa9191ff700 1 --2- 172.21.15.49:0/1687965039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9140b8a60 0x7fa9140ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.605+0000 7fa9191ff700 1 -- 172.21.15.49:0/1687965039 >> 172.21.15.49:0/1687965039 conn(0x7fa91409c1b0 msgr2=0x7fa91406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.605+0000 7fa9191ff700 1 -- 172.21.15.49:0/1687965039 shutdown_connections 2023-07-18T03:40:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.605+0000 7fa9191ff700 1 -- 172.21.15.49:0/1687965039 wait complete. 2023-07-18T03:40:58.618 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:40:58.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.830+0000 7f576bfff700 1 Processor -- start 2023-07-18T03:40:58.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.830+0000 7f576bfff700 1 -- start start 2023-07-18T03:40:58.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.831+0000 7f576bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f576415c860 0x7f57640d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:58.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.831+0000 7f576bfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57640cf2a0 0x7f57640c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:58.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f576bfff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57640cf710 con 0x7f57640c6fe0 2023-07-18T03:40:58.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f576bfff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5764146740 con 0x7f576415c860 2023-07-18T03:40:58.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f576bfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57640cd5a0 con 0x7f57640cf2a0 2023-07-18T03:40:58.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f5769d9b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f576415c860 0x7f57640d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:58.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f576a59c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f57640c6fe0 0x7f57640cf960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33384/0 (socket says 172.21.15.49:33384) 2023-07-18T03:40:58.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f576a59c700 1 -- 172.21.15.49:0/3459241167 learned_addr learned my addr 172.21.15.49:0/3459241167 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:58.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f576959a700 1 --2- 172.21.15.49:0/3459241167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57640cf2a0 0x7f57640c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:58.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f5769d9b700 1 -- 172.21.15.49:0/3459241167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57640cf2a0 msgr2=0x7f57640c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:58.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f5768d99700 1 -- 172.21.15.49:0/3459241167 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537775811 0 0) 0x7f575c001490 con 0x7f57640c6fe0 2023-07-18T03:40:58.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f5769d9b700 1 --2- 172.21.15.49:0/3459241167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57640cf2a0 0x7f57640c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f5769d9b700 1 -- 172.21.15.49:0/3459241167 >> v1:172.21.15.49:6789/0 conn(0x7f57640c6fe0 legacy=0x7f57640cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:58.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f5769d9b700 1 -- 172.21.15.49:0/3459241167 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57640cd7f0 con 0x7f576415c860 2023-07-18T03:40:58.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.832+0000 7f576959a700 1 --2- 172.21.15.49:0/3459241167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57640cf2a0 0x7f57640c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:58.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.833+0000 7f5769d9b700 1 --2- 172.21.15.49:0/3459241167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576415c860 0x7f57640d1e40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5760009840 tx=0x7f5760009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0ae0e9823f3ca2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:58.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.833+0000 7f5768d99700 1 -- 172.21.15.49:0/3459241167 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760011830 con 0x7f576415c860 2023-07-18T03:40:58.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.833+0000 7f5768d99700 1 -- 172.21.15.49:0/3459241167 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5760011990 con 0x7f576415c860 2023-07-18T03:40:58.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.833+0000 7f5768d99700 1 -- 172.21.15.49:0/3459241167 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760011d20 con 0x7f576415c860 2023-07-18T03:40:58.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.833+0000 7f576bfff700 1 -- 172.21.15.49:0/3459241167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576415c860 msgr2=0x7f57640d1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:58.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.833+0000 7f576bfff700 1 --2- 172.21.15.49:0/3459241167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576415c860 0x7f57640d1e40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5760009840 tx=0x7f5760009b50 comp rx=0 tx=0).stop 2023-07-18T03:40:58.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.833+0000 7f576bfff700 1 -- 172.21.15.49:0/3459241167 shutdown_connections 2023-07-18T03:40:58.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.833+0000 7f576bfff700 1 --2- 172.21.15.49:0/3459241167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576415c860 0x7f57640d1e40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.833+0000 7f576bfff700 1 --2- 172.21.15.49:0/3459241167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57640cf2a0 0x7f57640c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.833+0000 7f576bfff700 1 -- 172.21.15.49:0/3459241167 >> 172.21.15.49:0/3459241167 conn(0x7f5764065bd0 msgr2=0x7f5764065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:58.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.834+0000 7f576bfff700 1 -- 172.21.15.49:0/3459241167 shutdown_connections 2023-07-18T03:40:58.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.834+0000 7f576bfff700 1 -- 172.21.15.49:0/3459241167 wait complete. 2023-07-18T03:40:58.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.835+0000 7f576bfff700 1 Processor -- start 2023-07-18T03:40:58.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.835+0000 7f576bfff700 1 -- start start 2023-07-18T03:40:58.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.836+0000 7f576bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57640c6fe0 0x7f57640b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:58.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.836+0000 7f576bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57640cf2a0 0x7f57640cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:58.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.836+0000 7f576bfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f576415c860 0x7f57640b8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:58.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.836+0000 7f576bfff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640cbbf0 con 0x7f57640cf2a0 2023-07-18T03:40:58.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.836+0000 7f576bfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640cbed0 con 0x7f57640c6fe0 2023-07-18T03:40:58.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.836+0000 7f576bfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57640cc1b0 con 0x7f576415c860 2023-07-18T03:40:58.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.837+0000 7f576959a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57640cf2a0 0x7f57640cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:58.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.837+0000 7f576959a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57640cf2a0 0x7f57640cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54202/0 (socket says 172.21.15.49:54202) 2023-07-18T03:40:58.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.837+0000 7f576959a700 1 -- 172.21.15.49:0/445125931 learned_addr learned my addr 172.21.15.49:0/445125931 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:58.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.837+0000 7f576959a700 1 -- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f576415c860 msgr2=0x7f57640b8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:58.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.837+0000 7f576959a700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f576415c860 0x7f57640b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.837+0000 7f576959a700 1 -- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57640c6fe0 msgr2=0x7f57640b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:58.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.837+0000 7f5769d9b700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57640c6fe0 0x7f57640b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:58.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.837+0000 7f576959a700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57640c6fe0 0x7f57640b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:58.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.837+0000 7f576959a700 1 -- 172.21.15.49:0/445125931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600094a0 con 0x7f57640cf2a0 2023-07-18T03:40:58.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.837+0000 7f5769d9b700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57640c6fe0 0x7f57640b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:58.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.837+0000 7f576959a700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57640cf2a0 0x7f57640cb1a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f575400c750 tx=0x7f575400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:58.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.838+0000 7f575a7fc700 1 -- 172.21.15.49:0/445125931 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5754011800 con 0x7f57640cf2a0 2023-07-18T03:40:58.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.838+0000 7f575a7fc700 1 -- 172.21.15.49:0/445125931 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5754011960 con 0x7f57640cf2a0 2023-07-18T03:40:58.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.838+0000 7f575a7fc700 1 -- 172.21.15.49:0/445125931 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5754011cd0 con 0x7f57640cf2a0 2023-07-18T03:40:58.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.838+0000 7f576bfff700 1 -- 172.21.15.49:0/445125931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57640cc490 con 0x7f57640cf2a0 2023-07-18T03:40:58.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.838+0000 7f576bfff700 1 -- 172.21.15.49:0/445125931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57640b7ec0 con 0x7f57640cf2a0 2023-07-18T03:40:58.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.839+0000 7f576bfff700 1 -- 172.21.15.49:0/445125931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57640cc9b0 con 0x7f57640cf2a0 2023-07-18T03:40:58.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.840+0000 7f575a7fc700 1 -- 172.21.15.49:0/445125931 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f5754021350 con 0x7f57640cf2a0 2023-07-18T03:40:58.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.841+0000 7f575a7fc700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f574806ae00 0x7f574806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:58.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.841+0000 7f575a7fc700 1 -- 172.21.15.49:0/445125931 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57540896f0 con 0x7f57640cf2a0 2023-07-18T03:40:58.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.841+0000 7f5769d9b700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f574806ae00 0x7f574806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:58.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.842+0000 7f5769d9b700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f574806ae00 0x7f574806d2b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5760009b90 tx=0x7f5760008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:58.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:58.845+0000 7f575a7fc700 1 -- 172.21.15.49:0/445125931 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f57640cc9b0 con 0x7f57640cf2a0 2023-07-18T03:40:59.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.053+0000 7f576bfff700 1 -- 172.21.15.49:0/445125931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57640cc9b0 con 0x7f57640cf2a0 2023-07-18T03:40:59.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.053+0000 7f575a7fc700 1 -- 172.21.15.49:0/445125931 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57640cc9b0 con 0x7f57640cf2a0 2023-07-18T03:40:59.049 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:40:59.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.055+0000 7f576bfff700 1 -- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f574806ae00 msgr2=0x7f574806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:59.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.055+0000 7f576bfff700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f574806ae00 0x7f574806d2b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5760009b90 tx=0x7f5760008030 comp rx=0 tx=0).stop 2023-07-18T03:40:59.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.055+0000 7f576bfff700 1 -- 172.21.15.49:0/445125931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57640cf2a0 msgr2=0x7f57640cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:59.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.055+0000 7f576bfff700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57640cf2a0 0x7f57640cb1a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f575400c750 tx=0x7f575400cb10 comp rx=0 tx=0).stop 2023-07-18T03:40:59.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.055+0000 7f576bfff700 1 -- 172.21.15.49:0/445125931 shutdown_connections 2023-07-18T03:40:59.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.055+0000 7f576bfff700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f576415c860 0x7f57640b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:59.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.055+0000 7f576bfff700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57640c6fe0 0x7f57640b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:59.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.055+0000 7f576bfff700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f574806ae00 0x7f574806d2b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:59.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.055+0000 7f576bfff700 1 --2- 172.21.15.49:0/445125931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57640cf2a0 0x7f57640cb1a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:59.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.055+0000 7f576bfff700 1 -- 172.21.15.49:0/445125931 >> 172.21.15.49:0/445125931 conn(0x7f5764065bd0 msgr2=0x7f57640c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:59.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.055+0000 7f576bfff700 1 -- 172.21.15.49:0/445125931 shutdown_connections 2023-07-18T03:40:59.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.055+0000 7f576bfff700 1 -- 172.21.15.49:0/445125931 wait complete. 2023-07-18T03:40:59.069 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-18T03:40:59.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.281+0000 7fc37f5ff700 1 Processor -- start 2023-07-18T03:40:59.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.281+0000 7fc37f5ff700 1 -- start start 2023-07-18T03:40:59.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.281+0000 7fc37f5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3780b4d00 0x7fc3780d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:59.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.281+0000 7fc37f5ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3780d02a0 con 0x7fc3780b4860 2023-07-18T03:40:59.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.281+0000 7fc37f5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3780cfce0 con 0x7fc3780b8660 2023-07-18T03:40:59.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.281+0000 7fc37f5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3780cffc0 con 0x7fc3780b4d00 2023-07-18T03:40:59.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.281+0000 7fc37cb9a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc3780b4860 0x7fc3780d12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50468/0 (socket says 172.21.15.49:50468) 2023-07-18T03:40:59.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.281+0000 7fc37cb9a700 1 -- 172.21.15.49:0/3920817432 learned_addr learned my addr 172.21.15.49:0/3920817432 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:40:59.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.282+0000 7fc37db9c700 1 --2- 172.21.15.49:0/3920817432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3780b4d00 0x7fc3780d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:59.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.282+0000 7fc36ffff700 1 -- 172.21.15.49:0/3920817432 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182501557 0 0) 0x7fc3780d02a0 con 0x7fc3780b4860 2023-07-18T03:40:59.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.282+0000 7fc36ffff700 1 -- 172.21.15.49:0/3920817432 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3640035b0 con 0x7fc3780b4860 2023-07-18T03:40:59.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.282+0000 7fc36ffff700 1 -- 172.21.15.49:0/3920817432 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367984131 0 0) 0x7fc3780cfce0 con 0x7fc3780b8660 2023-07-18T03:40:59.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.282+0000 7fc36ffff700 1 -- 172.21.15.49:0/3920817432 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3780d02a0 con 0x7fc3780b8660 2023-07-18T03:40:59.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.282+0000 7fc37db9c700 1 -- 172.21.15.49:0/3920817432 >> v1:172.21.15.107:6789/0 conn(0x7fc3780b8660 legacy=0x7fc3780cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:59.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.282+0000 7fc37db9c700 1 -- 172.21.15.49:0/3920817432 >> v1:172.21.15.49:6789/0 conn(0x7fc3780b4860 legacy=0x7fc3780d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:59.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.282+0000 7fc36ffff700 1 -- 172.21.15.49:0/3920817432 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 943679094 0 0) 0x7fc3640035b0 con 0x7fc3780b4860 2023-07-18T03:40:59.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.282+0000 7fc37db9c700 1 -- 172.21.15.49:0/3920817432 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3780d0ac0 con 0x7fc3780b4d00 2023-07-18T03:40:59.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.283+0000 7fc37db9c700 1 --2- 172.21.15.49:0/3920817432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3780b4d00 0x7fc3780d19d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc370009dc0 tx=0x7fc37000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de28bb500fa3cb24 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:59.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.283+0000 7fc36ffff700 1 -- 172.21.15.49:0/3920817432 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37000e070 con 0x7fc3780b4d00 2023-07-18T03:40:59.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.283+0000 7fc36ffff700 1 -- 172.21.15.49:0/3920817432 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc370003a20 con 0x7fc3780b4d00 2023-07-18T03:40:59.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.283+0000 7fc36ffff700 1 -- 172.21.15.49:0/3920817432 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc368001670 con 0x7fc3780b4d00 2023-07-18T03:40:59.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.283+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3920817432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3780b4d00 msgr2=0x7fc3780d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:59.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.283+0000 7fc37f5ff700 1 --2- 172.21.15.49:0/3920817432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3780b4d00 0x7fc3780d19d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc370009dc0 tx=0x7fc37000b330 comp rx=0 tx=0).stop 2023-07-18T03:40:59.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.283+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3920817432 shutdown_connections 2023-07-18T03:40:59.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.283+0000 7fc37f5ff700 1 --2- 172.21.15.49:0/3920817432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3780b4d00 0x7fc3780d19d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:59.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.283+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3920817432 >> 172.21.15.49:0/3920817432 conn(0x7fc378066860 msgr2=0x7fc37806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:59.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.284+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3920817432 shutdown_connections 2023-07-18T03:40:59.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.284+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3920817432 wait complete. 2023-07-18T03:40:59.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.285+0000 7fc37f5ff700 1 Processor -- start 2023-07-18T03:40:59.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.286+0000 7fc37f5ff700 1 -- start start 2023-07-18T03:40:59.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.286+0000 7fc37f5ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3780b4860 0x7fc3780c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:59.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.286+0000 7fc37f5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3780b4d00 0x7fc3780c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:59.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37f5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3780b8660 0x7fc37815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:59.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37f5ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37815b7e0 con 0x7fc3780b4860 2023-07-18T03:40:59.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37f5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3780c8800 con 0x7fc3780b4d00 2023-07-18T03:40:59.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37f5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3780c7220 con 0x7fc3780b8660 2023-07-18T03:40:59.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37d39b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3780b4860 0x7fc3780c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:59.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37d39b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3780b4860 0x7fc3780c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58122/0 (socket says 172.21.15.49:58122) 2023-07-18T03:40:59.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37d39b700 1 -- 172.21.15.49:0/3666714128 learned_addr learned my addr 172.21.15.49:0/3666714128 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:40:59.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37d39b700 1 -- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3780b8660 msgr2=0x7fc37815a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:59.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37cb9a700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3780b4d00 0x7fc3780c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:59.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37db9c700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3780b8660 0x7fc37815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:59.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37d39b700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3780b8660 0x7fc37815a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:59.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37d39b700 1 -- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3780b4d00 msgr2=0x7fc3780c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:59.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37d39b700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3780b4d00 0x7fc3780c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:59.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37d39b700 1 -- 172.21.15.49:0/3666714128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc370009a70 con 0x7fc3780b4860 2023-07-18T03:40:59.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37cb9a700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3780b4d00 0x7fc3780c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:59.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.287+0000 7fc37db9c700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3780b8660 0x7fc37815a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:59.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.288+0000 7fc37d39b700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3780b4860 0x7fc3780c6bb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc37400c500 tx=0x7fc37400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:59.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.288+0000 7fc36dffb700 1 -- 172.21.15.49:0/3666714128 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3740052d0 con 0x7fc3780b4860 2023-07-18T03:40:59.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.288+0000 7fc36dffb700 1 -- 172.21.15.49:0/3666714128 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc374005450 con 0x7fc3780b4860 2023-07-18T03:40:59.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.288+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3666714128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3780c7500 con 0x7fc3780b4860 2023-07-18T03:40:59.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.288+0000 7fc36dffb700 1 -- 172.21.15.49:0/3666714128 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3740038c0 con 0x7fc3780b4860 2023-07-18T03:40:59.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.288+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3666714128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3780c7ac0 con 0x7fc3780b4860 2023-07-18T03:40:59.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.289+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3666714128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc378074350 con 0x7fc3780b4860 2023-07-18T03:40:59.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.290+0000 7fc36dffb700 1 -- 172.21.15.49:0/3666714128 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fc374003a20 con 0x7fc3780b4860 2023-07-18T03:40:59.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.291+0000 7fc36dffb700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc35c06ae00 0x7fc35c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:59.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.291+0000 7fc36dffb700 1 -- 172.21.15.49:0/3666714128 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc374088b30 con 0x7fc3780b4860 2023-07-18T03:40:59.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.291+0000 7fc37cb9a700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc35c06ae00 0x7fc35c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:59.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.292+0000 7fc37cb9a700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc35c06ae00 0x7fc35c06d2b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc3780b2050 tx=0x7fc368009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:59.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.295+0000 7fc36dffb700 1 -- 172.21.15.49:0/3666714128 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc374053ac0 con 0x7fc3780b4860 2023-07-18T03:40:59.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.525+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3666714128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc3780b2150 con 0x7fc3780b4860 2023-07-18T03:40:59.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.526+0000 7fc36dffb700 1 -- 172.21.15.49:0/3666714128 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc3740570e0 con 0x7fc3780b4860 2023-07-18T03:40:59.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.528+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc35c06ae00 msgr2=0x7fc35c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:59.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.528+0000 7fc37f5ff700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc35c06ae00 0x7fc35c06d2b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc3780b2050 tx=0x7fc368009040 comp rx=0 tx=0).stop 2023-07-18T03:40:59.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.528+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3666714128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3780b4860 msgr2=0x7fc3780c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:59.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.528+0000 7fc37f5ff700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3780b4860 0x7fc3780c6bb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc37400c500 tx=0x7fc37400c8c0 comp rx=0 tx=0).stop 2023-07-18T03:40:59.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.528+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3666714128 shutdown_connections 2023-07-18T03:40:59.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.528+0000 7fc37f5ff700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3780b8660 0x7fc37815a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:59.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.528+0000 7fc37f5ff700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc35c06ae00 0x7fc35c06d2b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:59.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.528+0000 7fc37f5ff700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3780b4860 0x7fc3780c6bb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:59.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.528+0000 7fc37f5ff700 1 --2- 172.21.15.49:0/3666714128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3780b4d00 0x7fc3780c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:59.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.528+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3666714128 >> 172.21.15.49:0/3666714128 conn(0x7fc378066860 msgr2=0x7fc3780d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:59.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.528+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3666714128 shutdown_connections 2023-07-18T03:40:59.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:40:59.528+0000 7fc37f5ff700 1 -- 172.21.15.49:0/3666714128 wait complete. 2023-07-18T03:40:59.524 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-07-18T03:40:59.541 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-18T03:40:59.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:59.542 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-18T03:40:59.637 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:40:59.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:59.638 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-18T03:40:59.684 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:40:59.686 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:59.736 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:40:59.737 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:59.822 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:41:00.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.033+0000 7f05d97ff700 1 Processor -- start 2023-07-18T03:41:00.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.033+0000 7f05d97ff700 1 -- start start 2023-07-18T03:41:00.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.034+0000 7f05d97ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f05d4160030 0x7f05d406c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.034+0000 7f05d97ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05d40d1160 0x7f05d40b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.034+0000 7f05d97ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05d40d2cc0 con 0x7f05d40b5cd0 2023-07-18T03:41:00.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.034+0000 7f05d97ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d40afbb0 con 0x7f05d4160030 2023-07-18T03:41:00.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.034+0000 7f05d97ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d40afe90 con 0x7f05d40d1160 2023-07-18T03:41:00.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.035+0000 7f05d2ffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f05d4160030 0x7f05d406c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.035+0000 7f05d27fc700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f05d40b5cd0 0x7f05d40d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50484/0 (socket says 172.21.15.49:50484) 2023-07-18T03:41:00.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.035+0000 7f05d27fc700 1 -- 172.21.15.49:0/1447854845 learned_addr learned my addr 172.21.15.49:0/1447854845 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:00.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.035+0000 7f05d37fe700 1 --2- 172.21.15.49:0/1447854845 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05d40d1160 0x7f05d40b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.035+0000 7f05d2ffd700 1 -- 172.21.15.49:0/1447854845 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05d40d1160 msgr2=0x7f05d40b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:00.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.035+0000 7f05d2ffd700 1 --2- 172.21.15.49:0/1447854845 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05d40d1160 0x7f05d40b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.035+0000 7f05d2ffd700 1 -- 172.21.15.49:0/1447854845 >> v1:172.21.15.49:6789/0 conn(0x7f05d40b5cd0 legacy=0x7f05d40d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.035+0000 7f05d2ffd700 1 -- 172.21.15.49:0/1447854845 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d40b0170 con 0x7f05d4160030 2023-07-18T03:41:00.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.035+0000 7f05d1ffb700 1 -- 172.21.15.49:0/1447854845 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489233926 0 0) 0x7f05d40d2cc0 con 0x7f05d40b5cd0 2023-07-18T03:41:00.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.035+0000 7f05d2ffd700 1 --2- 172.21.15.49:0/1447854845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05d4160030 0x7f05d406c0f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009840 tx=0x7f05c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f06909e941f530 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:00.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.036+0000 7f05d1ffb700 1 -- 172.21.15.49:0/1447854845 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c4011830 con 0x7f05d4160030 2023-07-18T03:41:00.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.036+0000 7f05d1ffb700 1 -- 172.21.15.49:0/1447854845 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c4011990 con 0x7f05d4160030 2023-07-18T03:41:00.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.036+0000 7f05d1ffb700 1 -- 172.21.15.49:0/1447854845 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c4011d20 con 0x7f05d4160030 2023-07-18T03:41:00.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.036+0000 7f05d97ff700 1 -- 172.21.15.49:0/1447854845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05d4160030 msgr2=0x7f05d406c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.036+0000 7f05d97ff700 1 --2- 172.21.15.49:0/1447854845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05d4160030 0x7f05d406c0f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009840 tx=0x7f05c4009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.036+0000 7f05d97ff700 1 -- 172.21.15.49:0/1447854845 shutdown_connections 2023-07-18T03:41:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.036+0000 7f05d97ff700 1 --2- 172.21.15.49:0/1447854845 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05d40d1160 0x7f05d40b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.036+0000 7f05d97ff700 1 --2- 172.21.15.49:0/1447854845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05d4160030 0x7f05d406c0f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.036+0000 7f05d97ff700 1 -- 172.21.15.49:0/1447854845 >> 172.21.15.49:0/1447854845 conn(0x7f05d409c190 msgr2=0x7f05d40d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.037+0000 7f05d97ff700 1 -- 172.21.15.49:0/1447854845 shutdown_connections 2023-07-18T03:41:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.037+0000 7f05d97ff700 1 -- 172.21.15.49:0/1447854845 wait complete. 2023-07-18T03:41:00.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.039+0000 7f05d97ff700 1 Processor -- start 2023-07-18T03:41:00.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.039+0000 7f05d97ff700 1 -- start start 2023-07-18T03:41:00.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.040+0000 7f05d97ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d40b5cd0 0x7f05d40c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.040+0000 7f05d97ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05d40d1160 0x7f05d4155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d97ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05d4160030 0x7f05d414bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d97ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d4156770 con 0x7f05d40d1160 2023-07-18T03:41:00.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d97ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d4156a50 con 0x7f05d40b5cd0 2023-07-18T03:41:00.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d97ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d4156d30 con 0x7f05d4160030 2023-07-18T03:41:00.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05d40d1160 0x7f05d4155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05d40d1160 0x7f05d4155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58128/0 (socket says 172.21.15.49:58128) 2023-07-18T03:41:00.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d27fc700 1 -- 172.21.15.49:0/818296345 learned_addr learned my addr 172.21.15.49:0/818296345 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:00.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d2ffd700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d40b5cd0 0x7f05d40c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d37fe700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05d4160030 0x7f05d414bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d27fc700 1 -- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05d4160030 msgr2=0x7f05d414bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d27fc700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05d4160030 0x7f05d414bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d27fc700 1 -- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d40b5cd0 msgr2=0x7f05d40c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d27fc700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d40b5cd0 0x7f05d40c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d27fc700 1 -- 172.21.15.49:0/818296345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c40094a0 con 0x7f05d40d1160 2023-07-18T03:41:00.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d37fe700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05d4160030 0x7f05d414bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:00.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d2ffd700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d40b5cd0 0x7f05d40c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:00.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.041+0000 7f05d27fc700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05d40d1160 0x7f05d4155cf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f05c80011e0 tx=0x7f05c800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:00.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.042+0000 7f05bb7fe700 1 -- 172.21.15.49:0/818296345 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8004c80 con 0x7f05d40d1160 2023-07-18T03:41:00.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.042+0000 7f05bb7fe700 1 -- 172.21.15.49:0/818296345 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c8004e00 con 0x7f05d40d1160 2023-07-18T03:41:00.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.042+0000 7f05bb7fe700 1 -- 172.21.15.49:0/818296345 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8007310 con 0x7f05d40d1160 2023-07-18T03:41:00.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.042+0000 7f05d97ff700 1 -- 172.21.15.49:0/818296345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d4157010 con 0x7f05d40d1160 2023-07-18T03:41:00.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.042+0000 7f05d97ff700 1 -- 172.21.15.49:0/818296345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d40b5300 con 0x7f05d40d1160 2023-07-18T03:41:00.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.043+0000 7f05d97ff700 1 -- 172.21.15.49:0/818296345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d4157530 con 0x7f05d40d1160 2023-07-18T03:41:00.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.044+0000 7f05bb7fe700 1 -- 172.21.15.49:0/818296345 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f05c8007470 con 0x7f05d40d1160 2023-07-18T03:41:00.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.045+0000 7f05bb7fe700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f05c006ae00 0x7f05c006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.045+0000 7f05bb7fe700 1 -- 172.21.15.49:0/818296345 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f05c8089310 con 0x7f05d40d1160 2023-07-18T03:41:00.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.045+0000 7f05d2ffd700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f05c006ae00 0x7f05c006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.046+0000 7f05d2ffd700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f05c006ae00 0x7f05c006d2b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f05c40090b0 tx=0x7f05c401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:00.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.049+0000 7f05bb7fe700 1 -- 172.21.15.49:0/818296345 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f05d4157530 con 0x7f05d40d1160 2023-07-18T03:41:00.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.265+0000 7f05d97ff700 1 -- 172.21.15.49:0/818296345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05d4157530 con 0x7f05d40d1160 2023-07-18T03:41:00.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.266+0000 7f05bb7fe700 1 -- 172.21.15.49:0/818296345 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18257 (secure 0 0 0) 0x7f05d4157530 con 0x7f05d40d1160 2023-07-18T03:41:00.261 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:00.261 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":49,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:40:41.947761+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:40:38.792808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-18T03:40:39.438590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:41:00.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.269+0000 7f05d97ff700 1 -- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f05c006ae00 msgr2=0x7f05c006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.269+0000 7f05d97ff700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f05c006ae00 0x7f05c006d2b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f05c40090b0 tx=0x7f05c401c040 comp rx=0 tx=0).stop 2023-07-18T03:41:00.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.269+0000 7f05d97ff700 1 -- 172.21.15.49:0/818296345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05d40d1160 msgr2=0x7f05d4155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.269+0000 7f05d97ff700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05d40d1160 0x7f05d4155cf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f05c80011e0 tx=0x7f05c800cae0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.269+0000 7f05d97ff700 1 -- 172.21.15.49:0/818296345 shutdown_connections 2023-07-18T03:41:00.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.269+0000 7f05d97ff700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05d4160030 0x7f05d414bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.269+0000 7f05d97ff700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d40b5cd0 0x7f05d40c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.269+0000 7f05d97ff700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f05c006ae00 0x7f05c006d2b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.269+0000 7f05d97ff700 1 --2- 172.21.15.49:0/818296345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f05d40d1160 0x7f05d4155cf0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.269+0000 7f05d97ff700 1 -- 172.21.15.49:0/818296345 >> 172.21.15.49:0/818296345 conn(0x7f05d409c190 msgr2=0x7f05d40b60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:00.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.269+0000 7f05d97ff700 1 -- 172.21.15.49:0/818296345 shutdown_connections 2023-07-18T03:41:00.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.269+0000 7f05d97ff700 1 -- 172.21.15.49:0/818296345 wait complete. 2023-07-18T03:41:00.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:40:38.792808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:40:39.438590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:41:00.283 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:41:00.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.514+0000 7f36adfff700 1 Processor -- start 2023-07-18T03:41:00.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.514+0000 7f36adfff700 1 -- start start 2023-07-18T03:41:00.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.515+0000 7f36adfff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f36a80d0f90 0x7f36a80b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.515+0000 7f36adfff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a80b8e30 con 0x7f36a80b8a60 2023-07-18T03:41:00.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.515+0000 7f36adfff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a80afaa0 con 0x7f36a80d0b10 2023-07-18T03:41:00.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.515+0000 7f36adfff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a80afd80 con 0x7f36a80d0f90 2023-07-18T03:41:00.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.516+0000 7f36a77fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f36a80b8a60 0x7f36a80b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50494/0 (socket says 172.21.15.49:50494) 2023-07-18T03:41:00.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.516+0000 7f36a77fe700 1 -- 172.21.15.49:0/189909257 learned_addr learned my addr 172.21.15.49:0/189909257 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:00.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.516+0000 7f36a7fff700 1 --2- 172.21.15.49:0/189909257 >> v2:172.21.15.107:3300/0 conn(0x7f36a80d0f90 0x7f36a80b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.516+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845678868 0 0) 0x7f36a80b8e30 con 0x7f36a80b8a60 2023-07-18T03:41:00.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.516+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36900035b0 con 0x7f36a80b8a60 2023-07-18T03:41:00.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.516+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 449686866 0 0) 0x7f36900035b0 con 0x7f36a80b8a60 2023-07-18T03:41:00.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.516+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 >> v2:172.21.15.107:3300/0 conn(0x7f36a80d0f90 msgr2=0x7f36a80b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:00.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.516+0000 7f36a67fc700 1 --2- 172.21.15.49:0/189909257 >> v2:172.21.15.107:3300/0 conn(0x7f36a80d0f90 0x7f36a80b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.516+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 >> v1:172.21.15.107:6789/0 conn(0x7f36a80d0b10 legacy=0x7f36a80b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.516+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a80b0060 con 0x7f36a80b8a60 2023-07-18T03:41:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.516+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f3698001670 con 0x7f36a80b8a60 2023-07-18T03:41:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.516+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 >> v1:172.21.15.49:6789/0 conn(0x7f36a80b8a60 legacy=0x7f36a80b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a67fc700 1 --2- 172.21.15.49:0/189909257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3690003a00 0x7f3690003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a67fc700 1 --2- 172.21.15.49:0/189909257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3690004850 0x7f3690004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a67fc700 1 --2- 172.21.15.49:0/189909257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 0x7f369000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80afd80 con 0x7f3690005200 2023-07-18T03:41:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369000c060 con 0x7f3690004850 2023-07-18T03:41:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369000c220 con 0x7f3690003a00 2023-07-18T03:41:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a7fff700 1 --2- 172.21.15.49:0/189909257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 0x7f369000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a77fe700 1 --2- 172.21.15.49:0/189909257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3690003a00 0x7f3690003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a6ffd700 1 --2- 172.21.15.49:0/189909257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3690004850 0x7f3690004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a7fff700 1 -- 172.21.15.49:0/189909257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3690003a00 msgr2=0x7f3690003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a7fff700 1 --2- 172.21.15.49:0/189909257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3690003a00 0x7f3690003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a7fff700 1 -- 172.21.15.49:0/189909257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3690004850 msgr2=0x7f3690004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a7fff700 1 --2- 172.21.15.49:0/189909257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3690004850 0x7f3690004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a7fff700 1 -- 172.21.15.49:0/189909257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36900035b0 con 0x7f3690005200 2023-07-18T03:41:00.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.517+0000 7f36a7fff700 1 -- 172.21.15.49:0/189909257 reap_dead start 2023-07-18T03:41:00.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.518+0000 7f36a7fff700 1 --2- 172.21.15.49:0/189909257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 0x7f369000ba10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009fc0 tx=0x7f3690005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aaedbb1b0fb085 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:00.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.518+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0003a20 con 0x7f3690005200 2023-07-18T03:41:00.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.518+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a000f040 con 0x7f3690005200 2023-07-18T03:41:00.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.518+0000 7f36a67fc700 1 -- 172.21.15.49:0/189909257 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0003da0 con 0x7f3690005200 2023-07-18T03:41:00.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.518+0000 7f36adfff700 1 -- 172.21.15.49:0/189909257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 msgr2=0x7f369000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.518+0000 7f36adfff700 1 --2- 172.21.15.49:0/189909257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 0x7f369000ba10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009fc0 tx=0x7f3690005880 comp rx=0 tx=0).stop 2023-07-18T03:41:00.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.518+0000 7f36adfff700 1 -- 172.21.15.49:0/189909257 shutdown_connections 2023-07-18T03:41:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.518+0000 7f36adfff700 1 --2- 172.21.15.49:0/189909257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 0x7f369000ba10 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.518+0000 7f36adfff700 1 -- 172.21.15.49:0/189909257 >> 172.21.15.49:0/189909257 conn(0x7f36a809c1b0 msgr2=0x7f36a80d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.518+0000 7f36adfff700 1 -- 172.21.15.49:0/189909257 shutdown_connections 2023-07-18T03:41:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.519+0000 7f36adfff700 1 -- 172.21.15.49:0/189909257 wait complete. 2023-07-18T03:41:00.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.520+0000 7f36adfff700 1 Processor -- start 2023-07-18T03:41:00.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.521+0000 7f36adfff700 1 -- start start 2023-07-18T03:41:00.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.521+0000 7f36adfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 0x7f36a80cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36adfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a814c370 0x7f36a80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36adfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a814c7e0 0x7f36a80ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36adfff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80ce590 con 0x7f3690005200 2023-07-18T03:41:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36adfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a815ae00 con 0x7f36a814c370 2023-07-18T03:41:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36adfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80cd910 con 0x7f36a814c7e0 2023-07-18T03:41:00.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36a77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 0x7f36a80cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36a77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 0x7f36a80cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58146/0 (socket says 172.21.15.49:58146) 2023-07-18T03:41:00.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36a77fe700 1 -- 172.21.15.49:0/1122825720 learned_addr learned my addr 172.21.15.49:0/1122825720 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:00.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36a7fff700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a814c7e0 0x7f36a80ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36a6ffd700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a814c370 0x7f36a80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36a77fe700 1 -- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a814c7e0 msgr2=0x7f36a80ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36a77fe700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a814c7e0 0x7f36a80ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36a77fe700 1 -- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a814c370 msgr2=0x7f36a80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36a77fe700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a814c370 0x7f36a80ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.522+0000 7f36a77fe700 1 -- 172.21.15.49:0/1122825720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a80cdbf0 con 0x7f3690005200 2023-07-18T03:41:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.523+0000 7f36a77fe700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 0x7f36a80cffe0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f36980072c0 tx=0x7f36980075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.523+0000 7f368ffff700 1 -- 172.21.15.49:0/1122825720 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36980077b0 con 0x7f3690005200 2023-07-18T03:41:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.523+0000 7f368ffff700 1 -- 172.21.15.49:0/1122825720 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369800f8d0 con 0x7f3690005200 2023-07-18T03:41:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.523+0000 7f36adfff700 1 -- 172.21.15.49:0/1122825720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a80cded0 con 0x7f3690005200 2023-07-18T03:41:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.523+0000 7f368ffff700 1 -- 172.21.15.49:0/1122825720 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369800fc80 con 0x7f3690005200 2023-07-18T03:41:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.523+0000 7f36adfff700 1 -- 172.21.15.49:0/1122825720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a80d0520 con 0x7f3690005200 2023-07-18T03:41:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.524+0000 7f36adfff700 1 -- 172.21.15.49:0/1122825720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a80c9de0 con 0x7f3690005200 2023-07-18T03:41:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.525+0000 7f368ffff700 1 -- 172.21.15.49:0/1122825720 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f369801d360 con 0x7f3690005200 2023-07-18T03:41:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.526+0000 7f368ffff700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f369406ae00 0x7f369406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.526+0000 7f368ffff700 1 -- 172.21.15.49:0/1122825720 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3698088d70 con 0x7f3690005200 2023-07-18T03:41:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.526+0000 7f36a6ffd700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f369406ae00 0x7f369406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.527+0000 7f36a6ffd700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f369406ae00 0x7f369406d2b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f36a815ffb0 tx=0x7f369c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.531+0000 7f368ffff700 1 -- 172.21.15.49:0/1122825720 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3698053da0 con 0x7f3690005200 2023-07-18T03:41:00.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.739+0000 7f36adfff700 1 -- 172.21.15.49:0/1122825720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f36a8074350 con 0x7f3690005200 2023-07-18T03:41:00.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.740+0000 7f368ffff700 1 -- 172.21.15.49:0/1122825720 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f369801a020 con 0x7f3690005200 2023-07-18T03:41:00.734 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:41:00.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.742+0000 7f36adfff700 1 -- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f369406ae00 msgr2=0x7f369406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.742+0000 7f36adfff700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f369406ae00 0x7f369406d2b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f36a815ffb0 tx=0x7f369c009040 comp rx=0 tx=0).stop 2023-07-18T03:41:00.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.742+0000 7f36adfff700 1 -- 172.21.15.49:0/1122825720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 msgr2=0x7f36a80cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.742+0000 7f36adfff700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 0x7f36a80cffe0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f36980072c0 tx=0x7f36980075d0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.742+0000 7f36adfff700 1 -- 172.21.15.49:0/1122825720 shutdown_connections 2023-07-18T03:41:00.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.742+0000 7f36adfff700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a814c7e0 0x7f36a80ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.742+0000 7f36adfff700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f369406ae00 0x7f369406d2b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.742+0000 7f36adfff700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3690005200 0x7f36a80cffe0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.742+0000 7f36adfff700 1 --2- 172.21.15.49:0/1122825720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a814c370 0x7f36a80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.742+0000 7f36adfff700 1 -- 172.21.15.49:0/1122825720 >> 172.21.15.49:0/1122825720 conn(0x7f36a809c1b0 msgr2=0x7f36a80d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:00.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.742+0000 7f36adfff700 1 -- 172.21.15.49:0/1122825720 shutdown_connections 2023-07-18T03:41:00.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.742+0000 7f36adfff700 1 -- 172.21.15.49:0/1122825720 wait complete. 2023-07-18T03:41:00.755 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:41:00.774 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:41:00.815 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:41:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.979+0000 7fe9701ff700 1 Processor -- start 2023-07-18T03:41:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.979+0000 7fe9701ff700 1 -- start start 2023-07-18T03:41:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.980+0000 7fe9701ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe96815c860 0x7fe9680d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.980+0000 7fe9701ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9680c71b0 0x7fe9680b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.980+0000 7fe9701ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9680cd5a0 con 0x7fe9680cf2a0 2023-07-18T03:41:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.980+0000 7fe9701ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe968146740 con 0x7fe9680c71b0 2023-07-18T03:41:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.980+0000 7fe9701ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9680cf740 con 0x7fe96815c860 2023-07-18T03:41:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.980+0000 7fe96e79c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9680c71b0 0x7fe9680b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.980+0000 7fe96d79a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe9680cf2a0 0x7fe9680c6aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50508/0 (socket says 172.21.15.49:50508) 2023-07-18T03:41:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.980+0000 7fe96d79a700 1 -- 172.21.15.49:0/3158385688 learned_addr learned my addr 172.21.15.49:0/3158385688 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.980+0000 7fe96df9b700 1 --2- 172.21.15.49:0/3158385688 >> v2:172.21.15.107:3300/0 conn(0x7fe96815c860 0x7fe9680d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.981+0000 7fe96e79c700 1 -- 172.21.15.49:0/3158385688 >> v2:172.21.15.107:3300/0 conn(0x7fe96815c860 msgr2=0x7fe9680d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.981+0000 7fe96e79c700 1 --2- 172.21.15.49:0/3158385688 >> v2:172.21.15.107:3300/0 conn(0x7fe96815c860 0x7fe9680d1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.981+0000 7fe96e79c700 1 -- 172.21.15.49:0/3158385688 >> v1:172.21.15.49:6789/0 conn(0x7fe9680cf2a0 legacy=0x7fe9680c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.981+0000 7fe96e79c700 1 -- 172.21.15.49:0/3158385688 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9680cd7f0 con 0x7fe9680c71b0 2023-07-18T03:41:00.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.981+0000 7fe96cf99700 1 -- 172.21.15.49:0/3158385688 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546231951 0 0) 0x7fe9680cd5a0 con 0x7fe9680cf2a0 2023-07-18T03:41:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.981+0000 7fe96e79c700 1 --2- 172.21.15.49:0/3158385688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9680c71b0 0x7fe9680b49d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe96808c100 tx=0x7fe96000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b994d154cc461ce server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.981+0000 7fe96cf99700 1 -- 172.21.15.49:0/3158385688 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96000e070 con 0x7fe9680c71b0 2023-07-18T03:41:00.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.981+0000 7fe96cf99700 1 -- 172.21.15.49:0/3158385688 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96000bb20 con 0x7fe9680c71b0 2023-07-18T03:41:00.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.981+0000 7fe96cf99700 1 -- 172.21.15.49:0/3158385688 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960012430 con 0x7fe9680c71b0 2023-07-18T03:41:00.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.982+0000 7fe9701ff700 1 -- 172.21.15.49:0/3158385688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9680c71b0 msgr2=0x7fe9680b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.982+0000 7fe9701ff700 1 --2- 172.21.15.49:0/3158385688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9680c71b0 0x7fe9680b49d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe96808c100 tx=0x7fe96000b330 comp rx=0 tx=0).stop 2023-07-18T03:41:00.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.982+0000 7fe9701ff700 1 -- 172.21.15.49:0/3158385688 shutdown_connections 2023-07-18T03:41:00.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.982+0000 7fe9701ff700 1 --2- 172.21.15.49:0/3158385688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9680c71b0 0x7fe9680b49d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.982+0000 7fe9701ff700 1 --2- 172.21.15.49:0/3158385688 >> v2:172.21.15.107:3300/0 conn(0x7fe96815c860 0x7fe9680d1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.982+0000 7fe9701ff700 1 -- 172.21.15.49:0/3158385688 >> 172.21.15.49:0/3158385688 conn(0x7fe968065bd0 msgr2=0x7fe968065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:00.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.982+0000 7fe9701ff700 1 -- 172.21.15.49:0/3158385688 shutdown_connections 2023-07-18T03:41:00.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.982+0000 7fe9701ff700 1 -- 172.21.15.49:0/3158385688 wait complete. 2023-07-18T03:41:00.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.984+0000 7fe9701ff700 1 Processor -- start 2023-07-18T03:41:00.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.984+0000 7fe9701ff700 1 -- start start 2023-07-18T03:41:00.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.985+0000 7fe9701ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9680c71b0 0x7fe968150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.985+0000 7fe9701ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9680cf2a0 0x7fe968158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.985+0000 7fe9701ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe96815c860 0x7fe968159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.985+0000 7fe9701ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9681f44c0 con 0x7fe9680cf2a0 2023-07-18T03:41:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.985+0000 7fe9701ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680cc210 con 0x7fe96815c860 2023-07-18T03:41:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.985+0000 7fe9701ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680cc4f0 con 0x7fe9680c71b0 2023-07-18T03:41:00.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.985+0000 7fe96d79a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9680cf2a0 0x7fe968158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.985+0000 7fe96d79a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9680cf2a0 0x7fe968158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58170/0 (socket says 172.21.15.49:58170) 2023-07-18T03:41:00.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.985+0000 7fe96d79a700 1 -- 172.21.15.49:0/280473660 learned_addr learned my addr 172.21.15.49:0/280473660 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:00.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.985+0000 7fe96df9b700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9680c71b0 0x7fe968150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.985+0000 7fe96e79c700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe96815c860 0x7fe968159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.986+0000 7fe96d79a700 1 -- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9680c71b0 msgr2=0x7fe968150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.986+0000 7fe96d79a700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9680c71b0 0x7fe968150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.986+0000 7fe96d79a700 1 -- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe96815c860 msgr2=0x7fe968159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.986+0000 7fe96d79a700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe96815c860 0x7fe968159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.986+0000 7fe96d79a700 1 -- 172.21.15.49:0/280473660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe960009a70 con 0x7fe9680cf2a0 2023-07-18T03:41:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.986+0000 7fe96d79a700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9680cf2a0 0x7fe968158e10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe9580011e0 tx=0x7fe95800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.986+0000 7fe95e7fc700 1 -- 172.21.15.49:0/280473660 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958004c80 con 0x7fe9680cf2a0 2023-07-18T03:41:00.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.986+0000 7fe95e7fc700 1 -- 172.21.15.49:0/280473660 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe958004e00 con 0x7fe9680cf2a0 2023-07-18T03:41:00.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.986+0000 7fe9701ff700 1 -- 172.21.15.49:0/280473660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9680cc7d0 con 0x7fe9680cf2a0 2023-07-18T03:41:00.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.986+0000 7fe95e7fc700 1 -- 172.21.15.49:0/280473660 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958007310 con 0x7fe9680cf2a0 2023-07-18T03:41:00.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.986+0000 7fe9701ff700 1 -- 172.21.15.49:0/280473660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9680ccd90 con 0x7fe9680cf2a0 2023-07-18T03:41:00.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.987+0000 7fe9701ff700 1 -- 172.21.15.49:0/280473660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe968074350 con 0x7fe9680cf2a0 2023-07-18T03:41:00.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.988+0000 7fe95e7fc700 1 -- 172.21.15.49:0/280473660 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fe958007470 con 0x7fe9680cf2a0 2023-07-18T03:41:00.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.989+0000 7fe95e7fc700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe94c06ae00 0x7fe94c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:00.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.989+0000 7fe95e7fc700 1 -- 172.21.15.49:0/280473660 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fe958089380 con 0x7fe9680cf2a0 2023-07-18T03:41:00.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.990+0000 7fe96df9b700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe94c06ae00 0x7fe94c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:00.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.991+0000 7fe96df9b700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe94c06ae00 0x7fe94c06d2b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe964009b40 tx=0x7fe964009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:00.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:00.994+0000 7fe95e7fc700 1 -- 172.21.15.49:0/280473660 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe958054240 con 0x7fe9680cf2a0 2023-07-18T03:41:01.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.205+0000 7fe9701ff700 1 -- 172.21.15.49:0/280473660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe96815bec0 con 0x7fe9680cf2a0 2023-07-18T03:41:01.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.206+0000 7fe95e7fc700 1 -- 172.21.15.49:0/280473660 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe958057860 con 0x7fe9680cf2a0 2023-07-18T03:41:01.200 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:41:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.208+0000 7fe9701ff700 1 -- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe94c06ae00 msgr2=0x7fe94c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.208+0000 7fe9701ff700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe94c06ae00 0x7fe94c06d2b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe964009b40 tx=0x7fe964009210 comp rx=0 tx=0).stop 2023-07-18T03:41:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.208+0000 7fe9701ff700 1 -- 172.21.15.49:0/280473660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9680cf2a0 msgr2=0x7fe968158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:01.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.208+0000 7fe9701ff700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9680cf2a0 0x7fe968158e10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe9580011e0 tx=0x7fe95800ca30 comp rx=0 tx=0).stop 2023-07-18T03:41:01.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.208+0000 7fe9701ff700 1 -- 172.21.15.49:0/280473660 shutdown_connections 2023-07-18T03:41:01.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.208+0000 7fe9701ff700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe96815c860 0x7fe968159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.208+0000 7fe9701ff700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9680c71b0 0x7fe968150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.208+0000 7fe9701ff700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe94c06ae00 0x7fe94c06d2b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.208+0000 7fe9701ff700 1 --2- 172.21.15.49:0/280473660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9680cf2a0 0x7fe968158e10 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.208+0000 7fe9701ff700 1 -- 172.21.15.49:0/280473660 >> 172.21.15.49:0/280473660 conn(0x7fe968065bd0 msgr2=0x7fe96815adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.208+0000 7fe9701ff700 1 -- 172.21.15.49:0/280473660 shutdown_connections 2023-07-18T03:41:01.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.208+0000 7fe9701ff700 1 -- 172.21.15.49:0/280473660 wait complete. 2023-07-18T03:41:01.221 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:41:01.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.438+0000 7fdadf7ff700 1 Processor -- start 2023-07-18T03:41:01.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.438+0000 7fdadf7ff700 1 -- start start 2023-07-18T03:41:01.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdadf7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdad815c860 0x7fdad80d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:01.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdadf7ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdad80c71b0 0x7fdad80b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:01.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdadf7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad80cf710 con 0x7fdad80cf2a0 2023-07-18T03:41:01.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdadf7ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad8146740 con 0x7fdad80c71b0 2023-07-18T03:41:01.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdadf7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad80cd5a0 con 0x7fdad815c860 2023-07-18T03:41:01.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdaddd9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdad80c71b0 0x7fdad80b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:01.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdaddd9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdad80c71b0 0x7fdad80b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58178/0 (socket says 172.21.15.49:58178) 2023-07-18T03:41:01.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdaddd9c700 1 -- 172.21.15.49:0/4114970484 learned_addr learned my addr 172.21.15.49:0/4114970484 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:01.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdadd59b700 1 --2- 172.21.15.49:0/4114970484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdad815c860 0x7fdad80d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:01.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdaddd9c700 1 -- 172.21.15.49:0/4114970484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdad815c860 msgr2=0x7fdad80d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:01.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdaddd9c700 1 --2- 172.21.15.49:0/4114970484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdad815c860 0x7fdad80d1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdaddd9c700 1 -- 172.21.15.49:0/4114970484 >> v1:172.21.15.107:6789/0 conn(0x7fdad80cf2a0 legacy=0x7fdad80c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:01.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.440+0000 7fdaddd9c700 1 -- 172.21.15.49:0/4114970484 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad80cd7f0 con 0x7fdad80c71b0 2023-07-18T03:41:01.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.441+0000 7fdaddd9c700 1 --2- 172.21.15.49:0/4114970484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdad80c71b0 0x7fdad80b49d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009e50 tx=0x7fdad000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ae9500e6b8a642 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:01.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.441+0000 7fdacffff700 1 -- 172.21.15.49:0/4114970484 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad000e070 con 0x7fdad80c71b0 2023-07-18T03:41:01.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.441+0000 7fdacffff700 1 -- 172.21.15.49:0/4114970484 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad000b910 con 0x7fdad80c71b0 2023-07-18T03:41:01.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.441+0000 7fdacffff700 1 -- 172.21.15.49:0/4114970484 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad000bcc0 con 0x7fdad80c71b0 2023-07-18T03:41:01.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.442+0000 7fdadf7ff700 1 -- 172.21.15.49:0/4114970484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdad80c71b0 msgr2=0x7fdad80b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:01.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.442+0000 7fdadf7ff700 1 --2- 172.21.15.49:0/4114970484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdad80c71b0 0x7fdad80b49d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009e50 tx=0x7fdad000b330 comp rx=0 tx=0).stop 2023-07-18T03:41:01.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.442+0000 7fdadf7ff700 1 -- 172.21.15.49:0/4114970484 shutdown_connections 2023-07-18T03:41:01.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.442+0000 7fdadf7ff700 1 --2- 172.21.15.49:0/4114970484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdad80c71b0 0x7fdad80b49d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.442+0000 7fdadf7ff700 1 --2- 172.21.15.49:0/4114970484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdad815c860 0x7fdad80d1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.442+0000 7fdadf7ff700 1 -- 172.21.15.49:0/4114970484 >> 172.21.15.49:0/4114970484 conn(0x7fdad8065bd0 msgr2=0x7fdad8065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:01.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.442+0000 7fdadf7ff700 1 -- 172.21.15.49:0/4114970484 shutdown_connections 2023-07-18T03:41:01.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.442+0000 7fdadf7ff700 1 -- 172.21.15.49:0/4114970484 wait complete. 2023-07-18T03:41:01.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.443+0000 7fdadf7ff700 1 Processor -- start 2023-07-18T03:41:01.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.444+0000 7fdadf7ff700 1 -- start start 2023-07-18T03:41:01.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadf7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdad80c71b0 0x7fdad8155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:01.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadf7ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdad80cf2a0 0x7fdad80b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:01.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadf7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad815c860 0x7fdad80b8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:01.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadf7ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad81f6430 con 0x7fdad80cf2a0 2023-07-18T03:41:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadf7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad81f57b0 con 0x7fdad815c860 2023-07-18T03:41:01.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadf7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad81f5a90 con 0x7fdad80c71b0 2023-07-18T03:41:01.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadcd9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdad80cf2a0 0x7fdad80b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:01.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadcd9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdad80cf2a0 0x7fdad80b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58182/0 (socket says 172.21.15.49:58182) 2023-07-18T03:41:01.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadcd9a700 1 -- 172.21.15.49:0/111222453 learned_addr learned my addr 172.21.15.49:0/111222453 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:01.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdaddd9c700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad815c860 0x7fdad80b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:01.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadd59b700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdad80c71b0 0x7fdad8155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadcd9a700 1 -- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdad80c71b0 msgr2=0x7fdad8155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:01.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadcd9a700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdad80c71b0 0x7fdad8155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadcd9a700 1 -- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad815c860 msgr2=0x7fdad80b8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadcd9a700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad815c860 0x7fdad80b8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdadcd9a700 1 -- 172.21.15.49:0/111222453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad0009b30 con 0x7fdad80cf2a0 2023-07-18T03:41:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.445+0000 7fdaddd9c700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad815c860 0x7fdad80b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.446+0000 7fdadcd9a700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdad80cf2a0 0x7fdad80b4860 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdac800c4e0 tx=0x7fdac800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.446+0000 7fdacdffb700 1 -- 172.21.15.49:0/111222453 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80052d0 con 0x7fdad80cf2a0 2023-07-18T03:41:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.446+0000 7fdacdffb700 1 -- 172.21.15.49:0/111222453 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac8005430 con 0x7fdad80cf2a0 2023-07-18T03:41:01.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.446+0000 7fdacdffb700 1 -- 172.21.15.49:0/111222453 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80038c0 con 0x7fdad80cf2a0 2023-07-18T03:41:01.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.446+0000 7fdadf7ff700 1 -- 172.21.15.49:0/111222453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad81f5d70 con 0x7fdad80cf2a0 2023-07-18T03:41:01.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.446+0000 7fdadf7ff700 1 -- 172.21.15.49:0/111222453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad81558a0 con 0x7fdad80cf2a0 2023-07-18T03:41:01.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.447+0000 7fdadf7ff700 1 -- 172.21.15.49:0/111222453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad8074350 con 0x7fdad80cf2a0 2023-07-18T03:41:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.448+0000 7fdacdffb700 1 -- 172.21.15.49:0/111222453 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fdac8003a20 con 0x7fdad80cf2a0 2023-07-18T03:41:01.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.449+0000 7fdacdffb700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdabc06ae00 0x7fdabc06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.449+0000 7fdacdffb700 1 -- 172.21.15.49:0/111222453 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fdac8088a50 con 0x7fdad80cf2a0 2023-07-18T03:41:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.450+0000 7fdadd59b700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdabc06ae00 0x7fdabc06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.450+0000 7fdadd59b700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdabc06ae00 0x7fdabc06d2b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009ad0 tx=0x7fdad4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.453+0000 7fdacdffb700 1 -- 172.21.15.49:0/111222453 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdac8053910 con 0x7fdad80cf2a0 2023-07-18T03:41:01.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.660+0000 7fdadf7ff700 1 -- 172.21.15.49:0/111222453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdad80cf710 con 0x7fdad80cf2a0 2023-07-18T03:41:01.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.660+0000 7fdacdffb700 1 -- 172.21.15.49:0/111222453 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdad80cf710 con 0x7fdad80cf2a0 2023-07-18T03:41:01.655 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:41:01.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.662+0000 7fdadf7ff700 1 -- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdabc06ae00 msgr2=0x7fdabc06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:01.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.662+0000 7fdadf7ff700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdabc06ae00 0x7fdabc06d2b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009ad0 tx=0x7fdad4008040 comp rx=0 tx=0).stop 2023-07-18T03:41:01.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.662+0000 7fdadf7ff700 1 -- 172.21.15.49:0/111222453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdad80cf2a0 msgr2=0x7fdad80b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:01.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.662+0000 7fdadf7ff700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdad80cf2a0 0x7fdad80b4860 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdac800c4e0 tx=0x7fdac800c8a0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.662+0000 7fdadf7ff700 1 -- 172.21.15.49:0/111222453 shutdown_connections 2023-07-18T03:41:01.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.662+0000 7fdadf7ff700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad815c860 0x7fdad80b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.662+0000 7fdadf7ff700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdad80c71b0 0x7fdad8155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.662+0000 7fdadf7ff700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fdabc06ae00 0x7fdabc06d2b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.662+0000 7fdadf7ff700 1 --2- 172.21.15.49:0/111222453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdad80cf2a0 0x7fdad80b4860 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.662+0000 7fdadf7ff700 1 -- 172.21.15.49:0/111222453 >> 172.21.15.49:0/111222453 conn(0x7fdad8065bd0 msgr2=0x7fdad8159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:01.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.663+0000 7fdadf7ff700 1 -- 172.21.15.49:0/111222453 shutdown_connections 2023-07-18T03:41:01.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.663+0000 7fdadf7ff700 1 -- 172.21.15.49:0/111222453 wait complete. 2023-07-18T03:41:01.675 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:41:01.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.893+0000 7f0e8c1ff700 1 Processor -- start 2023-07-18T03:41:01.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.893+0000 7f0e8c1ff700 1 -- start start 2023-07-18T03:41:01.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.894+0000 7f0e8c1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0e840d0b10 0x7f0e840d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:01.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.894+0000 7f0e8c1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e840b0e20 0x7f0e840b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:01.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.894+0000 7f0e8c1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e840b8e30 con 0x7f0e840b8a60 2023-07-18T03:41:01.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.894+0000 7f0e8c1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e840d14d0 con 0x7f0e840d0b10 2023-07-18T03:41:01.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.894+0000 7f0e8c1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840afaa0 con 0x7f0e840b0e20 2023-07-18T03:41:01.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.895+0000 7f0e8a79c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e840b0e20 0x7f0e840b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:01.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.895+0000 7f0e8979a700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0e840d0b10 0x7f0e840d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:01.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.895+0000 7f0e89f9b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f0e840b8a60 0x7f0e840b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:44678/0 (socket says 172.21.15.49:44678) 2023-07-18T03:41:01.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.895+0000 7f0e89f9b700 1 -- 172.21.15.49:0/2926220090 learned_addr learned my addr 172.21.15.49:0/2926220090 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:01.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.895+0000 7f0e8979a700 1 -- 172.21.15.49:0/2926220090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e840b0e20 msgr2=0x7f0e840b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:01.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.895+0000 7f0e8979a700 1 --2- 172.21.15.49:0/2926220090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e840b0e20 0x7f0e840b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.895+0000 7f0e8979a700 1 -- 172.21.15.49:0/2926220090 >> v1:172.21.15.107:6789/0 conn(0x7f0e840b8a60 legacy=0x7f0e840b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:01.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.895+0000 7f0e88f99700 1 -- 172.21.15.49:0/2926220090 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106372628 0 0) 0x7f0e840b8e30 con 0x7f0e840b8a60 2023-07-18T03:41:01.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.895+0000 7f0e8979a700 1 -- 172.21.15.49:0/2926220090 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e840afd80 con 0x7f0e840d0b10 2023-07-18T03:41:01.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.896+0000 7f0e8979a700 1 --2- 172.21.15.49:0/2926220090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e840d0b10 0x7f0e840d0f90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009840 tx=0x7f0e74009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e294171b473cc8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:01.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.896+0000 7f0e88f99700 1 -- 172.21.15.49:0/2926220090 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74011830 con 0x7f0e840d0b10 2023-07-18T03:41:01.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.896+0000 7f0e88f99700 1 -- 172.21.15.49:0/2926220090 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e74011990 con 0x7f0e840d0b10 2023-07-18T03:41:01.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.896+0000 7f0e88f99700 1 -- 172.21.15.49:0/2926220090 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74011d20 con 0x7f0e840d0b10 2023-07-18T03:41:01.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.896+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/2926220090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e840d0b10 msgr2=0x7f0e840d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:01.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.896+0000 7f0e8c1ff700 1 --2- 172.21.15.49:0/2926220090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e840d0b10 0x7f0e840d0f90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009840 tx=0x7f0e74009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:01.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.897+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/2926220090 shutdown_connections 2023-07-18T03:41:01.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.897+0000 7f0e8c1ff700 1 --2- 172.21.15.49:0/2926220090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e840b0e20 0x7f0e840b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.897+0000 7f0e8c1ff700 1 --2- 172.21.15.49:0/2926220090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e840d0b10 0x7f0e840d0f90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.897+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/2926220090 >> 172.21.15.49:0/2926220090 conn(0x7f0e8409c1b0 msgr2=0x7f0e840d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:01.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.897+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/2926220090 shutdown_connections 2023-07-18T03:41:01.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.897+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/2926220090 wait complete. 2023-07-18T03:41:01.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.898+0000 7f0e8c1ff700 1 Processor -- start 2023-07-18T03:41:01.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.899+0000 7f0e8c1ff700 1 -- start start 2023-07-18T03:41:01.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.899+0000 7f0e8c1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e840b0e20 0x7f0e8415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:01.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8c1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e840b8a60 0x7f0e840cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:01.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8c1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e840d0b10 0x7f0e840cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:01.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8c1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8415ef80 con 0x7f0e840b8a60 2023-07-18T03:41:01.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8c1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8415f1e0 con 0x7f0e840d0b10 2023-07-18T03:41:01.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8c1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840ca0b0 con 0x7f0e840b0e20 2023-07-18T03:41:01.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8979a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e840b8a60 0x7f0e840cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:01.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8979a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e840b8a60 0x7f0e840cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58194/0 (socket says 172.21.15.49:58194) 2023-07-18T03:41:01.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8979a700 1 -- 172.21.15.49:0/3545953265 learned_addr learned my addr 172.21.15.49:0/3545953265 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:01.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e89f9b700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e840b0e20 0x7f0e8415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:01.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8a79c700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e840d0b10 0x7f0e840cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:01.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8979a700 1 -- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e840b0e20 msgr2=0x7f0e8415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:01.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8979a700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e840b0e20 0x7f0e8415ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8979a700 1 -- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e840d0b10 msgr2=0x7f0e840cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:01.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8979a700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e840d0b10 0x7f0e840cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:01.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8979a700 1 -- 172.21.15.49:0/3545953265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e740094a0 con 0x7f0e840b8a60 2023-07-18T03:41:01.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.900+0000 7f0e8a79c700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e840d0b10 0x7f0e840cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:01.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.901+0000 7f0e8979a700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e840b8a60 0x7f0e840cae70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009fd0 tx=0x7f0e74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:01.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.901+0000 7f0e7a7fc700 1 -- 172.21.15.49:0/3545953265 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e740036a0 con 0x7f0e840b8a60 2023-07-18T03:41:01.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.901+0000 7f0e7a7fc700 1 -- 172.21.15.49:0/3545953265 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7401f070 con 0x7f0e840b8a60 2023-07-18T03:41:01.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.901+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/3545953265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e840ca390 con 0x7f0e840b8a60 2023-07-18T03:41:01.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.901+0000 7f0e7a7fc700 1 -- 172.21.15.49:0/3545953265 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74003a20 con 0x7f0e840b8a60 2023-07-18T03:41:01.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.901+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/3545953265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e840ca950 con 0x7f0e840b8a60 2023-07-18T03:41:01.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.902+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/3545953265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8408b870 con 0x7f0e840b8a60 2023-07-18T03:41:01.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.903+0000 7f0e7a7fc700 1 -- 172.21.15.49:0/3545953265 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f0e74003b80 con 0x7f0e840b8a60 2023-07-18T03:41:01.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.904+0000 7f0e7a7fc700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0e6806ae00 0x7f0e6806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:01.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.904+0000 7f0e7a7fc700 1 -- 172.21.15.49:0/3545953265 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f0e74021070 con 0x7f0e840b8a60 2023-07-18T03:41:01.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.905+0000 7f0e89f9b700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0e6806ae00 0x7f0e6806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:01.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.905+0000 7f0e89f9b700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0e6806ae00 0x7f0e6806d2b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0e80007fd0 tx=0x7f0e80007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:01.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:01.908+0000 7f0e7a7fc700 1 -- 172.21.15.49:0/3545953265 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0e74055e10 con 0x7f0e840b8a60 2023-07-18T03:41:02.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.117+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/3545953265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0e84074350 con 0x7f0e840b8a60 2023-07-18T03:41:02.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.118+0000 7f0e7a7fc700 1 -- 172.21.15.49:0/3545953265 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e74059430 con 0x7f0e840b8a60 2023-07-18T03:41:02.112 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:41:02.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.120+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0e6806ae00 msgr2=0x7f0e6806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:02.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.120+0000 7f0e8c1ff700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0e6806ae00 0x7f0e6806d2b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0e80007fd0 tx=0x7f0e80007de0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.120+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/3545953265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e840b8a60 msgr2=0x7f0e840cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:02.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.120+0000 7f0e8c1ff700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e840b8a60 0x7f0e840cae70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009fd0 tx=0x7f0e74004750 comp rx=0 tx=0).stop 2023-07-18T03:41:02.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.121+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/3545953265 shutdown_connections 2023-07-18T03:41:02.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.121+0000 7f0e8c1ff700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e840d0b10 0x7f0e840cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.121+0000 7f0e8c1ff700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e840b0e20 0x7f0e8415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.121+0000 7f0e8c1ff700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f0e6806ae00 0x7f0e6806d2b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.121+0000 7f0e8c1ff700 1 --2- 172.21.15.49:0/3545953265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e840b8a60 0x7f0e840cae70 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.121+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/3545953265 >> 172.21.15.49:0/3545953265 conn(0x7f0e8409c1b0 msgr2=0x7f0e840b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:02.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.121+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/3545953265 shutdown_connections 2023-07-18T03:41:02.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.121+0000 7f0e8c1ff700 1 -- 172.21.15.49:0/3545953265 wait complete. 2023-07-18T03:41:02.133 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-18T03:41:02.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.345+0000 7f8e19fff700 1 Processor -- start 2023-07-18T03:41:02.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.346+0000 7f8e19fff700 1 -- start start 2023-07-18T03:41:02.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.347+0000 7f8e19fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8e1406c9f0 0x7f8e140b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:02.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.347+0000 7f8e19fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e140b1e50 0x7f8e14071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:02.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.347+0000 7f8e19fff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1406bcc0 con 0x7f8e1415d9d0 2023-07-18T03:41:02.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.347+0000 7f8e19fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1406c4e0 con 0x7f8e140b1e50 2023-07-18T03:41:02.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.347+0000 7f8e19fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1415dda0 con 0x7f8e1406c9f0 2023-07-18T03:41:02.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.347+0000 7f8e13fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e140b1e50 0x7f8e14071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:02.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.347+0000 7f8e137fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8e1415d9d0 0x7f8e1406b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50524/0 (socket says 172.21.15.49:50524) 2023-07-18T03:41:02.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.347+0000 7f8e137fe700 1 -- 172.21.15.49:0/2258316812 learned_addr learned my addr 172.21.15.49:0/2258316812 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:02.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.348+0000 7f8e0bfff700 1 --2- 172.21.15.49:0/2258316812 >> v2:172.21.15.107:3300/0 conn(0x7f8e1406c9f0 0x7f8e140b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:02.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.348+0000 7f8e13fff700 1 -- 172.21.15.49:0/2258316812 >> v2:172.21.15.107:3300/0 conn(0x7f8e1406c9f0 msgr2=0x7f8e140b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:02.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.348+0000 7f8e13fff700 1 --2- 172.21.15.49:0/2258316812 >> v2:172.21.15.107:3300/0 conn(0x7f8e1406c9f0 0x7f8e140b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.348+0000 7f8e13fff700 1 -- 172.21.15.49:0/2258316812 >> v1:172.21.15.49:6789/0 conn(0x7f8e1415d9d0 legacy=0x7f8e1406b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:02.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.348+0000 7f8e13fff700 1 -- 172.21.15.49:0/2258316812 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1406bf20 con 0x7f8e140b1e50 2023-07-18T03:41:02.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.348+0000 7f8e12ffd700 1 -- 172.21.15.49:0/2258316812 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995048113 0 0) 0x7f8e1406bcc0 con 0x7f8e1415d9d0 2023-07-18T03:41:02.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.348+0000 7f8e13fff700 1 --2- 172.21.15.49:0/2258316812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e140b1e50 0x7f8e14071170 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8e140720b0 tx=0x7f8e0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0df472361f47d61 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:02.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.348+0000 7f8e12ffd700 1 -- 172.21.15.49:0/2258316812 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00e070 con 0x7f8e140b1e50 2023-07-18T03:41:02.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.348+0000 7f8e12ffd700 1 -- 172.21.15.49:0/2258316812 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c00bbd0 con 0x7f8e140b1e50 2023-07-18T03:41:02.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.348+0000 7f8e12ffd700 1 -- 172.21.15.49:0/2258316812 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c012430 con 0x7f8e140b1e50 2023-07-18T03:41:02.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.349+0000 7f8e19fff700 1 -- 172.21.15.49:0/2258316812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e140b1e50 msgr2=0x7f8e14071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:02.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.349+0000 7f8e19fff700 1 --2- 172.21.15.49:0/2258316812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e140b1e50 0x7f8e14071170 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8e140720b0 tx=0x7f8e0c00b330 comp rx=0 tx=0).stop 2023-07-18T03:41:02.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.349+0000 7f8e19fff700 1 -- 172.21.15.49:0/2258316812 shutdown_connections 2023-07-18T03:41:02.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.349+0000 7f8e19fff700 1 --2- 172.21.15.49:0/2258316812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e140b1e50 0x7f8e14071170 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.349+0000 7f8e19fff700 1 --2- 172.21.15.49:0/2258316812 >> v2:172.21.15.107:3300/0 conn(0x7f8e1406c9f0 0x7f8e140b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.349+0000 7f8e19fff700 1 -- 172.21.15.49:0/2258316812 >> 172.21.15.49:0/2258316812 conn(0x7f8e140b05d0 msgr2=0x7f8e140b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:02.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.349+0000 7f8e19fff700 1 -- 172.21.15.49:0/2258316812 shutdown_connections 2023-07-18T03:41:02.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.349+0000 7f8e19fff700 1 -- 172.21.15.49:0/2258316812 wait complete. 2023-07-18T03:41:02.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.351+0000 7f8e19fff700 1 Processor -- start 2023-07-18T03:41:02.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.351+0000 7f8e19fff700 1 -- start start 2023-07-18T03:41:02.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.352+0000 7f8e19fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e1406c9f0 0x7f8e140cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:02.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.352+0000 7f8e19fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e140b1e50 0x7f8e14070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:02.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.352+0000 7f8e19fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e1415d9d0 0x7f8e14071500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:02.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.352+0000 7f8e19fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e141f1d10 con 0x7f8e1406c9f0 2023-07-18T03:41:02.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.352+0000 7f8e19fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140cd640 con 0x7f8e1415d9d0 2023-07-18T03:41:02.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.352+0000 7f8e19fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140cd920 con 0x7f8e140b1e50 2023-07-18T03:41:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.352+0000 7f8e137fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e1406c9f0 0x7f8e140cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.352+0000 7f8e137fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e1406c9f0 0x7f8e140cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58218/0 (socket says 172.21.15.49:58218) 2023-07-18T03:41:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.352+0000 7f8e137fe700 1 -- 172.21.15.49:0/2325519314 learned_addr learned my addr 172.21.15.49:0/2325519314 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.353+0000 7f8e0bfff700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e140b1e50 0x7f8e14070fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.353+0000 7f8e13fff700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e1415d9d0 0x7f8e14071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:02.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.353+0000 7f8e137fe700 1 -- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e140b1e50 msgr2=0x7f8e14070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:02.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.353+0000 7f8e137fe700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e140b1e50 0x7f8e14070fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.353+0000 7f8e137fe700 1 -- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e1415d9d0 msgr2=0x7f8e14071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:02.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.353+0000 7f8e137fe700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e1415d9d0 0x7f8e14071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.353+0000 7f8e137fe700 1 -- 172.21.15.49:0/2325519314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c009a70 con 0x7f8e1406c9f0 2023-07-18T03:41:02.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.353+0000 7f8e137fe700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e1406c9f0 0x7f8e140cd060 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400c550 tx=0x7f8e0400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:02.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.353+0000 7f8e10ff9700 1 -- 172.21.15.49:0/2325519314 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04004c80 con 0x7f8e1406c9f0 2023-07-18T03:41:02.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.354+0000 7f8e19fff700 1 -- 172.21.15.49:0/2325519314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e140cdc00 con 0x7f8e1406c9f0 2023-07-18T03:41:02.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.354+0000 7f8e10ff9700 1 -- 172.21.15.49:0/2325519314 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e04004e00 con 0x7f8e1406c9f0 2023-07-18T03:41:02.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.354+0000 7f8e10ff9700 1 -- 172.21.15.49:0/2325519314 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04007310 con 0x7f8e1406c9f0 2023-07-18T03:41:02.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.354+0000 7f8e19fff700 1 -- 172.21.15.49:0/2325519314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e140ce1c0 con 0x7f8e1406c9f0 2023-07-18T03:41:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.355+0000 7f8e19fff700 1 -- 172.21.15.49:0/2325519314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e14074350 con 0x7f8e1406c9f0 2023-07-18T03:41:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.356+0000 7f8e10ff9700 1 -- 172.21.15.49:0/2325519314 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f8e04007470 con 0x7f8e1406c9f0 2023-07-18T03:41:02.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.356+0000 7f8e10ff9700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8df406ae00 0x7f8df406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.357+0000 7f8e10ff9700 1 -- 172.21.15.49:0/2325519314 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f8e040893e0 con 0x7f8e1406c9f0 2023-07-18T03:41:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.357+0000 7f8e0bfff700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8df406ae00 0x7f8df406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:02.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.358+0000 7f8e0bfff700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8df406ae00 0x7f8df406d2b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8e140c6b70 tx=0x7f8dfc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:02.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.361+0000 7f8e10ff9700 1 -- 172.21.15.49:0/2325519314 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8e040542b0 con 0x7f8e1406c9f0 2023-07-18T03:41:02.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.588+0000 7f8e19fff700 1 -- 172.21.15.49:0/2325519314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8e140733d0 con 0x7f8e1406c9f0 2023-07-18T03:41:02.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.589+0000 7f8e10ff9700 1 -- 172.21.15.49:0/2325519314 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8e040578d0 con 0x7f8e1406c9f0 2023-07-18T03:41:02.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.591+0000 7f8e19fff700 1 -- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8df406ae00 msgr2=0x7f8df406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:02.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.591+0000 7f8e19fff700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8df406ae00 0x7f8df406d2b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8e140c6b70 tx=0x7f8dfc008040 comp rx=0 tx=0).stop 2023-07-18T03:41:02.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.591+0000 7f8e19fff700 1 -- 172.21.15.49:0/2325519314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e1406c9f0 msgr2=0x7f8e140cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:02.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.591+0000 7f8e19fff700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e1406c9f0 0x7f8e140cd060 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400c550 tx=0x7f8e0400c910 comp rx=0 tx=0).stop 2023-07-18T03:41:02.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.591+0000 7f8e19fff700 1 -- 172.21.15.49:0/2325519314 shutdown_connections 2023-07-18T03:41:02.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.591+0000 7f8e19fff700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e1415d9d0 0x7f8e14071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.591+0000 7f8e19fff700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f8df406ae00 0x7f8df406d2b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.591+0000 7f8e19fff700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e1406c9f0 0x7f8e140cd060 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.591+0000 7f8e19fff700 1 --2- 172.21.15.49:0/2325519314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e140b1e50 0x7f8e14070fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:02.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.591+0000 7f8e19fff700 1 -- 172.21.15.49:0/2325519314 >> 172.21.15.49:0/2325519314 conn(0x7f8e140b05d0 msgr2=0x7f8e140b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:02.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.591+0000 7f8e19fff700 1 -- 172.21.15.49:0/2325519314 shutdown_connections 2023-07-18T03:41:02.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:02.591+0000 7f8e19fff700 1 -- 172.21.15.49:0/2325519314 wait complete. 2023-07-18T03:41:02.587 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-07-18T03:41:02.605 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-07-18T03:41:02.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:02.606 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-07-18T03:41:02.699 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:41:02.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:02.699 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-18T03:41:02.745 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:41:02.746 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:41:02.792 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345544k,nr_inodes=4086386,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-18T03:41:02.793 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:41:02.793 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:41:02.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345444k,nr_inodes=4086361,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-18T03:41:02.838 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:41:03.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.049+0000 7fe807bff700 1 Processor -- start 2023-07-18T03:41:03.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.050+0000 7fe807bff700 1 -- start start 2023-07-18T03:41:03.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.051+0000 7fe807bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe80015c950 0x7fe8000b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:03.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.051+0000 7fe807bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8000b1e00 0x7fe8000d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:03.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.051+0000 7fe807bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8000d2ce0 con 0x7fe8000b2350 2023-07-18T03:41:03.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.051+0000 7fe807bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8000b3350 con 0x7fe8000b1e00 2023-07-18T03:41:03.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.051+0000 7fe807bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8000b3630 con 0x7fe80015c950 2023-07-18T03:41:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.051+0000 7fe80519a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8000b1e00 0x7fe8000d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.051+0000 7fe80519a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8000b1e00 0x7fe8000d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58226/0 (socket says 172.21.15.49:58226) 2023-07-18T03:41:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.051+0000 7fe80519a700 1 -- 172.21.15.49:0/1428932983 learned_addr learned my addr 172.21.15.49:0/1428932983 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:03.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.051+0000 7fe80599b700 1 --2- 172.21.15.49:0/1428932983 >> v2:172.21.15.107:3300/0 conn(0x7fe80015c950 0x7fe8000b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.052+0000 7fe80519a700 1 -- 172.21.15.49:0/1428932983 >> v2:172.21.15.107:3300/0 conn(0x7fe80015c950 msgr2=0x7fe8000b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.052+0000 7fe80519a700 1 --2- 172.21.15.49:0/1428932983 >> v2:172.21.15.107:3300/0 conn(0x7fe80015c950 0x7fe8000b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.052+0000 7fe80519a700 1 -- 172.21.15.49:0/1428932983 >> v1:172.21.15.107:6789/0 conn(0x7fe8000b2350 legacy=0x7fe8000afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.052+0000 7fe80519a700 1 -- 172.21.15.49:0/1428932983 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8000b3910 con 0x7fe8000b1e00 2023-07-18T03:41:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.052+0000 7fe80519a700 1 --2- 172.21.15.49:0/1428932983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8000b1e00 0x7fe8000d0b10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe80008d170 tx=0x7fe7f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab49b8403c9ccca server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.052+0000 7fe804999700 1 -- 172.21.15.49:0/1428932983 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0011830 con 0x7fe8000b1e00 2023-07-18T03:41:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.052+0000 7fe804999700 1 -- 172.21.15.49:0/1428932983 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f0011990 con 0x7fe8000b1e00 2023-07-18T03:41:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.052+0000 7fe804999700 1 -- 172.21.15.49:0/1428932983 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0011d20 con 0x7fe8000b1e00 2023-07-18T03:41:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.053+0000 7fe807bff700 1 -- 172.21.15.49:0/1428932983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8000b1e00 msgr2=0x7fe8000d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:03.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.053+0000 7fe807bff700 1 --2- 172.21.15.49:0/1428932983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8000b1e00 0x7fe8000d0b10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe80008d170 tx=0x7fe7f0009bd0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.053+0000 7fe807bff700 1 -- 172.21.15.49:0/1428932983 shutdown_connections 2023-07-18T03:41:03.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.053+0000 7fe807bff700 1 --2- 172.21.15.49:0/1428932983 >> v2:172.21.15.107:3300/0 conn(0x7fe80015c950 0x7fe8000b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.053+0000 7fe807bff700 1 --2- 172.21.15.49:0/1428932983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8000b1e00 0x7fe8000d0b10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.053+0000 7fe807bff700 1 -- 172.21.15.49:0/1428932983 >> 172.21.15.49:0/1428932983 conn(0x7fe80009c1d0 msgr2=0x7fe8000d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:03.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.053+0000 7fe807bff700 1 -- 172.21.15.49:0/1428932983 shutdown_connections 2023-07-18T03:41:03.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.053+0000 7fe807bff700 1 -- 172.21.15.49:0/1428932983 wait complete. 2023-07-18T03:41:03.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.055+0000 7fe807bff700 1 Processor -- start 2023-07-18T03:41:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.055+0000 7fe807bff700 1 -- start start 2023-07-18T03:41:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.056+0000 7fe807bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8000b1e00 0x7fe80014e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.056+0000 7fe807bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8000b2350 0x7fe8000b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:03.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.056+0000 7fe807bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe80015c950 0x7fe8000b8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:03.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.056+0000 7fe807bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8000c7c40 con 0x7fe80015c950 2023-07-18T03:41:03.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.056+0000 7fe807bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8000c6ba0 con 0x7fe8000b1e00 2023-07-18T03:41:03.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.056+0000 7fe807bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8000c6e80 con 0x7fe8000b2350 2023-07-18T03:41:03.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.057+0000 7fe80619c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe80015c950 0x7fe8000b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.057+0000 7fe80599b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8000b1e00 0x7fe80014e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.057+0000 7fe80599b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8000b1e00 0x7fe80014e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:56200/0 (socket says 172.21.15.49:56200) 2023-07-18T03:41:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.057+0000 7fe80599b700 1 -- 172.21.15.49:0/3609372641 learned_addr learned my addr 172.21.15.49:0/3609372641 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.057+0000 7fe80519a700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8000b2350 0x7fe8000b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:03.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.057+0000 7fe80599b700 1 -- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8000b2350 msgr2=0x7fe8000b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:03.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.057+0000 7fe80599b700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8000b2350 0x7fe8000b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.057+0000 7fe80599b700 1 -- 172.21.15.49:0/3609372641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe80015c950 msgr2=0x7fe8000b8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.057+0000 7fe80599b700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe80015c950 0x7fe8000b8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.057+0000 7fe80599b700 1 -- 172.21.15.49:0/3609372641 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f0009550 con 0x7fe8000b1e00 2023-07-18T03:41:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.057+0000 7fe80619c700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe80015c950 0x7fe8000b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.057+0000 7fe80599b700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8000b1e00 0x7fe80014e620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00e900 tx=0x7fe7fc00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.058+0000 7fe7f67fc700 1 -- 172.21.15.49:0/3609372641 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc013680 con 0x7fe8000b1e00 2023-07-18T03:41:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.058+0000 7fe7f67fc700 1 -- 172.21.15.49:0/3609372641 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc0137e0 con 0x7fe8000b1e00 2023-07-18T03:41:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.058+0000 7fe7f67fc700 1 -- 172.21.15.49:0/3609372641 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc013b50 con 0x7fe8000b1e00 2023-07-18T03:41:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.058+0000 7fe807bff700 1 -- 172.21.15.49:0/3609372641 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8000c7160 con 0x7fe8000b1e00 2023-07-18T03:41:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.058+0000 7fe807bff700 1 -- 172.21.15.49:0/3609372641 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8000c7720 con 0x7fe8000b1e00 2023-07-18T03:41:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.059+0000 7fe807bff700 1 -- 172.21.15.49:0/3609372641 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80004e280 con 0x7fe8000b1e00 2023-07-18T03:41:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.060+0000 7fe7f67fc700 1 -- 172.21.15.49:0/3609372641 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fe7fc013cb0 con 0x7fe8000b1e00 2023-07-18T03:41:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.061+0000 7fe7f67fc700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe7e406ae00 0x7fe7e406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.061+0000 7fe7f67fc700 1 -- 172.21.15.49:0/3609372641 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fe7fc089930 con 0x7fe8000b1e00 2023-07-18T03:41:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.061+0000 7fe80519a700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe7e406ae00 0x7fe7e406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.062+0000 7fe80519a700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe7e406ae00 0x7fe7e406d2b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0000c00 tx=0x7fe7f0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:03.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.066+0000 7fe7f67fc700 1 -- 172.21.15.49:0/3609372641 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe7fc054880 con 0x7fe8000b1e00 2023-07-18T03:41:03.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.317+0000 7fe807bff700 1 -- 172.21.15.49:0/3609372641 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe80008c5a0 con 0x7fe8000b1e00 2023-07-18T03:41:03.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.318+0000 7fe7f67fc700 1 -- 172.21.15.49:0/3609372641 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fe7fc021090 con 0x7fe8000b1e00 2023-07-18T03:41:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:03.313 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/777524677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":777524677},{"type":"v1","addr":"172.21.15.107:6837","nonce":777524677}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/2382452558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2382452558},{"type":"v1","addr":"172.21.15.49:6834","nonce":2382452558}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/2843143219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2843143219},{"type":"v1","addr":"172.21.15.107:6836","nonce":2843143219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:40:40.451705+0000","modified":"2023-07-18T03:40:43.957663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5479":{"gid":5479,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1453078087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":1453078087},{"type":"v1","addr":"172.21.15.49:6835","nonce":1453078087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-18T03:41:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.321+0000 7fe807bff700 1 -- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe7e406ae00 msgr2=0x7fe7e406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.321+0000 7fe807bff700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe7e406ae00 0x7fe7e406d2b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0000c00 tx=0x7fe7f0008030 comp rx=0 tx=0).stop 2023-07-18T03:41:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.321+0000 7fe807bff700 1 -- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8000b1e00 msgr2=0x7fe80014e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.321+0000 7fe807bff700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8000b1e00 0x7fe80014e620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00e900 tx=0x7fe7fc00ec10 comp rx=0 tx=0).stop 2023-07-18T03:41:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.321+0000 7fe807bff700 1 -- 172.21.15.49:0/3609372641 shutdown_connections 2023-07-18T03:41:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.321+0000 7fe807bff700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fe7e406ae00 0x7fe7e406d2b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.321+0000 7fe807bff700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe80015c950 0x7fe8000b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.321+0000 7fe807bff700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8000b1e00 0x7fe80014e620 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.321+0000 7fe807bff700 1 --2- 172.21.15.49:0/3609372641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8000b2350 0x7fe8000b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.321+0000 7fe807bff700 1 -- 172.21.15.49:0/3609372641 >> 172.21.15.49:0/3609372641 conn(0x7fe80009c1d0 msgr2=0x7fe8000b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:03.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.321+0000 7fe807bff700 1 -- 172.21.15.49:0/3609372641 shutdown_connections 2023-07-18T03:41:03.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.321+0000 7fe807bff700 1 -- 172.21.15.49:0/3609372641 wait complete. 2023-07-18T03:41:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 29 2023-07-18T03:41:03.333 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:41:03.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.544+0000 7ff509dff700 1 Processor -- start 2023-07-18T03:41:03.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.545+0000 7ff509dff700 1 -- start start 2023-07-18T03:41:03.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.546+0000 7ff509dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5041500d0 0x7ff504070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:03.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.546+0000 7ff509dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff504070890 0x7ff5041597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:03.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.546+0000 7ff509dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5040cdf90 con 0x7ff504159d00 2023-07-18T03:41:03.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.546+0000 7ff509dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50415a2f0 con 0x7ff504070890 2023-07-18T03:41:03.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.546+0000 7ff509dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50415a5d0 con 0x7ff5041500d0 2023-07-18T03:41:03.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.546+0000 7ff503fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff504159d00 0x7ff5040cea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50526/0 (socket says 172.21.15.49:50526) 2023-07-18T03:41:03.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.546+0000 7ff503fff700 1 -- 172.21.15.49:0/3978591257 learned_addr learned my addr 172.21.15.49:0/3978591257 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:03.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.546+0000 7ff5037fe700 1 --2- 172.21.15.49:0/3978591257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5041500d0 0x7ff504070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:03.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.546+0000 7ff502ffd700 1 --2- 172.21.15.49:0/3978591257 >> v2:172.21.15.107:3300/0 conn(0x7ff504070890 0x7ff5041597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:03.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.547+0000 7ff5037fe700 1 -- 172.21.15.49:0/3978591257 >> v2:172.21.15.107:3300/0 conn(0x7ff504070890 msgr2=0x7ff5041597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:03.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.547+0000 7ff5037fe700 1 --2- 172.21.15.49:0/3978591257 >> v2:172.21.15.107:3300/0 conn(0x7ff504070890 0x7ff5041597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.547+0000 7ff5027fc700 1 -- 172.21.15.49:0/3978591257 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920725146 0 0) 0x7ff5040cdf90 con 0x7ff504159d00 2023-07-18T03:41:03.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.547+0000 7ff5037fe700 1 -- 172.21.15.49:0/3978591257 >> v1:172.21.15.49:6789/0 conn(0x7ff504159d00 legacy=0x7ff5040cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:03.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.547+0000 7ff5037fe700 1 -- 172.21.15.49:0/3978591257 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5040ce250 con 0x7ff5041500d0 2023-07-18T03:41:03.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.547+0000 7ff502ffd700 1 --2- 172.21.15.49:0/3978591257 >> v2:172.21.15.107:3300/0 conn(0x7ff504070890 0x7ff5041597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:03.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.547+0000 7ff5037fe700 1 --2- 172.21.15.49:0/3978591257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5041500d0 0x7ff504070350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009840 tx=0x7ff4f4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=436a1a86c83fca20 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:03.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.547+0000 7ff5027fc700 1 -- 172.21.15.49:0/3978591257 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4011830 con 0x7ff5041500d0 2023-07-18T03:41:03.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.548+0000 7ff5027fc700 1 -- 172.21.15.49:0/3978591257 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f4011990 con 0x7ff5041500d0 2023-07-18T03:41:03.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.548+0000 7ff5027fc700 1 -- 172.21.15.49:0/3978591257 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4011d20 con 0x7ff5041500d0 2023-07-18T03:41:03.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.548+0000 7ff509dff700 1 -- 172.21.15.49:0/3978591257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5041500d0 msgr2=0x7ff504070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:03.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.548+0000 7ff509dff700 1 --2- 172.21.15.49:0/3978591257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5041500d0 0x7ff504070350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009840 tx=0x7ff4f4009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:03.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.548+0000 7ff509dff700 1 -- 172.21.15.49:0/3978591257 shutdown_connections 2023-07-18T03:41:03.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.548+0000 7ff509dff700 1 --2- 172.21.15.49:0/3978591257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5041500d0 0x7ff504070350 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.548+0000 7ff509dff700 1 --2- 172.21.15.49:0/3978591257 >> v2:172.21.15.107:3300/0 conn(0x7ff504070890 0x7ff5041597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.548+0000 7ff509dff700 1 -- 172.21.15.49:0/3978591257 >> 172.21.15.49:0/3978591257 conn(0x7ff504069f30 msgr2=0x7ff50406a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:03.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.548+0000 7ff509dff700 1 -- 172.21.15.49:0/3978591257 shutdown_connections 2023-07-18T03:41:03.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.548+0000 7ff509dff700 1 -- 172.21.15.49:0/3978591257 wait complete. 2023-07-18T03:41:03.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.550+0000 7ff509dff700 1 Processor -- start 2023-07-18T03:41:03.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.550+0000 7ff509dff700 1 -- start start 2023-07-18T03:41:03.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.551+0000 7ff509dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff504070890 0x7ff50413a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:03.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.551+0000 7ff509dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5041500d0 0x7ff504158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:03.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.551+0000 7ff509dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff504159d00 0x7ff504158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:03.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.551+0000 7ff509dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50414dae0 con 0x7ff504070890 2023-07-18T03:41:03.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.551+0000 7ff509dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50414d800 con 0x7ff504159d00 2023-07-18T03:41:03.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.551+0000 7ff509dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff504137df0 con 0x7ff5041500d0 2023-07-18T03:41:03.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.552+0000 7ff5037fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff504070890 0x7ff50413a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:03.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.552+0000 7ff5037fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff504070890 0x7ff50413a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58246/0 (socket says 172.21.15.49:58246) 2023-07-18T03:41:03.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.552+0000 7ff5037fe700 1 -- 172.21.15.49:0/3447583049 learned_addr learned my addr 172.21.15.49:0/3447583049 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:03.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.552+0000 7ff502ffd700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5041500d0 0x7ff504158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:03.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.552+0000 7ff503fff700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff504159d00 0x7ff504158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:03.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.552+0000 7ff5037fe700 1 -- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5041500d0 msgr2=0x7ff504158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:03.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.552+0000 7ff5037fe700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5041500d0 0x7ff504158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.552+0000 7ff5037fe700 1 -- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff504159d00 msgr2=0x7ff504158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:03.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.552+0000 7ff5037fe700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff504159d00 0x7ff504158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.552+0000 7ff5037fe700 1 -- 172.21.15.49:0/3447583049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f80094a0 con 0x7ff504070890 2023-07-18T03:41:03.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.552+0000 7ff503fff700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff504159d00 0x7ff504158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:03.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.552+0000 7ff5037fe700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff504070890 0x7ff50413a240 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009fd0 tx=0x7ff4f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:03.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.553+0000 7ff4ebfff700 1 -- 172.21.15.49:0/3447583049 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f40036a0 con 0x7ff504070890 2023-07-18T03:41:03.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.553+0000 7ff4ebfff700 1 -- 172.21.15.49:0/3447583049 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f401f070 con 0x7ff504070890 2023-07-18T03:41:03.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.553+0000 7ff509dff700 1 -- 172.21.15.49:0/3447583049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f40094a0 con 0x7ff504070890 2023-07-18T03:41:03.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.553+0000 7ff4ebfff700 1 -- 172.21.15.49:0/3447583049 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4003a20 con 0x7ff504070890 2023-07-18T03:41:03.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.553+0000 7ff509dff700 1 -- 172.21.15.49:0/3447583049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5041383b0 con 0x7ff504070890 2023-07-18T03:41:03.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.554+0000 7ff509dff700 1 -- 172.21.15.49:0/3447583049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50404e280 con 0x7ff504070890 2023-07-18T03:41:03.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.555+0000 7ff4ebfff700 1 -- 172.21.15.49:0/3447583049 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7ff4f4003b80 con 0x7ff504070890 2023-07-18T03:41:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.556+0000 7ff4ebfff700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff4f006ae00 0x7ff4f006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.557+0000 7ff4ebfff700 1 -- 172.21.15.49:0/3447583049 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7ff4f4021070 con 0x7ff504070890 2023-07-18T03:41:03.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.557+0000 7ff502ffd700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff4f006ae00 0x7ff4f006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:03.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.558+0000 7ff502ffd700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff4f006ae00 0x7ff4f006d2b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff50414aae0 tx=0x7ff4f8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:03.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.561+0000 7ff4ebfff700 1 -- 172.21.15.49:0/3447583049 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff4f4055f90 con 0x7ff504070890 2023-07-18T03:41:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.771+0000 7ff509dff700 1 -- 172.21.15.49:0/3447583049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff504070130 con 0x7ff504070890 2023-07-18T03:41:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.772+0000 7ff4ebfff700 1 -- 172.21.15.49:0/3447583049 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18365 (secure 0 0 0) 0x7ff4f4015090 con 0x7ff504070890 2023-07-18T03:41:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:03.767 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":50,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:41:00.954181+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:40:38.792808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-18T03:40:39.438590+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":8,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:41:03.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.775+0000 7ff509dff700 1 -- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff4f006ae00 msgr2=0x7ff4f006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:03.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.775+0000 7ff509dff700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff4f006ae00 0x7ff4f006d2b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff50414aae0 tx=0x7ff4f8009210 comp rx=0 tx=0).stop 2023-07-18T03:41:03.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.775+0000 7ff509dff700 1 -- 172.21.15.49:0/3447583049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff504070890 msgr2=0x7ff50413a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:03.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.775+0000 7ff509dff700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff504070890 0x7ff50413a240 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009fd0 tx=0x7ff4f4004750 comp rx=0 tx=0).stop 2023-07-18T03:41:03.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.775+0000 7ff509dff700 1 -- 172.21.15.49:0/3447583049 shutdown_connections 2023-07-18T03:41:03.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.775+0000 7ff509dff700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff504159d00 0x7ff504158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.775+0000 7ff509dff700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7ff4f006ae00 0x7ff4f006d2b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.775+0000 7ff509dff700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff504070890 0x7ff50413a240 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.775+0000 7ff509dff700 1 --2- 172.21.15.49:0/3447583049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5041500d0 0x7ff504158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.775+0000 7ff509dff700 1 -- 172.21.15.49:0/3447583049 >> 172.21.15.49:0/3447583049 conn(0x7ff504069f30 msgr2=0x7ff50414e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.775+0000 7ff509dff700 1 -- 172.21.15.49:0/3447583049 shutdown_connections 2023-07-18T03:41:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:03.775+0000 7ff509dff700 1 -- 172.21.15.49:0/3447583049 wait complete. 2023-07-18T03:41:03.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-18T03:38:27.914778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:38:28.163363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:40:38.792808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:40:39.438590+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 50, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:41:03.789 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-18T03:41:03.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.005+0000 7f413ffff700 1 Processor -- start 2023-07-18T03:41:03.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.005+0000 7f413ffff700 1 -- start start 2023-07-18T03:41:04.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.006+0000 7f413ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41380d0b10 0x7f41380d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.006+0000 7f413ffff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f41380b0e20 0x7f41380b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.006+0000 7f413ffff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41380d14a0 con 0x7f41380b8a60 2023-07-18T03:41:04.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.006+0000 7f413ffff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41380b8e30 con 0x7f41380b0e20 2023-07-18T03:41:04.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.006+0000 7f413ffff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41380afaa0 con 0x7f41380d0b10 2023-07-18T03:41:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.007+0000 7f413d59a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41380d0b10 0x7f41380d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.007+0000 7f413e59c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f41380b0e20 0x7f41380b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.007+0000 7f413dd9b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f41380b8a60 0x7f41380b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:44706/0 (socket says 172.21.15.49:44706) 2023-07-18T03:41:04.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.007+0000 7f413dd9b700 1 -- 172.21.15.49:0/1206326744 learned_addr learned my addr 172.21.15.49:0/1206326744 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:04.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.007+0000 7f413e59c700 1 -- 172.21.15.49:0/1206326744 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41380d0b10 msgr2=0x7f41380d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:04.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.007+0000 7f413e59c700 1 --2- 172.21.15.49:0/1206326744 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41380d0b10 0x7f41380d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.007+0000 7f413cd99700 1 -- 172.21.15.49:0/1206326744 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995701696 0 0) 0x7f4134001110 con 0x7f41380b8a60 2023-07-18T03:41:04.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.007+0000 7f413e59c700 1 -- 172.21.15.49:0/1206326744 >> v1:172.21.15.107:6789/0 conn(0x7f41380b8a60 legacy=0x7f41380b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.007+0000 7f413e59c700 1 -- 172.21.15.49:0/1206326744 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41380afd80 con 0x7f41380b0e20 2023-07-18T03:41:04.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.007+0000 7f413d59a700 1 --2- 172.21.15.49:0/1206326744 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41380d0b10 0x7f41380d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:04.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.008+0000 7f413e59c700 1 --2- 172.21.15.49:0/1206326744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41380b0e20 0x7f41380b12a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4130009d90 tx=0x7f413000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e18ad691bdf2340b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:04.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.008+0000 7f413cd99700 1 -- 172.21.15.49:0/1206326744 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413000e070 con 0x7f41380b0e20 2023-07-18T03:41:04.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.008+0000 7f413cd99700 1 -- 172.21.15.49:0/1206326744 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413000bb20 con 0x7f41380b0e20 2023-07-18T03:41:04.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.008+0000 7f413cd99700 1 -- 172.21.15.49:0/1206326744 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130012430 con 0x7f41380b0e20 2023-07-18T03:41:04.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.008+0000 7f413ffff700 1 -- 172.21.15.49:0/1206326744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41380b0e20 msgr2=0x7f41380b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.008+0000 7f413ffff700 1 --2- 172.21.15.49:0/1206326744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41380b0e20 0x7f41380b12a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4130009d90 tx=0x7f413000b330 comp rx=0 tx=0).stop 2023-07-18T03:41:04.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.009+0000 7f413ffff700 1 -- 172.21.15.49:0/1206326744 shutdown_connections 2023-07-18T03:41:04.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.009+0000 7f413ffff700 1 --2- 172.21.15.49:0/1206326744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41380b0e20 0x7f41380b12a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.009+0000 7f413ffff700 1 --2- 172.21.15.49:0/1206326744 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41380d0b10 0x7f41380d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.009+0000 7f413ffff700 1 -- 172.21.15.49:0/1206326744 >> 172.21.15.49:0/1206326744 conn(0x7f413809c1b0 msgr2=0x7f41380d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:04.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.009+0000 7f413ffff700 1 -- 172.21.15.49:0/1206326744 shutdown_connections 2023-07-18T03:41:04.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.009+0000 7f413ffff700 1 -- 172.21.15.49:0/1206326744 wait complete. 2023-07-18T03:41:04.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.010+0000 7f413ffff700 1 Processor -- start 2023-07-18T03:41:04.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.011+0000 7f413ffff700 1 -- start start 2023-07-18T03:41:04.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41380b0e20 0x7f41381551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41380b8a60 0x7f4138153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41380d0b10 0x7f4138154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413ffff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413815fe50 con 0x7f41380b0e20 2023-07-18T03:41:04.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413ffff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413815edb0 con 0x7f41380b8a60 2023-07-18T03:41:04.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413ffff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413815f090 con 0x7f41380d0b10 2023-07-18T03:41:04.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41380b0e20 0x7f41381551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41380b0e20 0x7f41381551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58250/0 (socket says 172.21.15.49:58250) 2023-07-18T03:41:04.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413dd9b700 1 -- 172.21.15.49:0/48086345 learned_addr learned my addr 172.21.15.49:0/48086345 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:04.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413d59a700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41380b8a60 0x7f4138153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413e59c700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41380d0b10 0x7f4138154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413dd9b700 1 -- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41380d0b10 msgr2=0x7f4138154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413dd9b700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41380d0b10 0x7f4138154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413dd9b700 1 -- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41380b8a60 msgr2=0x7f4138153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413dd9b700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41380b8a60 0x7f4138153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.012+0000 7f413dd9b700 1 -- 172.21.15.49:0/48086345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41280094a0 con 0x7f41380b0e20 2023-07-18T03:41:04.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.013+0000 7f413dd9b700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41380b0e20 0x7f41381551b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f413400c730 tx=0x7f413400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:04.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.013+0000 7f412e7fc700 1 -- 172.21.15.49:0/48086345 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134004c80 con 0x7f41380b0e20 2023-07-18T03:41:04.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.013+0000 7f412e7fc700 1 -- 172.21.15.49:0/48086345 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4134004e00 con 0x7f41380b0e20 2023-07-18T03:41:04.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.013+0000 7f412e7fc700 1 -- 172.21.15.49:0/48086345 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134007310 con 0x7f41380b0e20 2023-07-18T03:41:04.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.013+0000 7f413ffff700 1 -- 172.21.15.49:0/48086345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4130009a70 con 0x7f41380b0e20 2023-07-18T03:41:04.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.013+0000 7f413ffff700 1 -- 172.21.15.49:0/48086345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413815f650 con 0x7f41380b0e20 2023-07-18T03:41:04.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.014+0000 7f413ffff700 1 -- 172.21.15.49:0/48086345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4138074350 con 0x7f41380b0e20 2023-07-18T03:41:04.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.015+0000 7f412e7fc700 1 -- 172.21.15.49:0/48086345 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f4134007470 con 0x7f41380b0e20 2023-07-18T03:41:04.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.016+0000 7f412e7fc700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f411c06ae00 0x7f411c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.016+0000 7f412e7fc700 1 -- 172.21.15.49:0/48086345 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f4134089150 con 0x7f41380b0e20 2023-07-18T03:41:04.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.016+0000 7f413d59a700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f411c06ae00 0x7f411c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.017+0000 7f413d59a700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f411c06ae00 0x7f411c06d2b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f413815b280 tx=0x7f4128009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:04.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.020+0000 7f412e7fc700 1 -- 172.21.15.49:0/48086345 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f41340540a0 con 0x7f41380b0e20 2023-07-18T03:41:04.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.229+0000 7f413ffff700 1 -- 172.21.15.49:0/48086345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f41380d14a0 con 0x7f41380b0e20 2023-07-18T03:41:04.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.229+0000 7f412e7fc700 1 -- 172.21.15.49:0/48086345 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v50) v1 ==== 92+0+10 (secure 0 0 0) 0x7f41380d14a0 con 0x7f41380b0e20 2023-07-18T03:41:04.223 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-07-18T03:41:04.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.232+0000 7f413ffff700 1 -- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f411c06ae00 msgr2=0x7f411c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.232+0000 7f413ffff700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f411c06ae00 0x7f411c06d2b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f413815b280 tx=0x7f4128009210 comp rx=0 tx=0).stop 2023-07-18T03:41:04.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.232+0000 7f413ffff700 1 -- 172.21.15.49:0/48086345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41380b0e20 msgr2=0x7f41381551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.232+0000 7f413ffff700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41380b0e20 0x7f41381551b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f413400c730 tx=0x7f413400ca40 comp rx=0 tx=0).stop 2023-07-18T03:41:04.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.232+0000 7f413ffff700 1 -- 172.21.15.49:0/48086345 shutdown_connections 2023-07-18T03:41:04.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.232+0000 7f413ffff700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41380d0b10 0x7f4138154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.232+0000 7f413ffff700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f411c06ae00 0x7f411c06d2b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.232+0000 7f413ffff700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41380b0e20 0x7f41381551b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.232+0000 7f413ffff700 1 --2- 172.21.15.49:0/48086345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41380b8a60 0x7f4138153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.232+0000 7f413ffff700 1 -- 172.21.15.49:0/48086345 >> 172.21.15.49:0/48086345 conn(0x7f413809c1b0 msgr2=0x7f413815ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:04.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.232+0000 7f413ffff700 1 -- 172.21.15.49:0/48086345 shutdown_connections 2023-07-18T03:41:04.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.232+0000 7f413ffff700 1 -- 172.21.15.49:0/48086345 wait complete. 2023-07-18T03:41:04.244 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:41:04.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.457+0000 7f34895ff700 1 Processor -- start 2023-07-18T03:41:04.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.457+0000 7f34895ff700 1 -- start start 2023-07-18T03:41:04.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f34895ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f34840b8a60 0x7f34840b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f34895ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f34840b4530 0x7f34840d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f34895ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34840d1490 con 0x7f34840d1050 2023-07-18T03:41:04.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f34895ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34840b8e30 con 0x7f34840b8a60 2023-07-18T03:41:04.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f34895ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34840b1e30 con 0x7f34840b4530 2023-07-18T03:41:04.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f3482ffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f34840b8a60 0x7f34840b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f3482ffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f34840b8a60 0x7f34840b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58262/0 (socket says 172.21.15.49:58262) 2023-07-18T03:41:04.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f3482ffd700 1 -- 172.21.15.49:0/3069446583 learned_addr learned my addr 172.21.15.49:0/3069446583 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:04.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f347bfff700 1 --2- 172.21.15.49:0/3069446583 >> v2:172.21.15.107:3300/0 conn(0x7f34840b4530 0x7f34840d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f3482ffd700 1 -- 172.21.15.49:0/3069446583 >> v2:172.21.15.107:3300/0 conn(0x7f34840b4530 msgr2=0x7f34840d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:04.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f3482ffd700 1 --2- 172.21.15.49:0/3069446583 >> v2:172.21.15.107:3300/0 conn(0x7f34840b4530 0x7f34840d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f3482ffd700 1 -- 172.21.15.49:0/3069446583 >> v1:172.21.15.107:6789/0 conn(0x7f34840d1050 legacy=0x7f34840b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.459+0000 7f3482ffd700 1 -- 172.21.15.49:0/3069446583 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34840afaa0 con 0x7f34840b8a60 2023-07-18T03:41:04.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.460+0000 7f3482ffd700 1 --2- 172.21.15.49:0/3069446583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34840b8a60 0x7f34840b3ff0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f348409c8e0 tx=0x7f3474009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8133e512e73c059 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:04.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.460+0000 7f34827fc700 1 -- 172.21.15.49:0/3069446583 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347400b870 con 0x7f34840b8a60 2023-07-18T03:41:04.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.460+0000 7f34827fc700 1 -- 172.21.15.49:0/3069446583 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347400b9d0 con 0x7f34840b8a60 2023-07-18T03:41:04.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.460+0000 7f34827fc700 1 -- 172.21.15.49:0/3069446583 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347400bd60 con 0x7f34840b8a60 2023-07-18T03:41:04.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.460+0000 7f34895ff700 1 -- 172.21.15.49:0/3069446583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34840b8a60 msgr2=0x7f34840b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.460+0000 7f34895ff700 1 --2- 172.21.15.49:0/3069446583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34840b8a60 0x7f34840b3ff0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f348409c8e0 tx=0x7f3474009bd0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.461+0000 7f34895ff700 1 -- 172.21.15.49:0/3069446583 shutdown_connections 2023-07-18T03:41:04.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.461+0000 7f34895ff700 1 --2- 172.21.15.49:0/3069446583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34840b8a60 0x7f34840b3ff0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.461+0000 7f34895ff700 1 --2- 172.21.15.49:0/3069446583 >> v2:172.21.15.107:3300/0 conn(0x7f34840b4530 0x7f34840d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.461+0000 7f34895ff700 1 -- 172.21.15.49:0/3069446583 >> 172.21.15.49:0/3069446583 conn(0x7f348409c1b0 msgr2=0x7f34840d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:04.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.461+0000 7f34895ff700 1 -- 172.21.15.49:0/3069446583 shutdown_connections 2023-07-18T03:41:04.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.461+0000 7f34895ff700 1 -- 172.21.15.49:0/3069446583 wait complete. 2023-07-18T03:41:04.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.462+0000 7f34895ff700 1 Processor -- start 2023-07-18T03:41:04.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.463+0000 7f34895ff700 1 -- start start 2023-07-18T03:41:04.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f34895ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34840b4530 0x7f34840cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f34895ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34840b8a60 0x7f34841ef650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f34895ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34840d1050 0x7f34840ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f34895ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34840cd360 con 0x7f34840b8a60 2023-07-18T03:41:04.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f34895ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34840ce360 con 0x7f34840d1050 2023-07-18T03:41:04.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f34895ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34841efb90 con 0x7f34840b4530 2023-07-18T03:41:04.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f347bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34840b8a60 0x7f34841ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f347bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34840b8a60 0x7f34841ef650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58268/0 (socket says 172.21.15.49:58268) 2023-07-18T03:41:04.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f347bfff700 1 -- 172.21.15.49:0/1711433136 learned_addr learned my addr 172.21.15.49:0/1711433136 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:04.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f3482ffd700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34840b4530 0x7f34840cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f34837fe700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34840d1050 0x7f34840ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f347bfff700 1 -- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34840b4530 msgr2=0x7f34840cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f347bfff700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34840b4530 0x7f34840cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f347bfff700 1 -- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34840d1050 msgr2=0x7f34840ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f347bfff700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34840d1050 0x7f34840ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f347bfff700 1 -- 172.21.15.49:0/1711433136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3474009550 con 0x7f34840b8a60 2023-07-18T03:41:04.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.464+0000 7f34837fe700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34840d1050 0x7f34840ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:04.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.465+0000 7f347bfff700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34840b8a60 0x7f34841ef650 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f346c00e840 tx=0x7f346c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:04.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.465+0000 7f347b7fe700 1 -- 172.21.15.49:0/1711433136 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c013610 con 0x7f34840b8a60 2023-07-18T03:41:04.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.465+0000 7f347b7fe700 1 -- 172.21.15.49:0/1711433136 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346c013770 con 0x7f34840b8a60 2023-07-18T03:41:04.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.465+0000 7f34895ff700 1 -- 172.21.15.49:0/1711433136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34840ccda0 con 0x7f34840b8a60 2023-07-18T03:41:04.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.465+0000 7f347b7fe700 1 -- 172.21.15.49:0/1711433136 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c013b20 con 0x7f34840b8a60 2023-07-18T03:41:04.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.465+0000 7f34895ff700 1 -- 172.21.15.49:0/1711433136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34840cdff0 con 0x7f34840b8a60 2023-07-18T03:41:04.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.466+0000 7f34895ff700 1 -- 172.21.15.49:0/1711433136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34840cde50 con 0x7f34840b8a60 2023-07-18T03:41:04.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.467+0000 7f347b7fe700 1 -- 172.21.15.49:0/1711433136 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f346c013c80 con 0x7f34840b8a60 2023-07-18T03:41:04.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.468+0000 7f347b7fe700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f346406ae00 0x7f346406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.468+0000 7f347b7fe700 1 -- 172.21.15.49:0/1711433136 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f346c0893f0 con 0x7f34840b8a60 2023-07-18T03:41:04.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.469+0000 7f3482ffd700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f346406ae00 0x7f346406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.469+0000 7f3482ffd700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f346406ae00 0x7f346406d2b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3474000c00 tx=0x7f3474008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:04.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.472+0000 7f347b7fe700 1 -- 172.21.15.49:0/1711433136 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f34840cde50 con 0x7f34840b8a60 2023-07-18T03:41:04.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.680+0000 7f34895ff700 1 -- 172.21.15.49:0/1711433136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34840ce510 con 0x7f34840b8a60 2023-07-18T03:41:04.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.680+0000 7f347b7fe700 1 -- 172.21.15.49:0/1711433136 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34840ce510 con 0x7f34840b8a60 2023-07-18T03:41:04.674 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-07-18T03:41:04.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.683+0000 7f34895ff700 1 -- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f346406ae00 msgr2=0x7f346406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.683+0000 7f34895ff700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f346406ae00 0x7f346406d2b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3474000c00 tx=0x7f3474008030 comp rx=0 tx=0).stop 2023-07-18T03:41:04.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.683+0000 7f34895ff700 1 -- 172.21.15.49:0/1711433136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34840b8a60 msgr2=0x7f34841ef650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.683+0000 7f34895ff700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34840b8a60 0x7f34841ef650 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f346c00e840 tx=0x7f346c00eb50 comp rx=0 tx=0).stop 2023-07-18T03:41:04.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.683+0000 7f34895ff700 1 -- 172.21.15.49:0/1711433136 shutdown_connections 2023-07-18T03:41:04.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.683+0000 7f34895ff700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34840d1050 0x7f34840ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.683+0000 7f34895ff700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34840b4530 0x7f34840cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.683+0000 7f34895ff700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f346406ae00 0x7f346406d2b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.683+0000 7f34895ff700 1 --2- 172.21.15.49:0/1711433136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34840b8a60 0x7f34841ef650 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.683+0000 7f34895ff700 1 -- 172.21.15.49:0/1711433136 >> 172.21.15.49:0/1711433136 conn(0x7f348409c1b0 msgr2=0x7f34840b7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:04.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.683+0000 7f34895ff700 1 -- 172.21.15.49:0/1711433136 shutdown_connections 2023-07-18T03:41:04.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.683+0000 7f34895ff700 1 -- 172.21.15.49:0/1711433136 wait complete. 2023-07-18T03:41:04.695 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:41:04.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.909+0000 7f83febff700 1 Processor -- start 2023-07-18T03:41:04.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.910+0000 7f83febff700 1 -- start start 2023-07-18T03:41:04.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.911+0000 7f83febff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f83f815d5c0 0x7f83f80cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.911+0000 7f83febff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f83f80b50c0 0x7f83f80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.911+0000 7f83febff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f80cd4e0 con 0x7f83f8156cf0 2023-07-18T03:41:04.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.911+0000 7f83febff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f815da60 con 0x7f83f80b50c0 2023-07-18T03:41:04.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.911+0000 7f83febff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f80cd730 con 0x7f83f815d5c0 2023-07-18T03:41:04.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.911+0000 7f83fd19c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f83f80b50c0 0x7f83f80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.911+0000 7f83fc99b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f83f8156cf0 0x7f83f80d0a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50532/0 (socket says 172.21.15.49:50532) 2023-07-18T03:41:04.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.911+0000 7f83fc99b700 1 -- 172.21.15.49:0/843625133 learned_addr learned my addr 172.21.15.49:0/843625133 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:04.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.911+0000 7f83f7fff700 1 --2- 172.21.15.49:0/843625133 >> v2:172.21.15.107:3300/0 conn(0x7f83f815d5c0 0x7f83f80cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.912+0000 7f83fd19c700 1 -- 172.21.15.49:0/843625133 >> v2:172.21.15.107:3300/0 conn(0x7f83f815d5c0 msgr2=0x7f83f80cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:04.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.912+0000 7f83fd19c700 1 --2- 172.21.15.49:0/843625133 >> v2:172.21.15.107:3300/0 conn(0x7f83f815d5c0 0x7f83f80cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.912+0000 7f83fd19c700 1 -- 172.21.15.49:0/843625133 >> v1:172.21.15.49:6789/0 conn(0x7f83f8156cf0 legacy=0x7f83f80d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.912+0000 7f83fd19c700 1 -- 172.21.15.49:0/843625133 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f80cda10 con 0x7f83f80b50c0 2023-07-18T03:41:04.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.912+0000 7f83fd19c700 1 --2- 172.21.15.49:0/843625133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f80b50c0 0x7f83f80cfb70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f83f80d0410 tx=0x7f83f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd57aec89a06251 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:04.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.912+0000 7f83f77fe700 1 -- 172.21.15.49:0/843625133 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f000e070 con 0x7f83f80b50c0 2023-07-18T03:41:04.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.912+0000 7f83f77fe700 1 -- 172.21.15.49:0/843625133 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f000bbd0 con 0x7f83f80b50c0 2023-07-18T03:41:04.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.912+0000 7f83f77fe700 1 -- 172.21.15.49:0/843625133 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f0012430 con 0x7f83f80b50c0 2023-07-18T03:41:04.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.913+0000 7f83febff700 1 -- 172.21.15.49:0/843625133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f80b50c0 msgr2=0x7f83f80cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.913+0000 7f83febff700 1 --2- 172.21.15.49:0/843625133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f80b50c0 0x7f83f80cfb70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f83f80d0410 tx=0x7f83f000b330 comp rx=0 tx=0).stop 2023-07-18T03:41:04.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.913+0000 7f83febff700 1 -- 172.21.15.49:0/843625133 shutdown_connections 2023-07-18T03:41:04.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.913+0000 7f83febff700 1 --2- 172.21.15.49:0/843625133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f80b50c0 0x7f83f80cfb70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.913+0000 7f83febff700 1 --2- 172.21.15.49:0/843625133 >> v2:172.21.15.107:3300/0 conn(0x7f83f815d5c0 0x7f83f80cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.913+0000 7f83febff700 1 -- 172.21.15.49:0/843625133 >> 172.21.15.49:0/843625133 conn(0x7f83f806a6c0 msgr2=0x7f83f806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:04.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.913+0000 7f83febff700 1 -- 172.21.15.49:0/843625133 shutdown_connections 2023-07-18T03:41:04.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.913+0000 7f83febff700 1 -- 172.21.15.49:0/843625133 wait complete. 2023-07-18T03:41:04.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.915+0000 7f83febff700 1 Processor -- start 2023-07-18T03:41:04.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.915+0000 7f83febff700 1 -- start start 2023-07-18T03:41:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.916+0000 7f83febff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83f80b50c0 0x7f83f80b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.916+0000 7f83febff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f8156cf0 0x7f83f81552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.916+0000 7f83febff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f815d5c0 0x7f83f80b8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.916+0000 7f83febff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80c70b0 con 0x7f83f8156cf0 2023-07-18T03:41:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.916+0000 7f83febff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80b6c10 con 0x7f83f80b50c0 2023-07-18T03:41:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.916+0000 7f83febff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80b7470 con 0x7f83f815d5c0 2023-07-18T03:41:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.916+0000 7f83f7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f8156cf0 0x7f83f81552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.916+0000 7f83f7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f8156cf0 0x7f83f81552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58282/0 (socket says 172.21.15.49:58282) 2023-07-18T03:41:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.916+0000 7f83f7fff700 1 -- 172.21.15.49:0/1405025862 learned_addr learned my addr 172.21.15.49:0/1405025862 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.916+0000 7f83fc99b700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83f80b50c0 0x7f83f80b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.916+0000 7f83fd19c700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f815d5c0 0x7f83f80b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.917+0000 7f83f7fff700 1 -- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f815d5c0 msgr2=0x7f83f80b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.917+0000 7f83f7fff700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f815d5c0 0x7f83f80b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.917+0000 7f83f7fff700 1 -- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83f80b50c0 msgr2=0x7f83f80b66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:04.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.917+0000 7f83f7fff700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83f80b50c0 0x7f83f80b66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:04.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.917+0000 7f83f7fff700 1 -- 172.21.15.49:0/1405025862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f0009a70 con 0x7f83f8156cf0 2023-07-18T03:41:04.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.917+0000 7f83f7fff700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f8156cf0 0x7f83f81552c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f83e400d800 tx=0x7f83e400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:04.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.917+0000 7f83f57fa700 1 -- 172.21.15.49:0/1405025862 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40125f0 con 0x7f83f8156cf0 2023-07-18T03:41:04.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.917+0000 7f83f57fa700 1 -- 172.21.15.49:0/1405025862 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e4012750 con 0x7f83f8156cf0 2023-07-18T03:41:04.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.917+0000 7f83febff700 1 -- 172.21.15.49:0/1405025862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f80b6eb0 con 0x7f83f8156cf0 2023-07-18T03:41:04.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.917+0000 7f83f57fa700 1 -- 172.21.15.49:0/1405025862 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4012b00 con 0x7f83f8156cf0 2023-07-18T03:41:04.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.917+0000 7f83febff700 1 -- 172.21.15.49:0/1405025862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f80c6d40 con 0x7f83f8156cf0 2023-07-18T03:41:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.919+0000 7f83febff700 1 -- 172.21.15.49:0/1405025862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f80ce760 con 0x7f83f8156cf0 2023-07-18T03:41:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.920+0000 7f83f57fa700 1 -- 172.21.15.49:0/1405025862 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f83e4012c60 con 0x7f83f8156cf0 2023-07-18T03:41:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.921+0000 7f83f57fa700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f83dc06ae00 0x7f83dc06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.921+0000 7f83f57fa700 1 -- 172.21.15.49:0/1405025862 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f83e40896d0 con 0x7f83f8156cf0 2023-07-18T03:41:04.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.921+0000 7f83fc99b700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f83dc06ae00 0x7f83dc06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:04.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.922+0000 7f83fc99b700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f83dc06ae00 0x7f83dc06d2b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f83ec005910 tx=0x7f83ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:04.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:04.925+0000 7f83f57fa700 1 -- 172.21.15.49:0/1405025862 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f83e4054630 con 0x7f83f8156cf0 2023-07-18T03:41:05.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.136+0000 7f83febff700 1 -- 172.21.15.49:0/1405025862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83f80d1b30 con 0x7f83f8156cf0 2023-07-18T03:41:05.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.136+0000 7f83f57fa700 1 -- 172.21.15.49:0/1405025862 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f83e4057c50 con 0x7f83f8156cf0 2023-07-18T03:41:05.131 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-07-18T03:41:05.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.139+0000 7f83febff700 1 -- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f83dc06ae00 msgr2=0x7f83dc06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.139+0000 7f83febff700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f83dc06ae00 0x7f83dc06d2b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f83ec005910 tx=0x7f83ec009040 comp rx=0 tx=0).stop 2023-07-18T03:41:05.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.139+0000 7f83febff700 1 -- 172.21.15.49:0/1405025862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f8156cf0 msgr2=0x7f83f81552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.139+0000 7f83febff700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f8156cf0 0x7f83f81552c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f83e400d800 tx=0x7f83e400db10 comp rx=0 tx=0).stop 2023-07-18T03:41:05.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.139+0000 7f83febff700 1 -- 172.21.15.49:0/1405025862 shutdown_connections 2023-07-18T03:41:05.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.139+0000 7f83febff700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f815d5c0 0x7f83f80b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.139+0000 7f83febff700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83f80b50c0 0x7f83f80b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.139+0000 7f83febff700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f83dc06ae00 0x7f83dc06d2b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.139+0000 7f83febff700 1 --2- 172.21.15.49:0/1405025862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83f8156cf0 0x7f83f81552c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.139+0000 7f83febff700 1 -- 172.21.15.49:0/1405025862 >> 172.21.15.49:0/1405025862 conn(0x7f83f806a6c0 msgr2=0x7f83f80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:05.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.139+0000 7f83febff700 1 -- 172.21.15.49:0/1405025862 shutdown_connections 2023-07-18T03:41:05.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.139+0000 7f83febff700 1 -- 172.21.15.49:0/1405025862 wait complete. 2023-07-18T03:41:05.151 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:41:05.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.364+0000 7f14885ff700 1 Processor -- start 2023-07-18T03:41:05.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.365+0000 7f14885ff700 1 -- start start 2023-07-18T03:41:05.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.366+0000 7f14885ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14800ce030 0x7f1480069f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:05.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.366+0000 7f14885ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f148006a470 0x7f14801601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:05.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.366+0000 7f14885ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14800cdba0 con 0x7f148006f010 2023-07-18T03:41:05.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.366+0000 7f14885ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148006f3e0 con 0x7f148006a470 2023-07-18T03:41:05.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.366+0000 7f14885ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148006a8f0 con 0x7f14800ce030 2023-07-18T03:41:05.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.366+0000 7f1485b9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14800ce030 0x7f1480069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:05.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.366+0000 7f1486b9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f148006a470 0x7f14801601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:05.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.367+0000 7f1485b9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14800ce030 0x7f1480069f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:58536/0 (socket says 172.21.15.49:58536) 2023-07-18T03:41:05.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.367+0000 7f1485b9a700 1 -- 172.21.15.49:0/1957111943 learned_addr learned my addr 172.21.15.49:0/1957111943 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:05.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.367+0000 7f1485b9a700 1 -- 172.21.15.49:0/1957111943 >> v2:172.21.15.107:3300/0 conn(0x7f148006a470 msgr2=0x7f14801601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:05.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.367+0000 7f1485b9a700 1 --2- 172.21.15.49:0/1957111943 >> v2:172.21.15.107:3300/0 conn(0x7f148006a470 0x7f14801601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.367+0000 7f1485b9a700 1 -- 172.21.15.49:0/1957111943 >> v1:172.21.15.107:6789/0 conn(0x7f148006f010 legacy=0x7f1480071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.367+0000 7f1485b9a700 1 -- 172.21.15.49:0/1957111943 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14800cf370 con 0x7f14800ce030 2023-07-18T03:41:05.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.367+0000 7f1486b9c700 1 --2- 172.21.15.49:0/1957111943 >> v2:172.21.15.107:3300/0 conn(0x7f148006a470 0x7f14801601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:05.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.367+0000 7f1485b9a700 1 --2- 172.21.15.49:0/1957111943 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14800ce030 0x7f1480069f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14700098f0 tx=0x7f1470009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adbefd5ef4df99df server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:05.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.368+0000 7f1485399700 1 -- 172.21.15.49:0/1957111943 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470011830 con 0x7f14800ce030 2023-07-18T03:41:05.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.368+0000 7f1485399700 1 -- 172.21.15.49:0/1957111943 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1470011990 con 0x7f14800ce030 2023-07-18T03:41:05.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.368+0000 7f1485399700 1 -- 172.21.15.49:0/1957111943 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470011d20 con 0x7f14800ce030 2023-07-18T03:41:05.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.368+0000 7f14885ff700 1 -- 172.21.15.49:0/1957111943 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14800ce030 msgr2=0x7f1480069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.368+0000 7f14885ff700 1 --2- 172.21.15.49:0/1957111943 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14800ce030 0x7f1480069f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14700098f0 tx=0x7f1470009c00 comp rx=0 tx=0).stop 2023-07-18T03:41:05.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.368+0000 7f14885ff700 1 -- 172.21.15.49:0/1957111943 shutdown_connections 2023-07-18T03:41:05.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.368+0000 7f14885ff700 1 --2- 172.21.15.49:0/1957111943 >> v2:172.21.15.107:3300/0 conn(0x7f148006a470 0x7f14801601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.368+0000 7f14885ff700 1 --2- 172.21.15.49:0/1957111943 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14800ce030 0x7f1480069f30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.368+0000 7f14885ff700 1 -- 172.21.15.49:0/1957111943 >> 172.21.15.49:0/1957111943 conn(0x7f14800cc640 msgr2=0x7f14800cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:05.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.368+0000 7f14885ff700 1 -- 172.21.15.49:0/1957111943 shutdown_connections 2023-07-18T03:41:05.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.368+0000 7f14885ff700 1 -- 172.21.15.49:0/1957111943 wait complete. 2023-07-18T03:41:05.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.370+0000 7f14885ff700 1 Processor -- start 2023-07-18T03:41:05.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.370+0000 7f14885ff700 1 -- start start 2023-07-18T03:41:05.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.371+0000 7f14885ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f148006a470 0x7f148015eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:05.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.371+0000 7f14885ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148006f010 0x7f14801f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:05.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.371+0000 7f14885ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14800ce030 0x7f14801f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:05.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.371+0000 7f14885ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148015e910 con 0x7f148006a470 2023-07-18T03:41:05.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.371+0000 7f14885ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1480146870 con 0x7f148006f010 2023-07-18T03:41:05.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.371+0000 7f14885ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148015da20 con 0x7f14800ce030 2023-07-18T03:41:05.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.371+0000 7f148639b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f148006a470 0x7f148015eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:05.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.372+0000 7f1486b9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14800ce030 0x7f14801f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:05.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.372+0000 7f148639b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f148006a470 0x7f148015eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58296/0 (socket says 172.21.15.49:58296) 2023-07-18T03:41:05.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.372+0000 7f1485b9a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148006f010 0x7f14801f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:05.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.372+0000 7f148639b700 1 -- 172.21.15.49:0/1245907926 learned_addr learned my addr 172.21.15.49:0/1245907926 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:05.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.372+0000 7f148639b700 1 -- 172.21.15.49:0/1245907926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14800ce030 msgr2=0x7f14801f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.372+0000 7f148639b700 1 --2- 172.21.15.49:0/1245907926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14800ce030 0x7f14801f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.372+0000 7f148639b700 1 -- 172.21.15.49:0/1245907926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148006f010 msgr2=0x7f14801f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.372+0000 7f148639b700 1 --2- 172.21.15.49:0/1245907926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148006f010 0x7f14801f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.372+0000 7f148639b700 1 -- 172.21.15.49:0/1245907926 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1478009a70 con 0x7f148006a470 2023-07-18T03:41:05.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.372+0000 7f148639b700 1 --2- 172.21.15.49:0/1245907926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f148006a470 0x7f148015eda0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f147c00c4e0 tx=0x7f147c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:05.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.372+0000 7f1476ffd700 1 -- 172.21.15.49:0/1245907926 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c0052d0 con 0x7f148006a470 2023-07-18T03:41:05.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.373+0000 7f1476ffd700 1 -- 172.21.15.49:0/1245907926 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c005430 con 0x7f148006a470 2023-07-18T03:41:05.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.373+0000 7f1476ffd700 1 -- 172.21.15.49:0/1245907926 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c0038c0 con 0x7f148006a470 2023-07-18T03:41:05.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.373+0000 7f14885ff700 1 -- 172.21.15.49:0/1245907926 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1470009550 con 0x7f148006a470 2023-07-18T03:41:05.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.373+0000 7f14885ff700 1 -- 172.21.15.49:0/1245907926 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148015dfe0 con 0x7f148006a470 2023-07-18T03:41:05.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.374+0000 7f14885ff700 1 -- 172.21.15.49:0/1245907926 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1480074350 con 0x7f148006a470 2023-07-18T03:41:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.375+0000 7f1476ffd700 1 -- 172.21.15.49:0/1245907926 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f147c003a20 con 0x7f148006a470 2023-07-18T03:41:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.376+0000 7f1476ffd700 1 --2- 172.21.15.49:0/1245907926 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f146406ae00 0x7f146406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.376+0000 7f1476ffd700 1 -- 172.21.15.49:0/1245907926 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f147c088a80 con 0x7f148006a470 2023-07-18T03:41:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.376+0000 7f1485b9a700 1 --2- 172.21.15.49:0/1245907926 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f146406ae00 0x7f146406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.377+0000 7f1485b9a700 1 --2- 172.21.15.49:0/1245907926 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f146406ae00 0x7f146406d2b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1470000c00 tx=0x7f1470008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.380+0000 7f1476ffd700 1 -- 172.21.15.49:0/1245907926 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f147c053940 con 0x7f148006a470 2023-07-18T03:41:05.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.585+0000 7f14885ff700 1 -- 172.21.15.49:0/1245907926 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14800710e0 con 0x7f148006a470 2023-07-18T03:41:05.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.585+0000 7f1476ffd700 1 -- 172.21.15.49:0/1245907926 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f147c017020 con 0x7f148006a470 2023-07-18T03:41:05.628 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-07-18T03:41:05.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.588+0000 7f14885ff700 1 -- 172.21.15.49:0/1245907926 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f146406ae00 msgr2=0x7f146406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.588+0000 7f14885ff700 1 --2- 172.21.15.49:0/1245907926 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f146406ae00 0x7f146406d2b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1470000c00 tx=0x7f1470008030 comp rx=0 tx=0).stop 2023-07-18T03:41:05.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.588+0000 7f14885ff700 1 -- 172.21.15.49:0/1245907926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f148006a470 msgr2=0x7f148015eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.588+0000 7f14885ff700 1 --2- 172.21.15.49:0/1245907926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f148006a470 0x7f148015eda0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f147c00c4e0 tx=0x7f147c00c7f0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.588+0000 7f14885ff700 1 -- 172.21.15.49:0/1245907926 shutdown_connections 2023-07-18T03:41:05.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.588+0000 7f14885ff700 1 --2- 172.21.15.49:0/1245907926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14800ce030 0x7f14801f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.588+0000 7f14885ff700 1 --2- 172.21.15.49:0/1245907926 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f146406ae00 0x7f146406d2b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.588+0000 7f14885ff700 1 --2- 172.21.15.49:0/1245907926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f148006a470 0x7f148015eda0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.588+0000 7f14885ff700 1 --2- 172.21.15.49:0/1245907926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f148006f010 0x7f14801f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.588+0000 7f14885ff700 1 -- 172.21.15.49:0/1245907926 >> 172.21.15.49:0/1245907926 conn(0x7f14800cc640 msgr2=0x7f14800addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:05.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.588+0000 7f14885ff700 1 -- 172.21.15.49:0/1245907926 shutdown_connections 2023-07-18T03:41:05.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.588+0000 7f14885ff700 1 -- 172.21.15.49:0/1245907926 wait complete. 2023-07-18T03:41:05.632 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:41:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.841+0000 7fcece1ff700 1 Processor -- start 2023-07-18T03:41:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.842+0000 7fcece1ff700 1 -- start start 2023-07-18T03:41:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.843+0000 7fcece1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcec80d1220 0x7fcec80b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.843+0000 7fcece1ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec80d2ce0 con 0x7fcec815c950 2023-07-18T03:41:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.843+0000 7fcece1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec80b3350 con 0x7fcec80b1e00 2023-07-18T03:41:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.843+0000 7fcece1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec80b3630 con 0x7fcec80d1220 2023-07-18T03:41:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.843+0000 7fcec77fe700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fcec815c950 0x7fcec80d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50548/0 (socket says 172.21.15.49:50548) 2023-07-18T03:41:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.843+0000 7fcec77fe700 1 -- 172.21.15.49:0/3542643691 learned_addr learned my addr 172.21.15.49:0/3542643691 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.843+0000 7fcebffff700 1 --2- 172.21.15.49:0/3542643691 >> v2:172.21.15.107:3300/0 conn(0x7fcec80d1220 0x7fcec80b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:05.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.844+0000 7fcec6ffd700 1 -- 172.21.15.49:0/3542643691 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123168482 0 0) 0x7fcec80d2ce0 con 0x7fcec815c950 2023-07-18T03:41:05.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.844+0000 7fcec6ffd700 1 -- 172.21.15.49:0/3542643691 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fceb40035b0 con 0x7fcec815c950 2023-07-18T03:41:05.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.844+0000 7fcebffff700 1 -- 172.21.15.49:0/3542643691 >> v1:172.21.15.107:6789/0 conn(0x7fcec80b1e00 legacy=0x7fcec80afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.844+0000 7fcebffff700 1 -- 172.21.15.49:0/3542643691 >> v1:172.21.15.49:6789/0 conn(0x7fcec815c950 legacy=0x7fcec80d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.844+0000 7fcebffff700 1 -- 172.21.15.49:0/3542643691 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec80b3910 con 0x7fcec80d1220 2023-07-18T03:41:05.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.844+0000 7fcebffff700 1 --2- 172.21.15.49:0/3542643691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec80d1220 0x7fcec80b43f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009840 tx=0x7fceb0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5210c6cf4e970142 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.845+0000 7fcec6ffd700 1 -- 172.21.15.49:0/3542643691 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0011830 con 0x7fcec80d1220 2023-07-18T03:41:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.845+0000 7fcec6ffd700 1 -- 172.21.15.49:0/3542643691 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb0011990 con 0x7fcec80d1220 2023-07-18T03:41:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.845+0000 7fcec6ffd700 1 -- 172.21.15.49:0/3542643691 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0011d20 con 0x7fcec80d1220 2023-07-18T03:41:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.845+0000 7fcece1ff700 1 -- 172.21.15.49:0/3542643691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec80d1220 msgr2=0x7fcec80b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.845+0000 7fcece1ff700 1 --2- 172.21.15.49:0/3542643691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec80d1220 0x7fcec80b43f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009840 tx=0x7fceb0009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:05.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.845+0000 7fcece1ff700 1 -- 172.21.15.49:0/3542643691 shutdown_connections 2023-07-18T03:41:05.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.845+0000 7fcece1ff700 1 --2- 172.21.15.49:0/3542643691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec80d1220 0x7fcec80b43f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.845+0000 7fcece1ff700 1 -- 172.21.15.49:0/3542643691 >> 172.21.15.49:0/3542643691 conn(0x7fcec809c1d0 msgr2=0x7fcec80d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:05.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.845+0000 7fcece1ff700 1 -- 172.21.15.49:0/3542643691 shutdown_connections 2023-07-18T03:41:05.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.845+0000 7fcece1ff700 1 -- 172.21.15.49:0/3542643691 wait complete. 2023-07-18T03:41:05.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.847+0000 7fcece1ff700 1 Processor -- start 2023-07-18T03:41:05.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.847+0000 7fcece1ff700 1 -- start start 2023-07-18T03:41:05.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.848+0000 7fcece1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcec80b1e00 0x7fcec80c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:05.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.848+0000 7fcece1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec80d1220 0x7fcec80cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:05.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.848+0000 7fcece1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec815c950 0x7fcec806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:05.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.848+0000 7fcece1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec814fce0 con 0x7fcec80d1220 2023-07-18T03:41:05.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.848+0000 7fcece1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80c9780 con 0x7fcec815c950 2023-07-18T03:41:05.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.848+0000 7fcece1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80d0360 con 0x7fcec80b1e00 2023-07-18T03:41:05.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.848+0000 7fcebffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec80d1220 0x7fcec80cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:05.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.848+0000 7fcebffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec80d1220 0x7fcec80cfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58306/0 (socket says 172.21.15.49:58306) 2023-07-18T03:41:05.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.848+0000 7fcebffff700 1 -- 172.21.15.49:0/2777642910 learned_addr learned my addr 172.21.15.49:0/2777642910 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:05.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.848+0000 7fcec7fff700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec815c950 0x7fcec806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:05.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.848+0000 7fcec77fe700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcec80b1e00 0x7fcec80c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:05.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.849+0000 7fcebffff700 1 -- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcec80b1e00 msgr2=0x7fcec80c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.849+0000 7fcebffff700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcec80b1e00 0x7fcec80c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.849+0000 7fcebffff700 1 -- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec815c950 msgr2=0x7fcec806b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:05.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.849+0000 7fcebffff700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec815c950 0x7fcec806b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:05.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.849+0000 7fcebffff700 1 -- 172.21.15.49:0/2777642910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb00094a0 con 0x7fcec80d1220 2023-07-18T03:41:05.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.849+0000 7fcebffff700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec80d1220 0x7fcec80cfe20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009fd0 tx=0x7fceb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:05.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.849+0000 7fcec4ff9700 1 -- 172.21.15.49:0/2777642910 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb00036a0 con 0x7fcec80d1220 2023-07-18T03:41:05.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.849+0000 7fcec4ff9700 1 -- 172.21.15.49:0/2777642910 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb001f070 con 0x7fcec80d1220 2023-07-18T03:41:05.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.849+0000 7fcec4ff9700 1 -- 172.21.15.49:0/2777642910 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0003a40 con 0x7fcec80d1220 2023-07-18T03:41:05.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.849+0000 7fcece1ff700 1 -- 172.21.15.49:0/2777642910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec80c6c10 con 0x7fcec80d1220 2023-07-18T03:41:05.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.850+0000 7fcece1ff700 1 -- 172.21.15.49:0/2777642910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec80b5e10 con 0x7fcec80d1220 2023-07-18T03:41:05.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.851+0000 7fcece1ff700 1 -- 172.21.15.49:0/2777642910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec806a3e0 con 0x7fcec80d1220 2023-07-18T03:41:05.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.851+0000 7fcec4ff9700 1 -- 172.21.15.49:0/2777642910 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fceb0003ba0 con 0x7fcec80d1220 2023-07-18T03:41:05.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.852+0000 7fcec4ff9700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcea806ae00 0x7fcea806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:05.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.853+0000 7fcec4ff9700 1 -- 172.21.15.49:0/2777642910 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fceb0021070 con 0x7fcec80d1220 2023-07-18T03:41:05.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.853+0000 7fcec77fe700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcea806ae00 0x7fcea806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:05.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.854+0000 7fcec77fe700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcea806ae00 0x7fcea806d2b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fceb80010e0 tx=0x7fceb8007c70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:05.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:05.857+0000 7fcec4ff9700 1 -- 172.21.15.49:0/2777642910 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fceb0055e20 con 0x7fcec80d1220 2023-07-18T03:41:06.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.112+0000 7fcece1ff700 1 -- 172.21.15.49:0/2777642910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcec804e280 con 0x7fcec80d1220 2023-07-18T03:41:06.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.113+0000 7fcec4ff9700 1 -- 172.21.15.49:0/2777642910 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fceb0059440 con 0x7fcec80d1220 2023-07-18T03:41:06.107 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:06.107 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/777524677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":777524677},{"type":"v1","addr":"172.21.15.107:6837","nonce":777524677}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/2382452558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2382452558},{"type":"v1","addr":"172.21.15.49:6834","nonce":2382452558}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/2843143219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2843143219},{"type":"v1","addr":"172.21.15.107:6836","nonce":2843143219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:40:40.451705+0000","modified":"2023-07-18T03:40:43.957663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5479":{"gid":5479,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1453078087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":1453078087},{"type":"v1","addr":"172.21.15.49:6835","nonce":1453078087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-18T03:41:06.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.116+0000 7fcece1ff700 1 -- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcea806ae00 msgr2=0x7fcea806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.116+0000 7fcece1ff700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcea806ae00 0x7fcea806d2b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fceb80010e0 tx=0x7fceb8007c70 comp rx=0 tx=0).stop 2023-07-18T03:41:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.116+0000 7fcece1ff700 1 -- 172.21.15.49:0/2777642910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec80d1220 msgr2=0x7fcec80cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.116+0000 7fcece1ff700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec80d1220 0x7fcec80cfe20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009fd0 tx=0x7fceb0004750 comp rx=0 tx=0).stop 2023-07-18T03:41:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.116+0000 7fcece1ff700 1 -- 172.21.15.49:0/2777642910 shutdown_connections 2023-07-18T03:41:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.116+0000 7fcece1ff700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec815c950 0x7fcec806b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.116+0000 7fcece1ff700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcec80b1e00 0x7fcec80c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.116+0000 7fcece1ff700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fcea806ae00 0x7fcea806d2b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.116+0000 7fcece1ff700 1 --2- 172.21.15.49:0/2777642910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec80d1220 0x7fcec80cfe20 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.116+0000 7fcece1ff700 1 -- 172.21.15.49:0/2777642910 >> 172.21.15.49:0/2777642910 conn(0x7fcec809c1d0 msgr2=0x7fcec80b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.116+0000 7fcece1ff700 1 -- 172.21.15.49:0/2777642910 shutdown_connections 2023-07-18T03:41:06.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.116+0000 7fcece1ff700 1 -- 172.21.15.49:0/2777642910 wait complete. 2023-07-18T03:41:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 29 2023-07-18T03:41:06.128 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:41:06.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.339+0000 7fbdda5ff700 1 Processor -- start 2023-07-18T03:41:06.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.340+0000 7fbdda5ff700 1 -- start start 2023-07-18T03:41:06.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.341+0000 7fbdda5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd415c710 0x7fbdd40cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:06.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.341+0000 7fbdda5ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbdd406db30 0x7fbdd406a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:06.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.341+0000 7fbdda5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd40cd320 con 0x7fbdd4069f30 2023-07-18T03:41:06.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.341+0000 7fbdda5ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd40ce750 con 0x7fbdd406db30 2023-07-18T03:41:06.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.341+0000 7fbdda5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40cea30 con 0x7fbdd415c710 2023-07-18T03:41:06.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.341+0000 7fbdd8b9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbdd406db30 0x7fbdd406a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:06.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.341+0000 7fbdd8b9c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbdd406db30 0x7fbdd406a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58318/0 (socket says 172.21.15.49:58318) 2023-07-18T03:41:06.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.341+0000 7fbdd3fff700 1 --2- 172.21.15.49:0/667346773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd415c710 0x7fbdd40cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:06.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.341+0000 7fbdd8b9c700 1 -- 172.21.15.49:0/667346773 learned_addr learned my addr 172.21.15.49:0/667346773 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:06.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.342+0000 7fbdd8b9c700 1 -- 172.21.15.49:0/667346773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd415c710 msgr2=0x7fbdd40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:06.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.342+0000 7fbdd8b9c700 1 --2- 172.21.15.49:0/667346773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd415c710 0x7fbdd40cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.342+0000 7fbdd8b9c700 1 -- 172.21.15.49:0/667346773 >> v1:172.21.15.107:6789/0 conn(0x7fbdd4069f30 legacy=0x7fbdd406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:06.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.342+0000 7fbdd8b9c700 1 -- 172.21.15.49:0/667346773 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd40ced10 con 0x7fbdd406db30 2023-07-18T03:41:06.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.342+0000 7fbdd8b9c700 1 --2- 172.21.15.49:0/667346773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdd406db30 0x7fbdd406a3a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009e50 tx=0x7fbdcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e09762c26355f3d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:06.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.342+0000 7fbdd2ffd700 1 -- 172.21.15.49:0/667346773 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00e070 con 0x7fbdd406db30 2023-07-18T03:41:06.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.342+0000 7fbdd2ffd700 1 -- 172.21.15.49:0/667346773 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc00b910 con 0x7fbdd406db30 2023-07-18T03:41:06.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.342+0000 7fbdd2ffd700 1 -- 172.21.15.49:0/667346773 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00bcc0 con 0x7fbdd406db30 2023-07-18T03:41:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.343+0000 7fbdda5ff700 1 -- 172.21.15.49:0/667346773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdd406db30 msgr2=0x7fbdd406a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.343+0000 7fbdda5ff700 1 --2- 172.21.15.49:0/667346773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdd406db30 0x7fbdd406a3a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009e50 tx=0x7fbdcc00b330 comp rx=0 tx=0).stop 2023-07-18T03:41:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.343+0000 7fbdda5ff700 1 -- 172.21.15.49:0/667346773 shutdown_connections 2023-07-18T03:41:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.343+0000 7fbdda5ff700 1 --2- 172.21.15.49:0/667346773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdd406db30 0x7fbdd406a3a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.343+0000 7fbdda5ff700 1 --2- 172.21.15.49:0/667346773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd415c710 0x7fbdd40cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.343+0000 7fbdda5ff700 1 -- 172.21.15.49:0/667346773 >> 172.21.15.49:0/667346773 conn(0x7fbdd40b16c0 msgr2=0x7fbdd40b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.343+0000 7fbdda5ff700 1 -- 172.21.15.49:0/667346773 shutdown_connections 2023-07-18T03:41:06.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.343+0000 7fbdda5ff700 1 -- 172.21.15.49:0/667346773 wait complete. 2023-07-18T03:41:06.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.345+0000 7fbdda5ff700 1 Processor -- start 2023-07-18T03:41:06.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.345+0000 7fbdda5ff700 1 -- start start 2023-07-18T03:41:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.346+0000 7fbdda5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd4069f30 0x7fbdd4159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.346+0000 7fbdda5ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdd406db30 0x7fbdd415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.346+0000 7fbdda5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd415ef80 0x7fbdd40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.346+0000 7fbdda5ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd4159150 con 0x7fbdd406db30 2023-07-18T03:41:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.346+0000 7fbdda5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd41584d0 con 0x7fbdd415ef80 2023-07-18T03:41:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.346+0000 7fbdda5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd41587b0 con 0x7fbdd4069f30 2023-07-18T03:41:06.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.346+0000 7fbdd37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdd406db30 0x7fbdd415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:06.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.346+0000 7fbdd37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdd406db30 0x7fbdd415ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58328/0 (socket says 172.21.15.49:58328) 2023-07-18T03:41:06.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.346+0000 7fbdd37fe700 1 -- 172.21.15.49:0/4021007402 learned_addr learned my addr 172.21.15.49:0/4021007402 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:06.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.346+0000 7fbdd3fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd4069f30 0x7fbdd4159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:06.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.346+0000 7fbdd8b9c700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd415ef80 0x7fbdd40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:06.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.347+0000 7fbdd37fe700 1 -- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd4069f30 msgr2=0x7fbdd4159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:06.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.347+0000 7fbdd37fe700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd4069f30 0x7fbdd4159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.347+0000 7fbdd37fe700 1 -- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd415ef80 msgr2=0x7fbdd40b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:06.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.347+0000 7fbdd37fe700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd415ef80 0x7fbdd40b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.347+0000 7fbdd37fe700 1 -- 172.21.15.49:0/4021007402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc009b30 con 0x7fbdd406db30 2023-07-18T03:41:06.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.347+0000 7fbdd37fe700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdd406db30 0x7fbdd415ea40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c4e0 tx=0x7fbdc800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:06.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.347+0000 7fbdd0ff9700 1 -- 172.21.15.49:0/4021007402 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80052d0 con 0x7fbdd406db30 2023-07-18T03:41:06.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.347+0000 7fbdd0ff9700 1 -- 172.21.15.49:0/4021007402 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc8005430 con 0x7fbdd406db30 2023-07-18T03:41:06.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.347+0000 7fbdd0ff9700 1 -- 172.21.15.49:0/4021007402 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80038c0 con 0x7fbdd406db30 2023-07-18T03:41:06.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.347+0000 7fbdda5ff700 1 -- 172.21.15.49:0/4021007402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd406ac20 con 0x7fbdd406db30 2023-07-18T03:41:06.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.348+0000 7fbdda5ff700 1 -- 172.21.15.49:0/4021007402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd4158d70 con 0x7fbdd406db30 2023-07-18T03:41:06.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.348+0000 7fbdda5ff700 1 -- 172.21.15.49:0/4021007402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd415f380 con 0x7fbdd406db30 2023-07-18T03:41:06.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.350+0000 7fbdd0ff9700 1 -- 172.21.15.49:0/4021007402 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fbdc8003a20 con 0x7fbdd406db30 2023-07-18T03:41:06.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.351+0000 7fbdd0ff9700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbdc006ae00 0x7fbdc006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:06.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.351+0000 7fbdd0ff9700 1 -- 172.21.15.49:0/4021007402 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fbdc8088a40 con 0x7fbdd406db30 2023-07-18T03:41:06.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.351+0000 7fbdd3fff700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbdc006ae00 0x7fbdc006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:06.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.352+0000 7fbdd3fff700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbdc006ae00 0x7fbdc006d2b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009b40 tx=0x7fbdc4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:06.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.355+0000 7fbdd0ff9700 1 -- 172.21.15.49:0/4021007402 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbdd415f380 con 0x7fbdd406db30 2023-07-18T03:41:06.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.569+0000 7fbdda5ff700 1 -- 172.21.15.49:0/4021007402 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdd415f380 con 0x7fbdd406db30 2023-07-18T03:41:06.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.571+0000 7fbdd0ff9700 1 -- 172.21.15.49:0/4021007402 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18365 (secure 0 0 0) 0x7fbdd415f380 con 0x7fbdd406db30 2023-07-18T03:41:06.565 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:06.565 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":50,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:41:00.954181+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:40:38.792808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-18T03:40:39.438590+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":8,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:41:06.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.573+0000 7fbdda5ff700 1 -- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbdc006ae00 msgr2=0x7fbdc006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:06.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.573+0000 7fbdda5ff700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbdc006ae00 0x7fbdc006d2b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009b40 tx=0x7fbdc4009210 comp rx=0 tx=0).stop 2023-07-18T03:41:06.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.573+0000 7fbdda5ff700 1 -- 172.21.15.49:0/4021007402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdd406db30 msgr2=0x7fbdd415ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:06.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.573+0000 7fbdda5ff700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdd406db30 0x7fbdd415ea40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c4e0 tx=0x7fbdc800c7f0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.573+0000 7fbdda5ff700 1 -- 172.21.15.49:0/4021007402 shutdown_connections 2023-07-18T03:41:06.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.573+0000 7fbdda5ff700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd415ef80 0x7fbdd40b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.573+0000 7fbdda5ff700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd4069f30 0x7fbdd4159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.573+0000 7fbdda5ff700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fbdc006ae00 0x7fbdc006d2b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.573+0000 7fbdda5ff700 1 --2- 172.21.15.49:0/4021007402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdd406db30 0x7fbdd415ea40 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.573+0000 7fbdda5ff700 1 -- 172.21.15.49:0/4021007402 >> 172.21.15.49:0/4021007402 conn(0x7fbdd40b16c0 msgr2=0x7fbdd4070850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:06.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.573+0000 7fbdda5ff700 1 -- 172.21.15.49:0/4021007402 shutdown_connections 2023-07-18T03:41:06.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.573+0000 7fbdda5ff700 1 -- 172.21.15.49:0/4021007402 wait complete. 2023-07-18T03:41:06.586 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-18T03:41:06.587 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:41:06.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.798+0000 7f86257ff700 1 Processor -- start 2023-07-18T03:41:06.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.798+0000 7f86257ff700 1 -- start start 2023-07-18T03:41:06.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.799+0000 7f86257ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f862015c950 0x7f86200b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:06.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.799+0000 7f86257ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86200b1e00 0x7f86200d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:06.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.800+0000 7f86257ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86200d2ce0 con 0x7f86200b2350 2023-07-18T03:41:06.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.800+0000 7f86257ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86200b3350 con 0x7f862015c950 2023-07-18T03:41:06.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.800+0000 7f86257ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86200b3630 con 0x7f86200b1e00 2023-07-18T03:41:06.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.800+0000 7f861effd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f862015c950 0x7f86200b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:06.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.800+0000 7f861effd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f862015c950 0x7f86200b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58330/0 (socket says 172.21.15.49:58330) 2023-07-18T03:41:06.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.800+0000 7f861effd700 1 -- 172.21.15.49:0/2619631297 learned_addr learned my addr 172.21.15.49:0/2619631297 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:06.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.800+0000 7f861effd700 1 -- 172.21.15.49:0/2619631297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86200b1e00 msgr2=0x7f86200d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:06.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.800+0000 7f8617fff700 1 --2- 172.21.15.49:0/2619631297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86200b1e00 0x7f86200d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:06.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.800+0000 7f861effd700 1 --2- 172.21.15.49:0/2619631297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86200b1e00 0x7f86200d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.800+0000 7f861effd700 1 -- 172.21.15.49:0/2619631297 >> v1:172.21.15.107:6789/0 conn(0x7f86200b2350 legacy=0x7f86200afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:06.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.800+0000 7f861effd700 1 -- 172.21.15.49:0/2619631297 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86200b3910 con 0x7f862015c950 2023-07-18T03:41:06.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.800+0000 7f8617fff700 1 --2- 172.21.15.49:0/2619631297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86200b1e00 0x7f86200d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:06.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.801+0000 7f861effd700 1 --2- 172.21.15.49:0/2619631297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f862015c950 0x7f86200b43f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8610006fd0 tx=0x7f8610009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ce007183c78175 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:06.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.801+0000 7f861e7fc700 1 -- 172.21.15.49:0/2619631297 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610005e30 con 0x7f862015c950 2023-07-18T03:41:06.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.801+0000 7f861e7fc700 1 -- 172.21.15.49:0/2619631297 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8610006030 con 0x7f862015c950 2023-07-18T03:41:06.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.801+0000 7f861e7fc700 1 -- 172.21.15.49:0/2619631297 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86100063c0 con 0x7f862015c950 2023-07-18T03:41:06.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.801+0000 7f86257ff700 1 -- 172.21.15.49:0/2619631297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f862015c950 msgr2=0x7f86200b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:06.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.801+0000 7f86257ff700 1 --2- 172.21.15.49:0/2619631297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f862015c950 0x7f86200b43f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8610006fd0 tx=0x7f8610009bd0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.801+0000 7f86257ff700 1 -- 172.21.15.49:0/2619631297 shutdown_connections 2023-07-18T03:41:06.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.802+0000 7f86257ff700 1 --2- 172.21.15.49:0/2619631297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f862015c950 0x7f86200b43f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.802+0000 7f86257ff700 1 --2- 172.21.15.49:0/2619631297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86200b1e00 0x7f86200d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.802+0000 7f86257ff700 1 -- 172.21.15.49:0/2619631297 >> 172.21.15.49:0/2619631297 conn(0x7f862009c1d0 msgr2=0x7f86200d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:06.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.802+0000 7f86257ff700 1 -- 172.21.15.49:0/2619631297 shutdown_connections 2023-07-18T03:41:06.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.802+0000 7f86257ff700 1 -- 172.21.15.49:0/2619631297 wait complete. 2023-07-18T03:41:06.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.803+0000 7f86257ff700 1 Processor -- start 2023-07-18T03:41:06.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.804+0000 7f86257ff700 1 -- start start 2023-07-18T03:41:06.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.804+0000 7f86257ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86200b1e00 0x7f86200b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:06.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.804+0000 7f86257ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86200b2350 0x7f86200c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:06.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f86257ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f862015c950 0x7f86200cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:06.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f86257ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86200c8800 con 0x7f862015c950 2023-07-18T03:41:06.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f86257ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86200c7960 con 0x7f86200b2350 2023-07-18T03:41:06.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f86257ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86200c7b80 con 0x7f86200b1e00 2023-07-18T03:41:06.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f861f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f862015c950 0x7f86200cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:06.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f861f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f862015c950 0x7f86200cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58346/0 (socket says 172.21.15.49:58346) 2023-07-18T03:41:06.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f861f7fe700 1 -- 172.21.15.49:0/3534381332 learned_addr learned my addr 172.21.15.49:0/3534381332 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:06.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f861effd700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86200b1e00 0x7f86200b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:06.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f8617fff700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86200b2350 0x7f86200c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:06.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f861f7fe700 1 -- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86200b1e00 msgr2=0x7f86200b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:06.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f861f7fe700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86200b1e00 0x7f86200b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f861f7fe700 1 -- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86200b2350 msgr2=0x7f86200c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f861f7fe700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86200b2350 0x7f86200c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f861f7fe700 1 -- 172.21.15.49:0/3534381332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8610009500 con 0x7f862015c950 2023-07-18T03:41:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.805+0000 7f861effd700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86200b1e00 0x7f86200b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.806+0000 7f861f7fe700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f862015c950 0x7f86200cf380 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f861800cc40 tx=0x7f861800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.806+0000 7f86177fe700 1 -- 172.21.15.49:0/3534381332 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861800aba0 con 0x7f862015c950 2023-07-18T03:41:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.806+0000 7f86177fe700 1 -- 172.21.15.49:0/3534381332 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861800ad00 con 0x7f862015c950 2023-07-18T03:41:06.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.806+0000 7f86177fe700 1 -- 172.21.15.49:0/3534381332 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618012660 con 0x7f862015c950 2023-07-18T03:41:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.806+0000 7f86257ff700 1 -- 172.21.15.49:0/3534381332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86200c7e60 con 0x7f862015c950 2023-07-18T03:41:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.806+0000 7f86257ff700 1 -- 172.21.15.49:0/3534381332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86200c8420 con 0x7f862015c950 2023-07-18T03:41:06.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.807+0000 7f86257ff700 1 -- 172.21.15.49:0/3534381332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862004e280 con 0x7f862015c950 2023-07-18T03:41:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.808+0000 7f86177fe700 1 -- 172.21.15.49:0/3534381332 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f86180127c0 con 0x7f862015c950 2023-07-18T03:41:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.809+0000 7f86177fe700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f860006ae00 0x7f860006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.809+0000 7f86177fe700 1 -- 172.21.15.49:0/3534381332 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f8618089de0 con 0x7f862015c950 2023-07-18T03:41:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.809+0000 7f861effd700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f860006ae00 0x7f860006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:06.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.810+0000 7f861effd700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f860006ae00 0x7f860006d2b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8610013040 tx=0x7f8610008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:06.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:06.813+0000 7f86177fe700 1 -- 172.21.15.49:0/3534381332 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8618054d30 con 0x7f862015c950 2023-07-18T03:41:07.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.070+0000 7f86257ff700 1 -- 172.21.15.49:0/3534381332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f862008c5a0 con 0x7f862015c950 2023-07-18T03:41:07.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.071+0000 7f86177fe700 1 -- 172.21.15.49:0/3534381332 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f8618016020 con 0x7f862015c950 2023-07-18T03:41:07.065 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:07.066 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/777524677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":777524677},{"type":"v1","addr":"172.21.15.107:6837","nonce":777524677}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/2382452558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2382452558},{"type":"v1","addr":"172.21.15.49:6834","nonce":2382452558}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/2843143219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2843143219},{"type":"v1","addr":"172.21.15.107:6836","nonce":2843143219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:40:40.451705+0000","modified":"2023-07-18T03:40:43.957663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5479":{"gid":5479,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1453078087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":1453078087},{"type":"v1","addr":"172.21.15.49:6835","nonce":1453078087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-18T03:41:07.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.073+0000 7f86257ff700 1 -- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f860006ae00 msgr2=0x7f860006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:07.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.073+0000 7f86257ff700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f860006ae00 0x7f860006d2b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8610013040 tx=0x7f8610008070 comp rx=0 tx=0).stop 2023-07-18T03:41:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.073+0000 7f86257ff700 1 -- 172.21.15.49:0/3534381332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f862015c950 msgr2=0x7f86200cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.073+0000 7f86257ff700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f862015c950 0x7f86200cf380 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f861800cc40 tx=0x7f861800a060 comp rx=0 tx=0).stop 2023-07-18T03:41:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.073+0000 7f86257ff700 1 -- 172.21.15.49:0/3534381332 shutdown_connections 2023-07-18T03:41:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.073+0000 7f86257ff700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f860006ae00 0x7f860006d2b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.073+0000 7f86257ff700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f862015c950 0x7f86200cf380 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.073+0000 7f86257ff700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86200b1e00 0x7f86200b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.073+0000 7f86257ff700 1 --2- 172.21.15.49:0/3534381332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86200b2350 0x7f86200c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.073+0000 7f86257ff700 1 -- 172.21.15.49:0/3534381332 >> 172.21.15.49:0/3534381332 conn(0x7f862009c1d0 msgr2=0x7f86200d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:07.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.073+0000 7f86257ff700 1 -- 172.21.15.49:0/3534381332 shutdown_connections 2023-07-18T03:41:07.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.074+0000 7f86257ff700 1 -- 172.21.15.49:0/3534381332 wait complete. 2023-07-18T03:41:07.070 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 29 2023-07-18T03:41:07.086 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:41:07.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.296+0000 7f518bdff700 1 Processor -- start 2023-07-18T03:41:07.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.297+0000 7f518bdff700 1 -- start start 2023-07-18T03:41:07.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.298+0000 7f518bdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f518415c710 0x7f51840cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:07.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.298+0000 7f518bdff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51840cd320 con 0x7f5184069f30 2023-07-18T03:41:07.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.298+0000 7f518bdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51840ce750 con 0x7f518406db30 2023-07-18T03:41:07.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.298+0000 7f518bdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51840cea30 con 0x7f518415c710 2023-07-18T03:41:07.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.298+0000 7f518939a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5184069f30 0x7f518406d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50564/0 (socket says 172.21.15.49:50564) 2023-07-18T03:41:07.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.298+0000 7f518939a700 1 -- 172.21.15.49:0/462327209 learned_addr learned my addr 172.21.15.49:0/462327209 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:07.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.298+0000 7f5189b9b700 1 --2- 172.21.15.49:0/462327209 >> v2:172.21.15.107:3300/0 conn(0x7f518415c710 0x7f51840cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:07.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.298+0000 7f5188b99700 1 -- 172.21.15.49:0/462327209 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885246957 0 0) 0x7f51840cd320 con 0x7f5184069f30 2023-07-18T03:41:07.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.299+0000 7f5188b99700 1 -- 172.21.15.49:0/462327209 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51700035b0 con 0x7f5184069f30 2023-07-18T03:41:07.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.299+0000 7f5189b9b700 1 -- 172.21.15.49:0/462327209 >> v1:172.21.15.107:6789/0 conn(0x7f518406db30 legacy=0x7f5184070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:07.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.299+0000 7f5189b9b700 1 -- 172.21.15.49:0/462327209 >> v1:172.21.15.49:6789/0 conn(0x7f5184069f30 legacy=0x7f518406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:07.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.299+0000 7f5189b9b700 1 -- 172.21.15.49:0/462327209 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51840ced10 con 0x7f518415c710 2023-07-18T03:41:07.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.299+0000 7f5189b9b700 1 --2- 172.21.15.49:0/462327209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f518415c710 0x7f51840cf510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5180009840 tx=0x7f5180009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec563a3470dd5d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:07.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.299+0000 7f5188b99700 1 -- 172.21.15.49:0/462327209 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180011830 con 0x7f518415c710 2023-07-18T03:41:07.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.299+0000 7f5188b99700 1 -- 172.21.15.49:0/462327209 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5180011990 con 0x7f518415c710 2023-07-18T03:41:07.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.299+0000 7f5188b99700 1 -- 172.21.15.49:0/462327209 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180011d20 con 0x7f518415c710 2023-07-18T03:41:07.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.300+0000 7f518bdff700 1 -- 172.21.15.49:0/462327209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f518415c710 msgr2=0x7f51840cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:07.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.300+0000 7f518bdff700 1 --2- 172.21.15.49:0/462327209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f518415c710 0x7f51840cf510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5180009840 tx=0x7f5180009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:07.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.300+0000 7f518bdff700 1 -- 172.21.15.49:0/462327209 shutdown_connections 2023-07-18T03:41:07.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.300+0000 7f518bdff700 1 --2- 172.21.15.49:0/462327209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f518415c710 0x7f51840cf510 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.300+0000 7f518bdff700 1 -- 172.21.15.49:0/462327209 >> 172.21.15.49:0/462327209 conn(0x7f51840b16c0 msgr2=0x7f51840b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:07.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.300+0000 7f518bdff700 1 -- 172.21.15.49:0/462327209 shutdown_connections 2023-07-18T03:41:07.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.300+0000 7f518bdff700 1 -- 172.21.15.49:0/462327209 wait complete. 2023-07-18T03:41:07.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.302+0000 7f518bdff700 1 Processor -- start 2023-07-18T03:41:07.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.302+0000 7f518bdff700 1 -- start start 2023-07-18T03:41:07.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.303+0000 7f518bdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5184069f30 0x7f518415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:07.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.303+0000 7f518bdff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518406db30 0x7f518415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:07.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.303+0000 7f518bdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5184159880 0x7f518414ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:07.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.303+0000 7f518bdff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518415b2f0 con 0x7f518406db30 2023-07-18T03:41:07.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.303+0000 7f518bdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518414b830 con 0x7f5184069f30 2023-07-18T03:41:07.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.303+0000 7f518bdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518415b010 con 0x7f5184159880 2023-07-18T03:41:07.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.303+0000 7f518939a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518406db30 0x7f518415cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:07.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.303+0000 7f518939a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518406db30 0x7f518415cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58348/0 (socket says 172.21.15.49:58348) 2023-07-18T03:41:07.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.303+0000 7f518939a700 1 -- 172.21.15.49:0/916413341 learned_addr learned my addr 172.21.15.49:0/916413341 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:07.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.303+0000 7f5189b9b700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5184069f30 0x7f518415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:07.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.303+0000 7f518a39c700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5184159880 0x7f518414ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:07.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.304+0000 7f518939a700 1 -- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5184159880 msgr2=0x7f518414ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:07.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.304+0000 7f518939a700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5184159880 0x7f518414ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.304+0000 7f518939a700 1 -- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5184069f30 msgr2=0x7f518415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:07.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.304+0000 7f518939a700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5184069f30 0x7f518415a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.304+0000 7f518939a700 1 -- 172.21.15.49:0/916413341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800094a0 con 0x7f518406db30 2023-07-18T03:41:07.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.304+0000 7f5189b9b700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5184069f30 0x7f518415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:07.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.304+0000 7f518939a700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518406db30 0x7f518415cd60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f517400c700 tx=0x7f517400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:07.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.304+0000 7f517a7fc700 1 -- 172.21.15.49:0/916413341 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174005d70 con 0x7f518406db30 2023-07-18T03:41:07.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.304+0000 7f517a7fc700 1 -- 172.21.15.49:0/916413341 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51740070d0 con 0x7f518406db30 2023-07-18T03:41:07.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.304+0000 7f517a7fc700 1 -- 172.21.15.49:0/916413341 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174007460 con 0x7f518406db30 2023-07-18T03:41:07.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.304+0000 7f518bdff700 1 -- 172.21.15.49:0/916413341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518406ac20 con 0x7f518406db30 2023-07-18T03:41:07.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.304+0000 7f518bdff700 1 -- 172.21.15.49:0/916413341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51840b3750 con 0x7f518406db30 2023-07-18T03:41:07.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.305+0000 7f518bdff700 1 -- 172.21.15.49:0/916413341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51840cf510 con 0x7f518406db30 2023-07-18T03:41:07.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.306+0000 7f517a7fc700 1 -- 172.21.15.49:0/916413341 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f51740075c0 con 0x7f518406db30 2023-07-18T03:41:07.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.307+0000 7f517a7fc700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f516806ae00 0x7f516806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:07.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.307+0000 7f517a7fc700 1 -- 172.21.15.49:0/916413341 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f5174088ce0 con 0x7f518406db30 2023-07-18T03:41:07.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.308+0000 7f5189b9b700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f516806ae00 0x7f516806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:07.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.308+0000 7f5189b9b700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f516806ae00 0x7f516806d2b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f51800090b0 tx=0x7f518001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:07.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.311+0000 7f517a7fc700 1 -- 172.21.15.49:0/916413341 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5174053bb0 con 0x7f518406db30 2023-07-18T03:41:07.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.567+0000 7f518bdff700 1 -- 172.21.15.49:0/916413341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f518404e280 con 0x7f518406db30 2023-07-18T03:41:07.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.567+0000 7f517a7fc700 1 -- 172.21.15.49:0/916413341 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f51740571d0 con 0x7f518406db30 2023-07-18T03:41:07.561 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:07.562 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/777524677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":777524677},{"type":"v1","addr":"172.21.15.107:6837","nonce":777524677}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/2382452558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2382452558},{"type":"v1","addr":"172.21.15.49:6834","nonce":2382452558}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/2843143219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2843143219},{"type":"v1","addr":"172.21.15.107:6836","nonce":2843143219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:40:40.451705+0000","modified":"2023-07-18T03:40:43.957663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5479":{"gid":5479,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1453078087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":1453078087},{"type":"v1","addr":"172.21.15.49:6835","nonce":1453078087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-18T03:41:07.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.570+0000 7f518bdff700 1 -- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f516806ae00 msgr2=0x7f516806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:07.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.570+0000 7f518bdff700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f516806ae00 0x7f516806d2b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f51800090b0 tx=0x7f518001c040 comp rx=0 tx=0).stop 2023-07-18T03:41:07.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.570+0000 7f518bdff700 1 -- 172.21.15.49:0/916413341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518406db30 msgr2=0x7f518415cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:07.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.570+0000 7f518bdff700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518406db30 0x7f518415cd60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f517400c700 tx=0x7f517400cac0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.570+0000 7f518bdff700 1 -- 172.21.15.49:0/916413341 shutdown_connections 2023-07-18T03:41:07.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.570+0000 7f518bdff700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5184159880 0x7f518414ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.570+0000 7f518bdff700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5184069f30 0x7f518415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.570+0000 7f518bdff700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f516806ae00 0x7f516806d2b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.570+0000 7f518bdff700 1 --2- 172.21.15.49:0/916413341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518406db30 0x7f518415cd60 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.570+0000 7f518bdff700 1 -- 172.21.15.49:0/916413341 >> 172.21.15.49:0/916413341 conn(0x7f51840b16c0 msgr2=0x7f51840b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:07.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.570+0000 7f518bdff700 1 -- 172.21.15.49:0/916413341 shutdown_connections 2023-07-18T03:41:07.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.570+0000 7f518bdff700 1 -- 172.21.15.49:0/916413341 wait complete. 2023-07-18T03:41:07.566 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 29 2023-07-18T03:41:07.582 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:41:07.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.793+0000 7f468f9ff700 1 Processor -- start 2023-07-18T03:41:07.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.793+0000 7f468f9ff700 1 -- start start 2023-07-18T03:41:07.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.794+0000 7f468f9ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f46880b4510 0x7f46880b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:07.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.794+0000 7f468f9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f46880b5d90 0x7f46880cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:07.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.794+0000 7f468f9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46880d0b10 0x7f46880d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:07.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.794+0000 7f468f9ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46880b48e0 con 0x7f46880b4510 2023-07-18T03:41:07.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.794+0000 7f468f9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46880d14d0 con 0x7f46880b5d90 2023-07-18T03:41:07.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.794+0000 7f468f9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46880b7f50 con 0x7f46880d0b10 2023-07-18T03:41:07.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.795+0000 7f468d79b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f46880b4510 0x7f46880b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:07.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.795+0000 7f468d79b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f46880b4510 0x7f46880b6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58352/0 (socket says 172.21.15.49:58352) 2023-07-18T03:41:07.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.795+0000 7f468d79b700 1 -- 172.21.15.49:0/1070304415 learned_addr learned my addr 172.21.15.49:0/1070304415 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:07.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.795+0000 7f468cf9a700 1 --2- 172.21.15.49:0/1070304415 >> v2:172.21.15.107:3300/0 conn(0x7f46880b5d90 0x7f46880cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:07.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.795+0000 7f468df9c700 1 --2- 172.21.15.49:0/1070304415 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46880d0b10 0x7f46880d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:07.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.795+0000 7f468d79b700 1 -- 172.21.15.49:0/1070304415 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46880d0b10 msgr2=0x7f46880d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:07.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.795+0000 7f468d79b700 1 --2- 172.21.15.49:0/1070304415 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46880d0b10 0x7f46880d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.795+0000 7f468d79b700 1 -- 172.21.15.49:0/1070304415 >> v2:172.21.15.107:3300/0 conn(0x7f46880b5d90 msgr2=0x7f46880cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:07.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.795+0000 7f468d79b700 1 --2- 172.21.15.49:0/1070304415 >> v2:172.21.15.107:3300/0 conn(0x7f46880b5d90 0x7f46880cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.795+0000 7f468d79b700 1 -- 172.21.15.49:0/1070304415 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880b7340 con 0x7f46880b4510 2023-07-18T03:41:07.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.795+0000 7f468d79b700 1 --2- 172.21.15.49:0/1070304415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46880b4510 0x7f46880b6c80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f46880b5b30 tx=0x7f4684009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7f8a2d9496f004 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:07.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.796+0000 7f467ffff700 1 -- 172.21.15.49:0/1070304415 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684005e30 con 0x7f46880b4510 2023-07-18T03:41:07.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.796+0000 7f467ffff700 1 -- 172.21.15.49:0/1070304415 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4684004030 con 0x7f46880b4510 2023-07-18T03:41:07.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.796+0000 7f467ffff700 1 -- 172.21.15.49:0/1070304415 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840043e0 con 0x7f46880b4510 2023-07-18T03:41:07.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.796+0000 7f468f9ff700 1 -- 172.21.15.49:0/1070304415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46880b4510 msgr2=0x7f46880b6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:07.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.796+0000 7f468f9ff700 1 --2- 172.21.15.49:0/1070304415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46880b4510 0x7f46880b6c80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f46880b5b30 tx=0x7f4684009bd0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.796+0000 7f468f9ff700 1 -- 172.21.15.49:0/1070304415 shutdown_connections 2023-07-18T03:41:07.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.796+0000 7f468f9ff700 1 --2- 172.21.15.49:0/1070304415 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46880d0b10 0x7f46880d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.796+0000 7f468f9ff700 1 --2- 172.21.15.49:0/1070304415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46880b4510 0x7f46880b6c80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.796+0000 7f468f9ff700 1 --2- 172.21.15.49:0/1070304415 >> v2:172.21.15.107:3300/0 conn(0x7f46880b5d90 0x7f46880cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.796+0000 7f468f9ff700 1 -- 172.21.15.49:0/1070304415 >> 172.21.15.49:0/1070304415 conn(0x7f468809c1d0 msgr2=0x7f46880d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:07.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.796+0000 7f468f9ff700 1 -- 172.21.15.49:0/1070304415 shutdown_connections 2023-07-18T03:41:07.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.796+0000 7f468f9ff700 1 -- 172.21.15.49:0/1070304415 wait complete. 2023-07-18T03:41:07.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.799+0000 7f468f9ff700 1 Processor -- start 2023-07-18T03:41:07.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.799+0000 7f468f9ff700 1 -- start start 2023-07-18T03:41:07.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.800+0000 7f468f9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46880b4510 0x7f46881514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:07.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.800+0000 7f468f9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46880b5d90 0x7f46881535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:07.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.800+0000 7f468f9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46880d0b10 0x7f468806b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:07.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.800+0000 7f468f9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46881578f0 con 0x7f46880b5d90 2023-07-18T03:41:07.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.800+0000 7f468f9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4688151a00 con 0x7f46880b4510 2023-07-18T03:41:07.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.800+0000 7f468f9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46881545f0 con 0x7f46880d0b10 2023-07-18T03:41:07.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.801+0000 7f468cf9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46880b5d90 0x7f46881535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:07.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.801+0000 7f468cf9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46880b5d90 0x7f46881535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58360/0 (socket says 172.21.15.49:58360) 2023-07-18T03:41:07.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.801+0000 7f468cf9a700 1 -- 172.21.15.49:0/3788949384 learned_addr learned my addr 172.21.15.49:0/3788949384 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:07.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.801+0000 7f468d79b700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46880b4510 0x7f46881514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:07.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.801+0000 7f468df9c700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46880d0b10 0x7f468806b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:07.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.801+0000 7f468cf9a700 1 -- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46880d0b10 msgr2=0x7f468806b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:07.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.801+0000 7f468cf9a700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46880d0b10 0x7f468806b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.801+0000 7f468cf9a700 1 -- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46880b4510 msgr2=0x7f46881514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:07.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.801+0000 7f468cf9a700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46880b4510 0x7f46881514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:07.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.801+0000 7f468cf9a700 1 -- 172.21.15.49:0/3788949384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4684009500 con 0x7f46880b5d90 2023-07-18T03:41:07.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.801+0000 7f468cf9a700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46880b5d90 0x7f46881535d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f467800e930 tx=0x7f467800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:07.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.802+0000 7f467dffb700 1 -- 172.21.15.49:0/3788949384 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678013680 con 0x7f46880b5d90 2023-07-18T03:41:07.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.802+0000 7f467dffb700 1 -- 172.21.15.49:0/3788949384 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46780137e0 con 0x7f46880b5d90 2023-07-18T03:41:07.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.802+0000 7f467dffb700 1 -- 172.21.15.49:0/3788949384 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678013b70 con 0x7f46880b5d90 2023-07-18T03:41:07.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.802+0000 7f468f9ff700 1 -- 172.21.15.49:0/3788949384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4688157460 con 0x7f46880b5d90 2023-07-18T03:41:07.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.802+0000 7f468f9ff700 1 -- 172.21.15.49:0/3788949384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4688156b30 con 0x7f46880b5d90 2023-07-18T03:41:07.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.803+0000 7f468f9ff700 1 -- 172.21.15.49:0/3788949384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468804e280 con 0x7f46880b5d90 2023-07-18T03:41:07.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.804+0000 7f467dffb700 1 -- 172.21.15.49:0/3788949384 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f4678013cd0 con 0x7f46880b5d90 2023-07-18T03:41:07.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.805+0000 7f467dffb700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f466c06ae00 0x7f466c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:07.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.805+0000 7f467dffb700 1 -- 172.21.15.49:0/3788949384 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f4678089570 con 0x7f46880b5d90 2023-07-18T03:41:07.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.805+0000 7f468d79b700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f466c06ae00 0x7f466c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:07.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.806+0000 7f468d79b700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f466c06ae00 0x7f466c06d2b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4684009c10 tx=0x7f4684008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:07.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:07.809+0000 7f467dffb700 1 -- 172.21.15.49:0/3788949384 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f46780544c0 con 0x7f46880b5d90 2023-07-18T03:41:08.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.018+0000 7f468f9ff700 1 -- 172.21.15.49:0/3788949384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f468815c600 con 0x7f46880b5d90 2023-07-18T03:41:08.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.019+0000 7f467dffb700 1 -- 172.21.15.49:0/3788949384 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f4678057ae0 con 0x7f46880b5d90 2023-07-18T03:41:08.014 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:08.014 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":50,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:41:00.954181+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:40:38.792808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-18T03:40:39.438590+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":8,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:41:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.022+0000 7f468f9ff700 1 -- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f466c06ae00 msgr2=0x7f466c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.022+0000 7f468f9ff700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f466c06ae00 0x7f466c06d2b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4684009c10 tx=0x7f4684008070 comp rx=0 tx=0).stop 2023-07-18T03:41:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.022+0000 7f468f9ff700 1 -- 172.21.15.49:0/3788949384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46880b5d90 msgr2=0x7f46881535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.022+0000 7f468f9ff700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46880b5d90 0x7f46881535d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f467800e930 tx=0x7f467800ec40 comp rx=0 tx=0).stop 2023-07-18T03:41:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.022+0000 7f468f9ff700 1 -- 172.21.15.49:0/3788949384 shutdown_connections 2023-07-18T03:41:08.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.022+0000 7f468f9ff700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46880d0b10 0x7f468806b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.022+0000 7f468f9ff700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46880b4510 0x7f46881514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.022+0000 7f468f9ff700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f466c06ae00 0x7f466c06d2b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.022+0000 7f468f9ff700 1 --2- 172.21.15.49:0/3788949384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46880b5d90 0x7f46881535d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.022+0000 7f468f9ff700 1 -- 172.21.15.49:0/3788949384 >> 172.21.15.49:0/3788949384 conn(0x7f468809c1d0 msgr2=0x7f46880c6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:08.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.022+0000 7f468f9ff700 1 -- 172.21.15.49:0/3788949384 shutdown_connections 2023-07-18T03:41:08.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.022+0000 7f468f9ff700 1 -- 172.21.15.49:0/3788949384 wait complete. 2023-07-18T03:41:08.035 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-18T03:41:08.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.247+0000 7f976e9ff700 1 Processor -- start 2023-07-18T03:41:08.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.248+0000 7f976e9ff700 1 -- start start 2023-07-18T03:41:08.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.249+0000 7f976e9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f97680b4600 0x7f97680cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:08.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.249+0000 7f976e9ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976815cd60 con 0x7f97680b6f00 2023-07-18T03:41:08.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.249+0000 7f976e9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97680ce360 con 0x7f97680cfe40 2023-07-18T03:41:08.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.249+0000 7f976e9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97680d1550 con 0x7f97680b4600 2023-07-18T03:41:08.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.249+0000 7f976cf9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f97680b6f00 0x7f97680cdc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50578/0 (socket says 172.21.15.49:50578) 2023-07-18T03:41:08.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.249+0000 7f976cf9c700 1 -- 172.21.15.49:0/2423703462 learned_addr learned my addr 172.21.15.49:0/2423703462 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:08.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.249+0000 7f9767fff700 1 --2- 172.21.15.49:0/2423703462 >> v2:172.21.15.107:3300/0 conn(0x7f97680b4600 0x7f97680cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:08.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.249+0000 7f9766ffd700 1 -- 172.21.15.49:0/2423703462 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770132195 0 0) 0x7f976815cd60 con 0x7f97680b6f00 2023-07-18T03:41:08.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.249+0000 7f9766ffd700 1 -- 172.21.15.49:0/2423703462 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97500035b0 con 0x7f97680b6f00 2023-07-18T03:41:08.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.250+0000 7f9767fff700 1 -- 172.21.15.49:0/2423703462 >> v1:172.21.15.107:6789/0 conn(0x7f97680cfe40 legacy=0x7f97680cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.250+0000 7f9767fff700 1 -- 172.21.15.49:0/2423703462 >> v1:172.21.15.49:6789/0 conn(0x7f97680b6f00 legacy=0x7f97680cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.250+0000 7f9767fff700 1 -- 172.21.15.49:0/2423703462 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97680cf340 con 0x7f97680b4600 2023-07-18T03:41:08.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.250+0000 7f9767fff700 1 --2- 172.21.15.49:0/2423703462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97680b4600 0x7f97680cf900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9758009840 tx=0x7f9758009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceb804996b8f4567 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.250+0000 7f9766ffd700 1 -- 172.21.15.49:0/2423703462 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758011830 con 0x7f97680b4600 2023-07-18T03:41:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.250+0000 7f9766ffd700 1 -- 172.21.15.49:0/2423703462 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9758011990 con 0x7f97680b4600 2023-07-18T03:41:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.250+0000 7f9766ffd700 1 -- 172.21.15.49:0/2423703462 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758011d20 con 0x7f97680b4600 2023-07-18T03:41:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.251+0000 7f976e9ff700 1 -- 172.21.15.49:0/2423703462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97680b4600 msgr2=0x7f97680cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.251+0000 7f976e9ff700 1 --2- 172.21.15.49:0/2423703462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97680b4600 0x7f97680cf900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9758009840 tx=0x7f9758009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.251+0000 7f976e9ff700 1 -- 172.21.15.49:0/2423703462 shutdown_connections 2023-07-18T03:41:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.251+0000 7f976e9ff700 1 --2- 172.21.15.49:0/2423703462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97680b4600 0x7f97680cf900 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.251+0000 7f976e9ff700 1 -- 172.21.15.49:0/2423703462 >> 172.21.15.49:0/2423703462 conn(0x7f976809c200 msgr2=0x7f97680d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.251+0000 7f976e9ff700 1 -- 172.21.15.49:0/2423703462 shutdown_connections 2023-07-18T03:41:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.251+0000 7f976e9ff700 1 -- 172.21.15.49:0/2423703462 wait complete. 2023-07-18T03:41:08.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.253+0000 7f976e9ff700 1 Processor -- start 2023-07-18T03:41:08.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.253+0000 7f976e9ff700 1 -- start start 2023-07-18T03:41:08.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f976e9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97680b6f00 0x7f9768154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f976e9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97680cfe40 0x7f97680c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f976e9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97680c95c0 0x7f976815ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f976e9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768156710 con 0x7f97680b6f00 2023-07-18T03:41:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f976e9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97681547f0 con 0x7f97680c95c0 2023-07-18T03:41:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f976e9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9768153be0 con 0x7f97680cfe40 2023-07-18T03:41:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f9767fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97680b6f00 0x7f9768154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f9767fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97680b6f00 0x7f9768154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58366/0 (socket says 172.21.15.49:58366) 2023-07-18T03:41:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f9767fff700 1 -- 172.21.15.49:0/1570353787 learned_addr learned my addr 172.21.15.49:0/1570353787 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:08.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f97677fe700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97680cfe40 0x7f97680c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:08.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f976cf9c700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97680c95c0 0x7f976815ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f9767fff700 1 -- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97680cfe40 msgr2=0x7f97680c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f9767fff700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97680cfe40 0x7f97680c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f9767fff700 1 -- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97680c95c0 msgr2=0x7f976815ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f9767fff700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97680c95c0 0x7f976815ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f9767fff700 1 -- 172.21.15.49:0/1570353787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580094a0 con 0x7f97680b6f00 2023-07-18T03:41:08.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.254+0000 7f976cf9c700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97680c95c0 0x7f976815ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:08.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.255+0000 7f9767fff700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97680b6f00 0x7f9768154c80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9758009fd0 tx=0x7f9758004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.255+0000 7f9764ff9700 1 -- 172.21.15.49:0/1570353787 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97580038a0 con 0x7f97680b6f00 2023-07-18T03:41:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.255+0000 7f9764ff9700 1 -- 172.21.15.49:0/1570353787 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975801f070 con 0x7f97680b6f00 2023-07-18T03:41:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.255+0000 7f976e9ff700 1 -- 172.21.15.49:0/1570353787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9768153ec0 con 0x7f97680b6f00 2023-07-18T03:41:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.255+0000 7f9764ff9700 1 -- 172.21.15.49:0/1570353787 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758003c40 con 0x7f97680b6f00 2023-07-18T03:41:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.255+0000 7f976e9ff700 1 -- 172.21.15.49:0/1570353787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9768154480 con 0x7f97680b6f00 2023-07-18T03:41:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.256+0000 7f976e9ff700 1 -- 172.21.15.49:0/1570353787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976809c600 con 0x7f97680b6f00 2023-07-18T03:41:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.257+0000 7f9764ff9700 1 -- 172.21.15.49:0/1570353787 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f9758026480 con 0x7f97680b6f00 2023-07-18T03:41:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.258+0000 7f9764ff9700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f975406ae00 0x7f975406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.258+0000 7f9764ff9700 1 -- 172.21.15.49:0/1570353787 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f9758021070 con 0x7f97680b6f00 2023-07-18T03:41:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.259+0000 7f97677fe700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f975406ae00 0x7f975406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.259+0000 7f97677fe700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f975406ae00 0x7f975406d2b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9768156290 tx=0x7f975c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:08.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.262+0000 7f9764ff9700 1 -- 172.21.15.49:0/1570353787 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9758055f40 con 0x7f97680b6f00 2023-07-18T03:41:08.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.517+0000 7f976e9ff700 1 -- 172.21.15.49:0/1570353787 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9768074350 con 0x7f97680b6f00 2023-07-18T03:41:08.512 INFO:tasks.ceph.mon.a.smithi049.stderr:2023-07-18T03:41:08.518+0000 7ff4cc5f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-18T03:41:08.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.523+0000 7f9764ff9700 1 -- 172.21.15.49:0/1570353787 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9758015090 con 0x7f97680b6f00 2023-07-18T03:41:08.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.525+0000 7f976e9ff700 1 -- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f975406ae00 msgr2=0x7f975406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.525+0000 7f976e9ff700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f975406ae00 0x7f975406d2b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9768156290 tx=0x7f975c009040 comp rx=0 tx=0).stop 2023-07-18T03:41:08.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.525+0000 7f976e9ff700 1 -- 172.21.15.49:0/1570353787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97680b6f00 msgr2=0x7f9768154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.525+0000 7f976e9ff700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97680b6f00 0x7f9768154c80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9758009fd0 tx=0x7f9758004750 comp rx=0 tx=0).stop 2023-07-18T03:41:08.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.526+0000 7f976e9ff700 1 -- 172.21.15.49:0/1570353787 shutdown_connections 2023-07-18T03:41:08.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.526+0000 7f976e9ff700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97680c95c0 0x7f976815ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.526+0000 7f976e9ff700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f975406ae00 0x7f975406d2b0 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9768156290 tx=0x7f975c009040 comp rx=0 tx=0).stop 2023-07-18T03:41:08.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.526+0000 7f976e9ff700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97680b6f00 0x7f9768154c80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.526+0000 7f976e9ff700 1 --2- 172.21.15.49:0/1570353787 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97680cfe40 0x7f97680c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.526+0000 7f976e9ff700 1 -- 172.21.15.49:0/1570353787 >> 172.21.15.49:0/1570353787 conn(0x7f976809c200 msgr2=0x7f97680b7310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:08.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.526+0000 7f976e9ff700 1 -- 172.21.15.49:0/1570353787 shutdown_connections 2023-07-18T03:41:08.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.526+0000 7f976e9ff700 1 -- 172.21.15.49:0/1570353787 wait complete. 2023-07-18T03:41:08.523 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-18T03:41:08.539 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-18T03:41:08.590 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-07-18T03:41:08.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.754+0000 7fce4f9ff700 1 Processor -- start 2023-07-18T03:41:08.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.754+0000 7fce4f9ff700 1 -- start start 2023-07-18T03:41:08.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.755+0000 7fce4f9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fce480d0b10 0x7fce480d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:08.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.756+0000 7fce4f9ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce480b8e30 con 0x7fce480b8a60 2023-07-18T03:41:08.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.756+0000 7fce4f9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce480afaa0 con 0x7fce480b0e20 2023-07-18T03:41:08.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.756+0000 7fce4f9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce480d14d0 con 0x7fce480d0b10 2023-07-18T03:41:08.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.756+0000 7fce4d79b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fce480b8a60 0x7fce480b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50592/0 (socket says 172.21.15.49:50592) 2023-07-18T03:41:08.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.756+0000 7fce4d79b700 1 -- 172.21.15.49:0/2493530410 learned_addr learned my addr 172.21.15.49:0/2493530410 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:08.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.756+0000 7fce4cf9a700 1 --2- 172.21.15.49:0/2493530410 >> v2:172.21.15.107:3300/0 conn(0x7fce480d0b10 0x7fce480d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:08.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.756+0000 7fce3ffff700 1 -- 172.21.15.49:0/2493530410 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706787693 0 0) 0x7fce480b8e30 con 0x7fce480b8a60 2023-07-18T03:41:08.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.756+0000 7fce3ffff700 1 -- 172.21.15.49:0/2493530410 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce340035b0 con 0x7fce480b8a60 2023-07-18T03:41:08.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.757+0000 7fce4cf9a700 1 -- 172.21.15.49:0/2493530410 >> v1:172.21.15.107:6789/0 conn(0x7fce480b0e20 legacy=0x7fce480b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.757+0000 7fce4cf9a700 1 -- 172.21.15.49:0/2493530410 >> v1:172.21.15.49:6789/0 conn(0x7fce480b8a60 legacy=0x7fce480b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.757+0000 7fce4cf9a700 1 -- 172.21.15.49:0/2493530410 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce480afd80 con 0x7fce480d0b10 2023-07-18T03:41:08.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.757+0000 7fce4cf9a700 1 --2- 172.21.15.49:0/2493530410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce480d0b10 0x7fce480d0f90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce38009840 tx=0x7fce38009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d21fa47aee53e4a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:08.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.757+0000 7fce3ffff700 1 -- 172.21.15.49:0/2493530410 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38011830 con 0x7fce480d0b10 2023-07-18T03:41:08.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.757+0000 7fce3ffff700 1 -- 172.21.15.49:0/2493530410 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce38011990 con 0x7fce480d0b10 2023-07-18T03:41:08.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.757+0000 7fce3ffff700 1 -- 172.21.15.49:0/2493530410 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38011d20 con 0x7fce480d0b10 2023-07-18T03:41:08.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.758+0000 7fce4f9ff700 1 -- 172.21.15.49:0/2493530410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce480d0b10 msgr2=0x7fce480d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.758+0000 7fce4f9ff700 1 --2- 172.21.15.49:0/2493530410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce480d0b10 0x7fce480d0f90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce38009840 tx=0x7fce38009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:08.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.758+0000 7fce4f9ff700 1 -- 172.21.15.49:0/2493530410 shutdown_connections 2023-07-18T03:41:08.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.758+0000 7fce4f9ff700 1 --2- 172.21.15.49:0/2493530410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce480d0b10 0x7fce480d0f90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.758+0000 7fce4f9ff700 1 -- 172.21.15.49:0/2493530410 >> 172.21.15.49:0/2493530410 conn(0x7fce4809c1b0 msgr2=0x7fce480d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:08.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.758+0000 7fce4f9ff700 1 -- 172.21.15.49:0/2493530410 shutdown_connections 2023-07-18T03:41:08.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.758+0000 7fce4f9ff700 1 -- 172.21.15.49:0/2493530410 wait complete. 2023-07-18T03:41:08.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.760+0000 7fce4f9ff700 1 Processor -- start 2023-07-18T03:41:08.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.760+0000 7fce4f9ff700 1 -- start start 2023-07-18T03:41:08.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4f9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce480b0e20 0x7fce48149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:08.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4f9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce480b8a60 0x7fce480cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:08.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4f9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce480d0b10 0x7fce480cf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:08.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4f9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce480cdf10 con 0x7fce480b8a60 2023-07-18T03:41:08.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4f9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce480cfba0 con 0x7fce480b0e20 2023-07-18T03:41:08.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4f9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce480cfe80 con 0x7fce480d0b10 2023-07-18T03:41:08.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4cf9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce480b8a60 0x7fce480cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:08.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4cf9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce480b8a60 0x7fce480cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58406/0 (socket says 172.21.15.49:58406) 2023-07-18T03:41:08.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4cf9a700 1 -- 172.21.15.49:0/3178688830 learned_addr learned my addr 172.21.15.49:0/3178688830 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:08.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4d79b700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce480b0e20 0x7fce48149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:08.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4df9c700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce480d0b10 0x7fce480cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:08.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4cf9a700 1 -- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce480d0b10 msgr2=0x7fce480cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4cf9a700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce480d0b10 0x7fce480cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4cf9a700 1 -- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce480b0e20 msgr2=0x7fce48149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:08.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4cf9a700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce480b0e20 0x7fce48149b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:08.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.761+0000 7fce4cf9a700 1 -- 172.21.15.49:0/3178688830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce380094a0 con 0x7fce480b8a60 2023-07-18T03:41:08.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.762+0000 7fce4df9c700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce480d0b10 0x7fce480cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:08.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.762+0000 7fce4d79b700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce480b0e20 0x7fce48149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:08.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.762+0000 7fce4cf9a700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce480b8a60 0x7fce480cc9a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fce38009b90 tx=0x7fce38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:08.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.762+0000 7fce3dffb700 1 -- 172.21.15.49:0/3178688830 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce380038a0 con 0x7fce480b8a60 2023-07-18T03:41:08.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.762+0000 7fce3dffb700 1 -- 172.21.15.49:0/3178688830 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3801e070 con 0x7fce480b8a60 2023-07-18T03:41:08.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.762+0000 7fce3dffb700 1 -- 172.21.15.49:0/3178688830 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38003c40 con 0x7fce480b8a60 2023-07-18T03:41:08.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.762+0000 7fce4f9ff700 1 -- 172.21.15.49:0/3178688830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce480d0160 con 0x7fce480b8a60 2023-07-18T03:41:08.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.762+0000 7fce4f9ff700 1 -- 172.21.15.49:0/3178688830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce480d0720 con 0x7fce480b8a60 2023-07-18T03:41:08.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.763+0000 7fce4f9ff700 1 -- 172.21.15.49:0/3178688830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce480b63e0 con 0x7fce480b8a60 2023-07-18T03:41:08.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.764+0000 7fce3dffb700 1 -- 172.21.15.49:0/3178688830 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fce3802d450 con 0x7fce480b8a60 2023-07-18T03:41:08.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.765+0000 7fce3dffb700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fce2c06ae00 0x7fce2c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:08.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.765+0000 7fce3dffb700 1 -- 172.21.15.49:0/3178688830 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6176+0+0 (secure 0 0 0) 0x7fce3808a790 con 0x7fce480b8a60 2023-07-18T03:41:08.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.766+0000 7fce4d79b700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fce2c06ae00 0x7fce2c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:08.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.766+0000 7fce4d79b700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fce2c06ae00 0x7fce2c06d2b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fce440011e0 tx=0x7fce44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:08.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:08.769+0000 7fce3dffb700 1 -- 172.21.15.49:0/3178688830 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fce38055640 con 0x7fce480b8a60 2023-07-18T03:41:09.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.023+0000 7fce4f9ff700 1 -- 172.21.15.49:0/3178688830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fce480d0910 con 0x7fce480b8a60 2023-07-18T03:41:09.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.524+0000 7fce3dffb700 1 -- 172.21.15.49:0/3178688830 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fce38058c60 con 0x7fce480b8a60 2023-07-18T03:41:09.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.527+0000 7fce4f9ff700 1 -- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fce2c06ae00 msgr2=0x7fce2c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:09.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.527+0000 7fce4f9ff700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fce2c06ae00 0x7fce2c06d2b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fce440011e0 tx=0x7fce44009040 comp rx=0 tx=0).stop 2023-07-18T03:41:09.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.527+0000 7fce4f9ff700 1 -- 172.21.15.49:0/3178688830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce480b8a60 msgr2=0x7fce480cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:09.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.527+0000 7fce4f9ff700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce480b8a60 0x7fce480cc9a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fce38009b90 tx=0x7fce38004750 comp rx=0 tx=0).stop 2023-07-18T03:41:09.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.527+0000 7fce4f9ff700 1 -- 172.21.15.49:0/3178688830 shutdown_connections 2023-07-18T03:41:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.527+0000 7fce4f9ff700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce480d0b10 0x7fce480cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.527+0000 7fce4f9ff700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce480b0e20 0x7fce48149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.527+0000 7fce4f9ff700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fce2c06ae00 0x7fce2c06d2b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.527+0000 7fce4f9ff700 1 --2- 172.21.15.49:0/3178688830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce480b8a60 0x7fce480cc9a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.527+0000 7fce4f9ff700 1 -- 172.21.15.49:0/3178688830 >> 172.21.15.49:0/3178688830 conn(0x7fce4809c1b0 msgr2=0x7fce480d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.527+0000 7fce4f9ff700 1 -- 172.21.15.49:0/3178688830 shutdown_connections 2023-07-18T03:41:09.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.527+0000 7fce4f9ff700 1 -- 172.21.15.49:0/3178688830 wait complete. 2023-07-18T03:41:09.541 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-18T03:41:09.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.758+0000 7fa076bff700 1 Processor -- start 2023-07-18T03:41:09.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.758+0000 7fa076bff700 1 -- start start 2023-07-18T03:41:09.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.759+0000 7fa076bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa07015c8f0 0x7fa0700d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:09.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.759+0000 7fa076bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa0700b1740 0x7fa0700d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:09.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.759+0000 7fa076bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0700af9b0 con 0x7fa0700b1e10 2023-07-18T03:41:09.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.759+0000 7fa076bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0700b45b0 con 0x7fa0700b1740 2023-07-18T03:41:09.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.759+0000 7fa076bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0700afc50 con 0x7fa07015c8f0 2023-07-18T03:41:09.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.760+0000 7fa06ffff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa0700b1e10 0x7fa0700b2740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49530/0 (socket says 172.21.15.49:49530) 2023-07-18T03:41:09.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.760+0000 7fa06ffff700 1 -- 172.21.15.49:0/4003636794 learned_addr learned my addr 172.21.15.49:0/4003636794 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:09.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.760+0000 7fa07519c700 1 --2- 172.21.15.49:0/4003636794 >> v2:172.21.15.49:3300/0 conn(0x7fa0700b1740 0x7fa0700d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:09.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.760+0000 7fa07499b700 1 --2- 172.21.15.49:0/4003636794 >> v2:172.21.15.107:3300/0 conn(0x7fa07015c8f0 0x7fa0700d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:09.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.760+0000 7fa06f7fe700 1 -- 172.21.15.49:0/4003636794 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213436644 0 0) 0x7fa0700af9b0 con 0x7fa0700b1e10 2023-07-18T03:41:09.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.760+0000 7fa06f7fe700 1 -- 172.21.15.49:0/4003636794 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0600035b0 con 0x7fa0700b1e10 2023-07-18T03:41:09.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.760+0000 7fa07519c700 1 -- 172.21.15.49:0/4003636794 >> v2:172.21.15.107:3300/0 conn(0x7fa07015c8f0 msgr2=0x7fa0700d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:09.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.760+0000 7fa07519c700 1 --2- 172.21.15.49:0/4003636794 >> v2:172.21.15.107:3300/0 conn(0x7fa07015c8f0 0x7fa0700d09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:09.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.760+0000 7fa07519c700 1 -- 172.21.15.49:0/4003636794 >> v1:172.21.15.49:6789/0 conn(0x7fa0700b1e10 legacy=0x7fa0700b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:09.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.760+0000 7fa07519c700 1 -- 172.21.15.49:0/4003636794 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0700aff30 con 0x7fa0700b1740 2023-07-18T03:41:09.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.761+0000 7fa07519c700 1 --2- 172.21.15.49:0/4003636794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0700b1740 0x7fa0700d0ef0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa068009d90 tx=0x7fa06800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d2b44c6b2eb075 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:09.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.761+0000 7fa06f7fe700 1 -- 172.21.15.49:0/4003636794 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06800e070 con 0x7fa0700b1740 2023-07-18T03:41:09.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.761+0000 7fa06f7fe700 1 -- 172.21.15.49:0/4003636794 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06800bae0 con 0x7fa0700b1740 2023-07-18T03:41:09.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.761+0000 7fa06f7fe700 1 -- 172.21.15.49:0/4003636794 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06800be90 con 0x7fa0700b1740 2023-07-18T03:41:09.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.761+0000 7fa076bff700 1 -- 172.21.15.49:0/4003636794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0700b1740 msgr2=0x7fa0700d0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:09.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.761+0000 7fa076bff700 1 --2- 172.21.15.49:0/4003636794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0700b1740 0x7fa0700d0ef0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa068009d90 tx=0x7fa06800b330 comp rx=0 tx=0).stop 2023-07-18T03:41:09.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.761+0000 7fa076bff700 1 -- 172.21.15.49:0/4003636794 shutdown_connections 2023-07-18T03:41:09.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.762+0000 7fa076bff700 1 --2- 172.21.15.49:0/4003636794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0700b1740 0x7fa0700d0ef0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:09.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.762+0000 7fa076bff700 1 --2- 172.21.15.49:0/4003636794 >> v2:172.21.15.107:3300/0 conn(0x7fa07015c8f0 0x7fa0700d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:09.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.762+0000 7fa076bff700 1 -- 172.21.15.49:0/4003636794 >> 172.21.15.49:0/4003636794 conn(0x7fa07009c150 msgr2=0x7fa0700d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:09.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.762+0000 7fa076bff700 1 -- 172.21.15.49:0/4003636794 shutdown_connections 2023-07-18T03:41:09.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.762+0000 7fa076bff700 1 -- 172.21.15.49:0/4003636794 wait complete. 2023-07-18T03:41:09.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.763+0000 7fa076bff700 1 Processor -- start 2023-07-18T03:41:09.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.763+0000 7fa076bff700 1 -- start start 2023-07-18T03:41:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.764+0000 7fa076bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0700b1740 0x7fa07015b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.764+0000 7fa076bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0700b1e10 0x7fa0700c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:09.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.764+0000 7fa076bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07015c8f0 0x7fa0700c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:09.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.764+0000 7fa076bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700c89c0 con 0x7fa0700b1740 2023-07-18T03:41:09.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.764+0000 7fa076bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700b6160 con 0x7fa0700b1e10 2023-07-18T03:41:09.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.764+0000 7fa076bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700b6440 con 0x7fa07015c8f0 2023-07-18T03:41:09.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.765+0000 7fa07499b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0700b1740 0x7fa07015b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:09.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.765+0000 7fa07499b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0700b1740 0x7fa07015b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58172/0 (socket says 172.21.15.49:58172) 2023-07-18T03:41:09.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.765+0000 7fa07499b700 1 -- 172.21.15.49:0/750729089 learned_addr learned my addr 172.21.15.49:0/750729089 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.765+0000 7fa06ffff700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0700b1e10 0x7fa0700c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.765+0000 7fa07519c700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07015c8f0 0x7fa0700c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.765+0000 7fa07499b700 1 -- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07015c8f0 msgr2=0x7fa0700c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.765+0000 7fa07499b700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07015c8f0 0x7fa0700c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.765+0000 7fa07499b700 1 -- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0700b1e10 msgr2=0x7fa0700c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.765+0000 7fa07499b700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0700b1e10 0x7fa0700c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.765+0000 7fa07499b700 1 -- 172.21.15.49:0/750729089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa068009a70 con 0x7fa0700b1740 2023-07-18T03:41:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.765+0000 7fa07499b700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0700b1740 0x7fa07015b100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa06400e870 tx=0x7fa06400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.766+0000 7fa06d7fa700 1 -- 172.21.15.49:0/750729089 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064009590 con 0x7fa0700b1740 2023-07-18T03:41:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.766+0000 7fa06d7fa700 1 -- 172.21.15.49:0/750729089 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06400ee70 con 0x7fa0700b1740 2023-07-18T03:41:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.766+0000 7fa076bff700 1 -- 172.21.15.49:0/750729089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0700b6720 con 0x7fa0700b1740 2023-07-18T03:41:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.766+0000 7fa06d7fa700 1 -- 172.21.15.49:0/750729089 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06400ca50 con 0x7fa0700b1740 2023-07-18T03:41:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.766+0000 7fa076bff700 1 -- 172.21.15.49:0/750729089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0700b7480 con 0x7fa0700b1740 2023-07-18T03:41:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.767+0000 7fa076bff700 1 -- 172.21.15.49:0/750729089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0700b76a0 con 0x7fa0700b1740 2023-07-18T03:41:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.768+0000 7fa06d7fa700 1 -- 172.21.15.49:0/750729089 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fa06400cbb0 con 0x7fa0700b1740 2023-07-18T03:41:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.769+0000 7fa06d7fa700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa05406ae00 0x7fa05406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.769+0000 7fa06d7fa700 1 -- 172.21.15.49:0/750729089 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6176+0+0 (secure 0 0 0) 0x7fa064089710 con 0x7fa0700b1740 2023-07-18T03:41:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.769+0000 7fa06ffff700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa05406ae00 0x7fa05406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:09.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.770+0000 7fa06ffff700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa05406ae00 0x7fa05406d2b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa070159bc0 tx=0x7fa05c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:09.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.773+0000 7fa06d7fa700 1 -- 172.21.15.49:0/750729089 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa064058050 con 0x7fa0700b1740 2023-07-18T03:41:09.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:09.982+0000 7fa076bff700 1 -- 172.21.15.49:0/750729089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa070074350 con 0x7fa0700b1740 2023-07-18T03:41:10.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.928+0000 7fa06d7fa700 1 -- 172.21.15.49:0/750729089 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa06400ce60 con 0x7fa0700b1740 2023-07-18T03:41:10.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.932+0000 7fa076bff700 1 -- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa05406ae00 msgr2=0x7fa05406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:10.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.932+0000 7fa076bff700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa05406ae00 0x7fa05406d2b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa070159bc0 tx=0x7fa05c009040 comp rx=0 tx=0).stop 2023-07-18T03:41:10.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.932+0000 7fa076bff700 1 -- 172.21.15.49:0/750729089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0700b1740 msgr2=0x7fa07015b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:10.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.932+0000 7fa076bff700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0700b1740 0x7fa07015b100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa06400e870 tx=0x7fa06400ec30 comp rx=0 tx=0).stop 2023-07-18T03:41:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.932+0000 7fa076bff700 1 -- 172.21.15.49:0/750729089 shutdown_connections 2023-07-18T03:41:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.932+0000 7fa076bff700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa07015c8f0 0x7fa0700c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.932+0000 7fa076bff700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fa05406ae00 0x7fa05406d2b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.932+0000 7fa076bff700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0700b1740 0x7fa07015b100 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.932+0000 7fa076bff700 1 --2- 172.21.15.49:0/750729089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0700b1e10 0x7fa0700c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.932+0000 7fa076bff700 1 -- 172.21.15.49:0/750729089 >> 172.21.15.49:0/750729089 conn(0x7fa07009c150 msgr2=0x7fa0700c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.932+0000 7fa076bff700 1 -- 172.21.15.49:0/750729089 shutdown_connections 2023-07-18T03:41:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:10.933+0000 7fa076bff700 1 -- 172.21.15.49:0/750729089 wait complete. 2023-07-18T03:41:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2023-07-18T03:41:10.949 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-18T03:41:11.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.168+0000 7f05519ff700 1 Processor -- start 2023-07-18T03:41:11.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.168+0000 7f05519ff700 1 -- start start 2023-07-18T03:41:11.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f05519ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f054c0b0480 0x7f054c0cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f05519ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f054c15cb30 0x7f054c15cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f05519ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054c06b5c0 con 0x7f054c160c00 2023-07-18T03:41:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f05519ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054c06ca60 con 0x7f054c0b0480 2023-07-18T03:41:11.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f05519ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c0caa00 con 0x7f054c15cb30 2023-07-18T03:41:11.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f054affd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f054c0b0480 0x7f054c0cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:11.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f054a7fc700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f054c160c00 0x7f054c0b0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49536/0 (socket says 172.21.15.49:49536) 2023-07-18T03:41:11.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f054a7fc700 1 -- 172.21.15.49:0/1055109256 learned_addr learned my addr 172.21.15.49:0/1055109256 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f054affd700 1 --2- 172.21.15.49:0/1055109256 >> v2:172.21.15.49:3300/0 conn(0x7f054c0b0480 0x7f054c0cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58180/0 (socket says 172.21.15.49:58180) 2023-07-18T03:41:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f054b7fe700 1 --2- 172.21.15.49:0/1055109256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f054c15cb30 0x7f054c15cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f0549ffb700 1 -- 172.21.15.49:0/1055109256 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942023790 0 0) 0x7f054c06b5c0 con 0x7f054c160c00 2023-07-18T03:41:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f054affd700 1 -- 172.21.15.49:0/1055109256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f054c15cb30 msgr2=0x7f054c15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.170+0000 7f054affd700 1 --2- 172.21.15.49:0/1055109256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f054c15cb30 0x7f054c15cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.171+0000 7f054affd700 1 -- 172.21.15.49:0/1055109256 >> v1:172.21.15.49:6789/0 conn(0x7f054c160c00 legacy=0x7f054c0b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:11.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.171+0000 7f054affd700 1 -- 172.21.15.49:0/1055109256 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c0cdf10 con 0x7f054c0b0480 2023-07-18T03:41:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.171+0000 7f054affd700 1 --2- 172.21.15.49:0/1055109256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f054c0b0480 0x7f054c0cc740 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f053c009840 tx=0x7f053c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a3bd737f482df8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.171+0000 7f0549ffb700 1 -- 172.21.15.49:0/1055109256 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c011830 con 0x7f054c0b0480 2023-07-18T03:41:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.171+0000 7f0549ffb700 1 -- 172.21.15.49:0/1055109256 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c011990 con 0x7f054c0b0480 2023-07-18T03:41:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.171+0000 7f0549ffb700 1 -- 172.21.15.49:0/1055109256 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c011d20 con 0x7f054c0b0480 2023-07-18T03:41:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.172+0000 7f05519ff700 1 -- 172.21.15.49:0/1055109256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f054c0b0480 msgr2=0x7f054c0cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.172+0000 7f05519ff700 1 --2- 172.21.15.49:0/1055109256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f054c0b0480 0x7f054c0cc740 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f053c009840 tx=0x7f053c009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:11.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.172+0000 7f05519ff700 1 -- 172.21.15.49:0/1055109256 shutdown_connections 2023-07-18T03:41:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.172+0000 7f05519ff700 1 --2- 172.21.15.49:0/1055109256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f054c15cb30 0x7f054c15cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.172+0000 7f05519ff700 1 --2- 172.21.15.49:0/1055109256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f054c0b0480 0x7f054c0cc740 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.172+0000 7f05519ff700 1 -- 172.21.15.49:0/1055109256 >> 172.21.15.49:0/1055109256 conn(0x7f054c0cea00 msgr2=0x7f054c0cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.172+0000 7f05519ff700 1 -- 172.21.15.49:0/1055109256 shutdown_connections 2023-07-18T03:41:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.172+0000 7f05519ff700 1 -- 172.21.15.49:0/1055109256 wait complete. 2023-07-18T03:41:11.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.173+0000 7f05519ff700 1 Processor -- start 2023-07-18T03:41:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.174+0000 7f05519ff700 1 -- start start 2023-07-18T03:41:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f05519ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f054c0b0480 0x7f054c0c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f05519ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f054c15cb30 0x7f054c15e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f05519ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c160c00 0x7f054c15ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f05519ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c0c7640 con 0x7f054c0b0480 2023-07-18T03:41:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f05519ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c0c6a30 con 0x7f054c160c00 2023-07-18T03:41:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f05519ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c0c6d10 con 0x7f054c15cb30 2023-07-18T03:41:11.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f054affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f054c0b0480 0x7f054c0c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f054affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f054c0b0480 0x7f054c0c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58184/0 (socket says 172.21.15.49:58184) 2023-07-18T03:41:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f054affd700 1 -- 172.21.15.49:0/905367851 learned_addr learned my addr 172.21.15.49:0/905367851 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f054a7fc700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f054c15cb30 0x7f054c15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f054affd700 1 -- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f054c15cb30 msgr2=0x7f054c15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f054b7fe700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c160c00 0x7f054c15ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:11.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f054affd700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f054c15cb30 0x7f054c15e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:11.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f054affd700 1 -- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c160c00 msgr2=0x7f054c15ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:11.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f054affd700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c160c00 0x7f054c15ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:11.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f054affd700 1 -- 172.21.15.49:0/905367851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c0094a0 con 0x7f054c0b0480 2023-07-18T03:41:11.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f054b7fe700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c160c00 0x7f054c15ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:11.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.175+0000 7f054a7fc700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f054c15cb30 0x7f054c15e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.176+0000 7f054affd700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f054c0b0480 0x7f054c0c6450 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f053c009b90 tx=0x7f053c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.176+0000 7f0533fff700 1 -- 172.21.15.49:0/905367851 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c0036a0 con 0x7f054c0b0480 2023-07-18T03:41:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.176+0000 7f0533fff700 1 -- 172.21.15.49:0/905367851 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c01f070 con 0x7f054c0b0480 2023-07-18T03:41:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.176+0000 7f05519ff700 1 -- 172.21.15.49:0/905367851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c0c6ff0 con 0x7f054c0b0480 2023-07-18T03:41:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.176+0000 7f0533fff700 1 -- 172.21.15.49:0/905367851 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c003a40 con 0x7f054c0b0480 2023-07-18T03:41:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.176+0000 7f05519ff700 1 -- 172.21.15.49:0/905367851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c0c7ad0 con 0x7f054c0b0480 2023-07-18T03:41:11.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.177+0000 7f05519ff700 1 -- 172.21.15.49:0/905367851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c0c7920 con 0x7f054c0b0480 2023-07-18T03:41:11.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.178+0000 7f0533fff700 1 -- 172.21.15.49:0/905367851 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f053c003ba0 con 0x7f054c0b0480 2023-07-18T03:41:11.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.179+0000 7f0533fff700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f053806ae00 0x7f053806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:11.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.179+0000 7f0533fff700 1 -- 172.21.15.49:0/905367851 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f053c021070 con 0x7f054c0b0480 2023-07-18T03:41:11.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.179+0000 7f054a7fc700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f053806ae00 0x7f053806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:11.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.180+0000 7f054a7fc700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f053806ae00 0x7f053806d2b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0540007fd0 tx=0x7f0540009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:11.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.184+0000 7f0533fff700 1 -- 172.21.15.49:0/905367851 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f053c055fb0 con 0x7f054c0b0480 2023-07-18T03:41:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:11.395+0000 7f05519ff700 1 -- 172.21.15.49:0/905367851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f054c074350 con 0x7f054c0b0480 2023-07-18T03:41:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.005+0000 7f0533fff700 1 -- 172.21.15.49:0/905367851 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) v1 ==== 169+0+0 (secure 0 0 0) 0x7f053c0150d0 con 0x7f054c0b0480 2023-07-18T03:41:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.009+0000 7f05519ff700 1 -- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f053806ae00 msgr2=0x7f053806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:12.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.009+0000 7f05519ff700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f053806ae00 0x7f053806d2b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0540007fd0 tx=0x7f0540009040 comp rx=0 tx=0).stop 2023-07-18T03:41:12.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.009+0000 7f05519ff700 1 -- 172.21.15.49:0/905367851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f054c0b0480 msgr2=0x7f054c0c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:12.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.009+0000 7f05519ff700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f054c0b0480 0x7f054c0c6450 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f053c009b90 tx=0x7f053c004750 comp rx=0 tx=0).stop 2023-07-18T03:41:12.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.010+0000 7f05519ff700 1 -- 172.21.15.49:0/905367851 shutdown_connections 2023-07-18T03:41:12.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.010+0000 7f05519ff700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c160c00 0x7f054c15ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.010+0000 7f05519ff700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f053806ae00 0x7f053806d2b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.010+0000 7f05519ff700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f054c0b0480 0x7f054c0c6450 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.010+0000 7f05519ff700 1 --2- 172.21.15.49:0/905367851 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f054c15cb30 0x7f054c15e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.010+0000 7f05519ff700 1 -- 172.21.15.49:0/905367851 >> 172.21.15.49:0/905367851 conn(0x7f054c0cea00 msgr2=0x7f054c0acc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:12.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.010+0000 7f05519ff700 1 -- 172.21.15.49:0/905367851 shutdown_connections 2023-07-18T03:41:12.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.010+0000 7f05519ff700 1 -- 172.21.15.49:0/905367851 wait complete. 2023-07-18T03:41:12.006 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2023-07-18T03:41:12.025 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-18T03:41:12.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.240+0000 7fba5e1ff700 1 Processor -- start 2023-07-18T03:41:12.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.240+0000 7fba5e1ff700 1 -- start start 2023-07-18T03:41:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.241+0000 7fba5e1ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fba580b8660 0x7fba580cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.241+0000 7fba5e1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fba580b4f70 0x7fba580d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:12.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.241+0000 7fba5e1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba580d02a0 con 0x7fba580cd970 2023-07-18T03:41:12.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.241+0000 7fba5e1ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba580cfce0 con 0x7fba580b8660 2023-07-18T03:41:12.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.241+0000 7fba5e1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba580cffc0 con 0x7fba580b4f70 2023-07-18T03:41:12.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.242+0000 7fba577fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fba580b8660 0x7fba580cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:12.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.242+0000 7fba577fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fba580b8660 0x7fba580cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58196/0 (socket says 172.21.15.49:58196) 2023-07-18T03:41:12.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.242+0000 7fba577fe700 1 -- 172.21.15.49:0/2399374517 learned_addr learned my addr 172.21.15.49:0/2399374517 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:12.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.242+0000 7fba57fff700 1 --2- 172.21.15.49:0/2399374517 >> v2:172.21.15.107:3300/0 conn(0x7fba580b4f70 0x7fba580d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:12.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.242+0000 7fba577fe700 1 -- 172.21.15.49:0/2399374517 >> v2:172.21.15.107:3300/0 conn(0x7fba580b4f70 msgr2=0x7fba580d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:12.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.242+0000 7fba577fe700 1 --2- 172.21.15.49:0/2399374517 >> v2:172.21.15.107:3300/0 conn(0x7fba580b4f70 0x7fba580d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.242+0000 7fba577fe700 1 -- 172.21.15.49:0/2399374517 >> v1:172.21.15.107:6789/0 conn(0x7fba580cd970 legacy=0x7fba580b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:12.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.242+0000 7fba577fe700 1 -- 172.21.15.49:0/2399374517 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba580d0ac0 con 0x7fba580b8660 2023-07-18T03:41:12.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.242+0000 7fba577fe700 1 --2- 172.21.15.49:0/2399374517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba580b8660 0x7fba580cd430 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fba580b8f20 tx=0x7fba48009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4aeb78fa2acfe0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:12.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.243+0000 7fba567fc700 1 -- 172.21.15.49:0/2399374517 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48011830 con 0x7fba580b8660 2023-07-18T03:41:12.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.243+0000 7fba567fc700 1 -- 172.21.15.49:0/2399374517 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba48011990 con 0x7fba580b8660 2023-07-18T03:41:12.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.243+0000 7fba567fc700 1 -- 172.21.15.49:0/2399374517 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48011d20 con 0x7fba580b8660 2023-07-18T03:41:12.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.243+0000 7fba5e1ff700 1 -- 172.21.15.49:0/2399374517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba580b8660 msgr2=0x7fba580cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:12.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.243+0000 7fba5e1ff700 1 --2- 172.21.15.49:0/2399374517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba580b8660 0x7fba580cd430 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fba580b8f20 tx=0x7fba48009bd0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.243+0000 7fba5e1ff700 1 -- 172.21.15.49:0/2399374517 shutdown_connections 2023-07-18T03:41:12.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.243+0000 7fba5e1ff700 1 --2- 172.21.15.49:0/2399374517 >> v2:172.21.15.107:3300/0 conn(0x7fba580b4f70 0x7fba580d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.243+0000 7fba5e1ff700 1 --2- 172.21.15.49:0/2399374517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba580b8660 0x7fba580cd430 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.243+0000 7fba5e1ff700 1 -- 172.21.15.49:0/2399374517 >> 172.21.15.49:0/2399374517 conn(0x7fba58066860 msgr2=0x7fba5806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:12.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.243+0000 7fba5e1ff700 1 -- 172.21.15.49:0/2399374517 shutdown_connections 2023-07-18T03:41:12.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.243+0000 7fba5e1ff700 1 -- 172.21.15.49:0/2399374517 wait complete. 2023-07-18T03:41:12.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.246+0000 7fba5e1ff700 1 Processor -- start 2023-07-18T03:41:12.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.246+0000 7fba5e1ff700 1 -- start start 2023-07-18T03:41:12.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.247+0000 7fba5e1ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba580b4f70 0x7fba58158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:12.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.247+0000 7fba5e1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba580b8660 0x7fba5815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.247+0000 7fba5e1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba580cd970 0x7fba5815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.247+0000 7fba5e1ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba58158cd0 con 0x7fba580b4f70 2023-07-18T03:41:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.247+0000 7fba5e1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba580ca3b0 con 0x7fba580b8660 2023-07-18T03:41:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.247+0000 7fba5e1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba580ca690 con 0x7fba580cd970 2023-07-18T03:41:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.248+0000 7fba577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba580b4f70 0x7fba58158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.248+0000 7fba577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba580b4f70 0x7fba58158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58204/0 (socket says 172.21.15.49:58204) 2023-07-18T03:41:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.248+0000 7fba577fe700 1 -- 172.21.15.49:0/4116883708 learned_addr learned my addr 172.21.15.49:0/4116883708 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:12.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.248+0000 7fba56ffd700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba580b8660 0x7fba5815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.248+0000 7fba57fff700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba580cd970 0x7fba5815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.248+0000 7fba577fe700 1 -- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba580cd970 msgr2=0x7fba5815e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.248+0000 7fba577fe700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba580cd970 0x7fba5815e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.248+0000 7fba577fe700 1 -- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba580b8660 msgr2=0x7fba5815dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.248+0000 7fba577fe700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba580b8660 0x7fba5815dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.248+0000 7fba577fe700 1 -- 172.21.15.49:0/4116883708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba48009550 con 0x7fba580b4f70 2023-07-18T03:41:12.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.248+0000 7fba577fe700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba580b4f70 0x7fba58158790 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fba48000c00 tx=0x7fba48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:12.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.249+0000 7fba3ffff700 1 -- 172.21.15.49:0/4116883708 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48004570 con 0x7fba580b4f70 2023-07-18T03:41:12.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.249+0000 7fba3ffff700 1 -- 172.21.15.49:0/4116883708 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4801f070 con 0x7fba580b4f70 2023-07-18T03:41:12.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.249+0000 7fba3ffff700 1 -- 172.21.15.49:0/4116883708 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba480038e0 con 0x7fba580b4f70 2023-07-18T03:41:12.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.249+0000 7fba5e1ff700 1 -- 172.21.15.49:0/4116883708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba580ca970 con 0x7fba580b4f70 2023-07-18T03:41:12.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.249+0000 7fba5e1ff700 1 -- 172.21.15.49:0/4116883708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba581e7ba0 con 0x7fba580b4f70 2023-07-18T03:41:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.250+0000 7fba5e1ff700 1 -- 172.21.15.49:0/4116883708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba58074350 con 0x7fba580b4f70 2023-07-18T03:41:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.251+0000 7fba3ffff700 1 -- 172.21.15.49:0/4116883708 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fba48021030 con 0x7fba580b4f70 2023-07-18T03:41:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.252+0000 7fba3ffff700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fba4406ae00 0x7fba4406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:12.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.252+0000 7fba3ffff700 1 -- 172.21.15.49:0/4116883708 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fba4808a7d0 con 0x7fba580b4f70 2023-07-18T03:41:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.252+0000 7fba56ffd700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fba4406ae00 0x7fba4406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:12.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.253+0000 7fba56ffd700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fba4406ae00 0x7fba4406d2b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fba580b8290 tx=0x7fba4c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:12.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.256+0000 7fba3ffff700 1 -- 172.21.15.49:0/4116883708 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fba48055a80 con 0x7fba580b4f70 2023-07-18T03:41:12.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.466+0000 7fba5e1ff700 1 -- 172.21.15.49:0/4116883708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fba580b2150 con 0x7fba580b4f70 2023-07-18T03:41:12.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.470+0000 7fba3ffff700 1 -- 172.21.15.49:0/4116883708 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fba480590a0 con 0x7fba580b4f70 2023-07-18T03:41:12.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.472+0000 7fba5e1ff700 1 -- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fba4406ae00 msgr2=0x7fba4406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:12.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.472+0000 7fba5e1ff700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fba4406ae00 0x7fba4406d2b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fba580b8290 tx=0x7fba4c009040 comp rx=0 tx=0).stop 2023-07-18T03:41:12.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.472+0000 7fba5e1ff700 1 -- 172.21.15.49:0/4116883708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba580b4f70 msgr2=0x7fba58158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:12.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.472+0000 7fba5e1ff700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba580b4f70 0x7fba58158790 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fba48000c00 tx=0x7fba48004750 comp rx=0 tx=0).stop 2023-07-18T03:41:12.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.472+0000 7fba5e1ff700 1 -- 172.21.15.49:0/4116883708 shutdown_connections 2023-07-18T03:41:12.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.472+0000 7fba5e1ff700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba580cd970 0x7fba5815e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.472+0000 7fba5e1ff700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fba4406ae00 0x7fba4406d2b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.472+0000 7fba5e1ff700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba580b4f70 0x7fba58158790 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.472+0000 7fba5e1ff700 1 --2- 172.21.15.49:0/4116883708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba580b8660 0x7fba5815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.472+0000 7fba5e1ff700 1 -- 172.21.15.49:0/4116883708 >> 172.21.15.49:0/4116883708 conn(0x7fba58066860 msgr2=0x7fba580d2540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:12.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.472+0000 7fba5e1ff700 1 -- 172.21.15.49:0/4116883708 shutdown_connections 2023-07-18T03:41:12.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.472+0000 7fba5e1ff700 1 -- 172.21.15.49:0/4116883708 wait complete. 2023-07-18T03:41:12.485 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2023-07-18T03:41:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.697+0000 7fb497fff700 1 Processor -- start 2023-07-18T03:41:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.697+0000 7fb497fff700 1 -- start start 2023-07-18T03:41:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.699+0000 7fb497fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb49015e890 0x7fb4900cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.699+0000 7fb497fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb49015afd0 0x7fb4900b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.699+0000 7fb497fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4900ce110 con 0x7fb4900b3780 2023-07-18T03:41:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.699+0000 7fb497fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49015b450 con 0x7fb49015afd0 2023-07-18T03:41:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.699+0000 7fb497fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900cf380 con 0x7fb49015e890 2023-07-18T03:41:12.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.699+0000 7fb49659c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb49015afd0 0x7fb4900b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:12.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.699+0000 7fb49659c700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb49015afd0 0x7fb4900b3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58212/0 (socket says 172.21.15.49:58212) 2023-07-18T03:41:12.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.699+0000 7fb49659c700 1 -- 172.21.15.49:0/499398045 learned_addr learned my addr 172.21.15.49:0/499398045 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:12.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.699+0000 7fb495d9b700 1 --2- 172.21.15.49:0/499398045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb49015e890 0x7fb4900cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:12.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.700+0000 7fb49659c700 1 -- 172.21.15.49:0/499398045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb49015e890 msgr2=0x7fb4900cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:12.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.700+0000 7fb49659c700 1 --2- 172.21.15.49:0/499398045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb49015e890 0x7fb4900cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.700+0000 7fb49659c700 1 -- 172.21.15.49:0/499398045 >> v1:172.21.15.107:6789/0 conn(0x7fb4900b3780 legacy=0x7fb49015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.700+0000 7fb49659c700 1 -- 172.21.15.49:0/499398045 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49015b660 con 0x7fb49015afd0 2023-07-18T03:41:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.700+0000 7fb49659c700 1 --2- 172.21.15.49:0/499398045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb49015afd0 0x7fb4900b3bf0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb488009e50 tx=0x7fb48800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7722465efa180d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.700+0000 7fb494d99700 1 -- 172.21.15.49:0/499398045 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48800e070 con 0x7fb49015afd0 2023-07-18T03:41:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.700+0000 7fb494d99700 1 -- 172.21.15.49:0/499398045 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48800b910 con 0x7fb49015afd0 2023-07-18T03:41:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.700+0000 7fb494d99700 1 -- 172.21.15.49:0/499398045 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48800bcc0 con 0x7fb49015afd0 2023-07-18T03:41:12.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.701+0000 7fb497fff700 1 -- 172.21.15.49:0/499398045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb49015afd0 msgr2=0x7fb4900b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:12.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.701+0000 7fb497fff700 1 --2- 172.21.15.49:0/499398045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb49015afd0 0x7fb4900b3bf0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb488009e50 tx=0x7fb48800b330 comp rx=0 tx=0).stop 2023-07-18T03:41:12.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.701+0000 7fb497fff700 1 -- 172.21.15.49:0/499398045 shutdown_connections 2023-07-18T03:41:12.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.701+0000 7fb497fff700 1 --2- 172.21.15.49:0/499398045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb49015afd0 0x7fb4900b3bf0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.701+0000 7fb497fff700 1 --2- 172.21.15.49:0/499398045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb49015e890 0x7fb4900cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.701+0000 7fb497fff700 1 -- 172.21.15.49:0/499398045 >> 172.21.15.49:0/499398045 conn(0x7fb490069f30 msgr2=0x7fb49006a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:12.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.701+0000 7fb497fff700 1 -- 172.21.15.49:0/499398045 shutdown_connections 2023-07-18T03:41:12.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.701+0000 7fb497fff700 1 -- 172.21.15.49:0/499398045 wait complete. 2023-07-18T03:41:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.703+0000 7fb497fff700 1 Processor -- start 2023-07-18T03:41:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.703+0000 7fb497fff700 1 -- start start 2023-07-18T03:41:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb497fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4900b3780 0x7fb49013e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb497fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb49015afd0 0x7fb49014f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb497fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb49015e890 0x7fb49014f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb497fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49013e9b0 con 0x7fb4900b3780 2023-07-18T03:41:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb497fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900af0f0 con 0x7fb49015e890 2023-07-18T03:41:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb497fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900b1110 con 0x7fb49015afd0 2023-07-18T03:41:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb495d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4900b3780 0x7fb49013e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:12.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb495d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4900b3780 0x7fb49013e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58216/0 (socket says 172.21.15.49:58216) 2023-07-18T03:41:12.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb495d9b700 1 -- 172.21.15.49:0/2924260425 learned_addr learned my addr 172.21.15.49:0/2924260425 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:12.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb49659c700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb49015e890 0x7fb49014f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:12.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb49559a700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb49015afd0 0x7fb49014f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:12.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb495d9b700 1 -- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb49015afd0 msgr2=0x7fb49014f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:12.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.704+0000 7fb495d9b700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb49015afd0 0x7fb49014f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.705+0000 7fb495d9b700 1 -- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb49015e890 msgr2=0x7fb49014f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:12.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.705+0000 7fb495d9b700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb49015e890 0x7fb49014f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:12.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.705+0000 7fb495d9b700 1 -- 172.21.15.49:0/2924260425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb488009b30 con 0x7fb4900b3780 2023-07-18T03:41:12.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.705+0000 7fb49659c700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb49015e890 0x7fb49014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:12.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.705+0000 7fb495d9b700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4900b3780 0x7fb49013e470 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00e9b0 tx=0x7fb48c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:12.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.705+0000 7fb4867fc700 1 -- 172.21.15.49:0/2924260425 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c009710 con 0x7fb4900b3780 2023-07-18T03:41:12.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.705+0000 7fb4867fc700 1 -- 172.21.15.49:0/2924260425 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48c00c850 con 0x7fb4900b3780 2023-07-18T03:41:12.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.705+0000 7fb4867fc700 1 -- 172.21.15.49:0/2924260425 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c00cbc0 con 0x7fb4900b3780 2023-07-18T03:41:12.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.705+0000 7fb497fff700 1 -- 172.21.15.49:0/2924260425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4900af340 con 0x7fb4900b3780 2023-07-18T03:41:12.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.705+0000 7fb497fff700 1 -- 172.21.15.49:0/2924260425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4900af900 con 0x7fb4900b3780 2023-07-18T03:41:12.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.706+0000 7fb497fff700 1 -- 172.21.15.49:0/2924260425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49004e280 con 0x7fb4900b3780 2023-07-18T03:41:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.707+0000 7fb4867fc700 1 -- 172.21.15.49:0/2924260425 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fb48c00cd20 con 0x7fb4900b3780 2023-07-18T03:41:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.708+0000 7fb4867fc700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb47406ae00 0x7fb47406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.708+0000 7fb4867fc700 1 -- 172.21.15.49:0/2924260425 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb48c089400 con 0x7fb4900b3780 2023-07-18T03:41:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.709+0000 7fb49559a700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb47406ae00 0x7fb47406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:12.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.709+0000 7fb49559a700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb47406ae00 0x7fb47406d2b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb480000f60 tx=0x7fb480009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:12.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:12.713+0000 7fb4867fc700 1 -- 172.21.15.49:0/2924260425 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb48c058140 con 0x7fb4900b3780 2023-07-18T03:41:12.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:13.004+0000 7fb497fff700 1 -- 172.21.15.49:0/2924260425 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7fb4900afc70 con 0x7fb4900b3780 2023-07-18T03:41:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.013+0000 7fb4867fc700 1 -- 172.21.15.49:0/2924260425 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v170) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb48c057cd0 con 0x7fb4900b3780 2023-07-18T03:41:14.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.017+0000 7fb497fff700 1 -- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb47406ae00 msgr2=0x7fb47406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:14.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.017+0000 7fb497fff700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb47406ae00 0x7fb47406d2b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb480000f60 tx=0x7fb480009040 comp rx=0 tx=0).stop 2023-07-18T03:41:14.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.017+0000 7fb497fff700 1 -- 172.21.15.49:0/2924260425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4900b3780 msgr2=0x7fb49013e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:14.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.017+0000 7fb497fff700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4900b3780 0x7fb49013e470 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00e9b0 tx=0x7fb48c00ed70 comp rx=0 tx=0).stop 2023-07-18T03:41:14.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.017+0000 7fb497fff700 1 -- 172.21.15.49:0/2924260425 shutdown_connections 2023-07-18T03:41:14.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.017+0000 7fb497fff700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb49015e890 0x7fb49014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:14.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.017+0000 7fb497fff700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fb47406ae00 0x7fb47406d2b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:14.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.017+0000 7fb497fff700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4900b3780 0x7fb49013e470 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:14.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.017+0000 7fb497fff700 1 --2- 172.21.15.49:0/2924260425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb49015afd0 0x7fb49014f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:14.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.017+0000 7fb497fff700 1 -- 172.21.15.49:0/2924260425 >> 172.21.15.49:0/2924260425 conn(0x7fb490069f30 msgr2=0x7fb490154780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:14.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.017+0000 7fb497fff700 1 -- 172.21.15.49:0/2924260425 shutdown_connections 2023-07-18T03:41:14.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:14.017+0000 7fb497fff700 1 -- 172.21.15.49:0/2924260425 wait complete. 2023-07-18T03:41:14.034 INFO:tasks.cephfs_test_runner:test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ERROR 2023-07-18T03:41:14.035 INFO:tasks.cephfs_test_runner: 2023-07-18T03:41:14.035 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-18T03:41:14.035 INFO:tasks.cephfs_test_runner:ERROR: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-07-18T03:41:14.035 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-18T03:41:14.036 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-18T03:41:14.036 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/cephfs/test_volumes.py", line 5576, in test_clean_stale_subvolume_snapshot_metadata 2023-07-18T03:41:14.036 INFO:tasks.cephfs_test_runner: _ = json.loads(self._get_subvolume_info(self.volname, subvolname, group)) 2023-07-18T03:41:14.036 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/cephfs/test_volumes.py", line 245, in _get_subvolume_info 2023-07-18T03:41:14.037 INFO:tasks.cephfs_test_runner: subvol_md = self._fs_cmd(*args) 2023-07-18T03:41:14.037 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-07-18T03:41:14.037 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-07-18T03:41:14.037 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd 2023-07-18T03:41:14.038 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-07-18T03:41:14.038 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd 2023-07-18T03:41:14.038 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-07-18T03:41:14.038 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/remote.py", line 522, in run 2023-07-18T03:41:14.038 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-07-18T03:41:14.038 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/run.py", line 455, in run 2023-07-18T03:41:14.039 INFO:tasks.cephfs_test_runner: r.wait() 2023-07-18T03:41:14.039 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/run.py", line 161, in wait 2023-07-18T03:41:14.039 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-07-18T03:41:14.039 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-07-18T03:41:14.039 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-07-18T03:41:14.040 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi049 with status 61: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvolume_0000000000974124 group_0000000000030717' 2023-07-18T03:41:14.040 INFO:tasks.cephfs_test_runner: 2023-07-18T03:41:14.040 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-18T03:41:14.040 INFO:tasks.cephfs_test_runner:Ran 7 tests in 115.864s 2023-07-18T03:41:14.040 INFO:tasks.cephfs_test_runner: 2023-07-18T03:41:14.041 INFO:tasks.cephfs_test_runner:FAILED (errors=1, skipped=5) 2023-07-18T03:41:14.041 INFO:tasks.cephfs_test_runner: 2023-07-18T03:41:14.041 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-18T03:41:14.041 INFO:tasks.cephfs_test_runner:ERROR: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-07-18T03:41:14.041 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-18T03:41:14.042 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-18T03:41:14.042 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/cephfs/test_volumes.py", line 5576, in test_clean_stale_subvolume_snapshot_metadata 2023-07-18T03:41:14.042 INFO:tasks.cephfs_test_runner: _ = json.loads(self._get_subvolume_info(self.volname, subvolname, group)) 2023-07-18T03:41:14.042 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/cephfs/test_volumes.py", line 245, in _get_subvolume_info 2023-07-18T03:41:14.042 INFO:tasks.cephfs_test_runner: subvol_md = self._fs_cmd(*args) 2023-07-18T03:41:14.043 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-07-18T03:41:14.043 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-07-18T03:41:14.043 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/ceph_manager.py", line 1597, in raw_cluster_cmd 2023-07-18T03:41:14.043 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-07-18T03:41:14.043 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/ceph_manager.py", line 1588, in run_cluster_cmd 2023-07-18T03:41:14.043 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-07-18T03:41:14.044 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/remote.py", line 522, in run 2023-07-18T03:41:14.044 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-07-18T03:41:14.044 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/run.py", line 455, in run 2023-07-18T03:41:14.044 INFO:tasks.cephfs_test_runner: r.wait() 2023-07-18T03:41:14.044 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/run.py", line 161, in wait 2023-07-18T03:41:14.045 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-07-18T03:41:14.045 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-07-18T03:41:14.045 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-07-18T03:41:14.045 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi049 with status 61: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvolume_0000000000974124 group_0000000000030717' 2023-07-18T03:41:14.045 INFO:tasks.cephfs_test_runner: 2023-07-18T03:41:14.046 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-07-18T03:41:14.107 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9fd979daa84f4190bd86dceeab7afd6e Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-07-18T03:41:14.109 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-07-18T03:41:14.117 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-18T03:41:14.137 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-18T03:41:14.138 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:41:14.138 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:41:14.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345544k,nr_inodes=4086386,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-18T03:41:14.190 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-07-18T03:41:14.190 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:41:14.276 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345544k,nr_inodes=4086386,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-18T03:41:14.277 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:41:14.278 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:41:14.326 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345444k,nr_inodes=4086361,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-18T03:41:14.327 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-18T03:41:14.327 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:41:14.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345444k,nr_inodes=4086361,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-18T03:41:14.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:14.412 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-07-18T03:41:14.459 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-18T03:41:14.459 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-18T03:41:14.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:14.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-18T03:41:14.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:14.575 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-18T03:41:14.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:14.657 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2023-07-18T03:41:14.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:14.799 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-07-18T03:41:14.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:14.849 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2023-07-18T03:41:14.924 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2023-07-18T03:41:14.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:41:14.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:14.929 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-18T03:41:14.978 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-18T03:41:14.978 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-18T03:41:14.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:14.980 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-18T03:41:15.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:15.054 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-18T03:41:15.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:15.134 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2023-07-18T03:41:15.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:15.266 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-18T03:41:15.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:41:15.312 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2023-07-18T03:41:15.426 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2023-07-18T03:41:15.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:41:15.431 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-07-18T03:41:15.443 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-18T03:41:15.444 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:41:15.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.658+0000 7f45633ff700 1 Processor -- start 2023-07-18T03:41:15.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.658+0000 7f45633ff700 1 -- start start 2023-07-18T03:41:15.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.659+0000 7f45633ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f455c15e890 0x7f455c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:15.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.659+0000 7f45633ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f455c0b3780 0x7f455c0b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:15.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.659+0000 7f45633ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455c15b430 con 0x7f455c15a8c0 2023-07-18T03:41:15.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.659+0000 7f45633ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455c0cf380 con 0x7f455c0b3780 2023-07-18T03:41:15.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.659+0000 7f45633ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455c0b4130 con 0x7f455c15e890 2023-07-18T03:41:15.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.659+0000 7f456099a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f455c0b3780 0x7f455c0b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:15.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.659+0000 7f456199c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f455c15a8c0 0x7f455c15ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49542/0 (socket says 172.21.15.49:49542) 2023-07-18T03:41:15.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.660+0000 7f456199c700 1 -- 172.21.15.49:0/446619531 learned_addr learned my addr 172.21.15.49:0/446619531 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:15.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.660+0000 7f456119b700 1 --2- 172.21.15.49:0/446619531 >> v2:172.21.15.107:3300/0 conn(0x7f455c15e890 0x7f455c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:15.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.660+0000 7f4553fff700 1 -- 172.21.15.49:0/446619531 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928006339 0 0) 0x7f455c15b430 con 0x7f455c15a8c0 2023-07-18T03:41:15.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.660+0000 7f456099a700 1 -- 172.21.15.49:0/446619531 >> v2:172.21.15.107:3300/0 conn(0x7f455c15e890 msgr2=0x7f455c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:15.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.660+0000 7f456099a700 1 --2- 172.21.15.49:0/446619531 >> v2:172.21.15.107:3300/0 conn(0x7f455c15e890 0x7f455c0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:15.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.660+0000 7f456099a700 1 -- 172.21.15.49:0/446619531 >> v1:172.21.15.49:6789/0 conn(0x7f455c15a8c0 legacy=0x7f455c15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:15.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.660+0000 7f456099a700 1 -- 172.21.15.49:0/446619531 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c0ce110 con 0x7f455c0b3780 2023-07-18T03:41:15.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.660+0000 7f456099a700 1 --2- 172.21.15.49:0/446619531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f455c0b3780 0x7f455c0b3bf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f455c06f670 tx=0x7f454c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09a7ab699759e98 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:15.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.660+0000 7f4553fff700 1 -- 172.21.15.49:0/446619531 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c011830 con 0x7f455c0b3780 2023-07-18T03:41:15.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.661+0000 7f4553fff700 1 -- 172.21.15.49:0/446619531 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454c011990 con 0x7f455c0b3780 2023-07-18T03:41:15.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.661+0000 7f4553fff700 1 -- 172.21.15.49:0/446619531 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c011d20 con 0x7f455c0b3780 2023-07-18T03:41:15.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.661+0000 7f45633ff700 1 -- 172.21.15.49:0/446619531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f455c0b3780 msgr2=0x7f455c0b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:15.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.661+0000 7f45633ff700 1 --2- 172.21.15.49:0/446619531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f455c0b3780 0x7f455c0b3bf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f455c06f670 tx=0x7f454c009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:15.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.661+0000 7f45633ff700 1 -- 172.21.15.49:0/446619531 shutdown_connections 2023-07-18T03:41:15.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.661+0000 7f45633ff700 1 --2- 172.21.15.49:0/446619531 >> v2:172.21.15.107:3300/0 conn(0x7f455c15e890 0x7f455c0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:15.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.661+0000 7f45633ff700 1 --2- 172.21.15.49:0/446619531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f455c0b3780 0x7f455c0b3bf0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:15.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.661+0000 7f45633ff700 1 -- 172.21.15.49:0/446619531 >> 172.21.15.49:0/446619531 conn(0x7f455c069f30 msgr2=0x7f455c06a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:15.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.661+0000 7f45633ff700 1 -- 172.21.15.49:0/446619531 shutdown_connections 2023-07-18T03:41:15.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.661+0000 7f45633ff700 1 -- 172.21.15.49:0/446619531 wait complete. 2023-07-18T03:41:15.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.663+0000 7f45633ff700 1 Processor -- start 2023-07-18T03:41:15.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.663+0000 7f45633ff700 1 -- start start 2023-07-18T03:41:15.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.664+0000 7f45633ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c0b3780 0x7f455c0ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:15.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.664+0000 7f45633ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f455c15a8c0 0x7f455c14e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:15.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.664+0000 7f45633ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f455c15e890 0x7f455c1f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:15.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.664+0000 7f45633ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0ca760 con 0x7f455c15e890 2023-07-18T03:41:15.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.664+0000 7f45633ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c14fac0 con 0x7f455c15a8c0 2023-07-18T03:41:15.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.664+0000 7f45633ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c14fda0 con 0x7f455c0b3780 2023-07-18T03:41:15.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.665+0000 7f456199c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f455c15e890 0x7f455c1f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:15.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.665+0000 7f456119b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c0b3780 0x7f455c0ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:15.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.665+0000 7f456099a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f455c15a8c0 0x7f455c14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:15.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.665+0000 7f456199c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f455c15e890 0x7f455c1f1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58228/0 (socket says 172.21.15.49:58228) 2023-07-18T03:41:15.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.665+0000 7f456199c700 1 -- 172.21.15.49:0/185151105 learned_addr learned my addr 172.21.15.49:0/185151105 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:15.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.665+0000 7f456199c700 1 -- 172.21.15.49:0/185151105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c0b3780 msgr2=0x7f455c0ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:15.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.665+0000 7f456199c700 1 --2- 172.21.15.49:0/185151105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c0b3780 0x7f455c0ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:15.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.665+0000 7f456199c700 1 -- 172.21.15.49:0/185151105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f455c15a8c0 msgr2=0x7f455c14e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:15.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.665+0000 7f456199c700 1 --2- 172.21.15.49:0/185151105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f455c15a8c0 0x7f455c14e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:15.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.665+0000 7f456199c700 1 -- 172.21.15.49:0/185151105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c0094a0 con 0x7f455c15e890 2023-07-18T03:41:15.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.665+0000 7f456199c700 1 --2- 172.21.15.49:0/185151105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f455c15e890 0x7f455c1f1bb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4554002230 tx=0x7f455400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:15.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.666+0000 7f4551ffb700 1 -- 172.21.15.49:0/185151105 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455400aac0 con 0x7f455c15e890 2023-07-18T03:41:15.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.666+0000 7f4551ffb700 1 -- 172.21.15.49:0/185151105 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455400ac40 con 0x7f455c15e890 2023-07-18T03:41:15.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.666+0000 7f4551ffb700 1 -- 172.21.15.49:0/185151105 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45540070d0 con 0x7f455c15e890 2023-07-18T03:41:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.666+0000 7f45633ff700 1 -- 172.21.15.49:0/185151105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455c150080 con 0x7f455c15e890 2023-07-18T03:41:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.666+0000 7f45633ff700 1 -- 172.21.15.49:0/185151105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c150640 con 0x7f455c15e890 2023-07-18T03:41:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.667+0000 7f45633ff700 1 -- 172.21.15.49:0/185151105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455c04e280 con 0x7f455c15e890 2023-07-18T03:41:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.668+0000 7f4551ffb700 1 -- 172.21.15.49:0/185151105 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f4554007250 con 0x7f455c15e890 2023-07-18T03:41:15.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.669+0000 7f4551ffb700 1 --2- 172.21.15.49:0/185151105 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f454006ae00 0x7f454006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.669+0000 7f4551ffb700 1 -- 172.21.15.49:0/185151105 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4554088de0 con 0x7f455c15e890 2023-07-18T03:41:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.669+0000 7f456119b700 1 --2- 172.21.15.49:0/185151105 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f454006ae00 0x7f454006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.670+0000 7f456119b700 1 --2- 172.21.15.49:0/185151105 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f454006ae00 0x7f454006d2b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4558009b40 tx=0x7f4558009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:15.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.674+0000 7f4551ffb700 1 -- 172.21.15.49:0/185151105 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4554054000 con 0x7f455c15e890 2023-07-18T03:41:15.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.880+0000 7f45633ff700 1 -- 172.21.15.49:0/185151105 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f455c0ce360 con 0x7f454006ae00 2023-07-18T03:41:15.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.883+0000 7f4551ffb700 1 -- 172.21.15.49:0/185151105 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73363 (secure 0 0 0) 0x7f4558011810 con 0x7f454006ae00 2023-07-18T03:41:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:15.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.885+0000 7f45633ff700 1 -- 172.21.15.49:0/185151105 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f454006ae00 msgr2=0x7f454006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:15.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.885+0000 7f45633ff700 1 --2- 172.21.15.49:0/185151105 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f454006ae00 0x7f454006d2b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4558009b40 tx=0x7f4558009210 comp rx=0 tx=0).stop 2023-07-18T03:41:15.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.885+0000 7f45633ff700 1 -- 172.21.15.49:0/185151105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f455c15e890 msgr2=0x7f455c1f1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:15.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.885+0000 7f45633ff700 1 --2- 172.21.15.49:0/185151105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f455c15e890 0x7f455c1f1bb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4554002230 tx=0x7f455400cf50 comp rx=0 tx=0).stop 2023-07-18T03:41:15.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.885+0000 7f45633ff700 1 -- 172.21.15.49:0/185151105 shutdown_connections 2023-07-18T03:41:15.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.885+0000 7f45633ff700 1 --2- 172.21.15.49:0/185151105 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f454006ae00 0x7f454006d2b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:15.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.885+0000 7f45633ff700 1 --2- 172.21.15.49:0/185151105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f455c15e890 0x7f455c1f1bb0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:15.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.885+0000 7f45633ff700 1 --2- 172.21.15.49:0/185151105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c0b3780 0x7f455c0ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:15.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.885+0000 7f45633ff700 1 --2- 172.21.15.49:0/185151105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f455c15a8c0 0x7f455c14e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:15.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.885+0000 7f45633ff700 1 -- 172.21.15.49:0/185151105 >> 172.21.15.49:0/185151105 conn(0x7f455c069f30 msgr2=0x7f455c154680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:15.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.885+0000 7f45633ff700 1 -- 172.21.15.49:0/185151105 shutdown_connections 2023-07-18T03:41:15.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:15.885+0000 7f45633ff700 1 -- 172.21.15.49:0/185151105 wait complete. 2023-07-18T03:41:15.882 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:41:15.898 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2023-07-18T03:41:14.136021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238072,"kb_used_data":6072,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754736648,"statfs":{"total":773094113280,"available":772850327552,"internally_reserved":0,"allocated":6217728,"data_stored":3375832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-137598,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":-5,"num_read_kb":-3,"num_write":-107,"num_write_kb":-183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.614540"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.013833+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:41:12.013833+0000","last_peered":"2023-07-18T03:41:12.013833+0000","last_clean":"2023-07-18T03:41:12.013833+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:41:12.013833+0000","last_undegraded":"2023-07-18T03:41:12.013833+0000","last_fullsized":"2023-07-18T03:41:12.013833+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:06:30.033507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-18T03:41:11.646563+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:41:11.646563+0000","last_peered":"2023-07-18T03:41:11.646563+0000","last_clean":"2023-07-18T03:41:11.646563+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:41:11.646563+0000","last_undegraded":"2023-07-18T03:41:11.646563+0000","last_fullsized":"2023-07-18T03:41:11.646563+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:50.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.007265+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:41:12.007265+0000","last_peered":"2023-07-18T03:41:12.007265+0000","last_clean":"2023-07-18T03:41:12.007265+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:41:12.007265+0000","last_undegraded":"2023-07-18T03:41:12.007265+0000","last_fullsized":"2023-07-18T03:41:12.007265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:01:17.117290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.012586+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:41:12.012586+0000","last_peered":"2023-07-18T03:41:12.012586+0000","last_clean":"2023-07-18T03:41:12.012586+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:41:12.012586+0000","last_undegraded":"2023-07-18T03:41:12.012586+0000","last_fullsized":"2023-07-18T03:41:12.012586+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:19:41.650574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.015761+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:41:12.015761+0000","last_peered":"2023-07-18T03:41:12.015761+0000","last_clean":"2023-07-18T03:41:12.015761+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:41:12.015761+0000","last_undegraded":"2023-07-18T03:41:12.015761+0000","last_fullsized":"2023-07-18T03:41:12.015761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:29:32.125624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.012398+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:41:12.012398+0000","last_peered":"2023-07-18T03:41:12.012398+0000","last_clean":"2023-07-18T03:41:12.012398+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:41:12.012398+0000","last_undegraded":"2023-07-18T03:41:12.012398+0000","last_fullsized":"2023-07-18T03:41:12.012398+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:14:41.054472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-18T03:41:11.645503+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:41:11.645503+0000","last_peered":"2023-07-18T03:41:11.645503+0000","last_clean":"2023-07-18T03:41:11.645503+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:41:11.645503+0000","last_undegraded":"2023-07-18T03:41:11.645503+0000","last_fullsized":"2023-07-18T03:41:11.645503+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:39.926749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.438393+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:41:12.438393+0000","last_peered":"2023-07-18T03:41:12.438393+0000","last_clean":"2023-07-18T03:41:12.438393+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:41:12.438393+0000","last_undegraded":"2023-07-18T03:41:12.438393+0000","last_fullsized":"2023-07-18T03:41:12.438393+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:15:38.470063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.438614+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:41:12.438614+0000","last_peered":"2023-07-18T03:41:12.438614+0000","last_clean":"2023-07-18T03:41:12.438614+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:41:12.438614+0000","last_undegraded":"2023-07-18T03:41:12.438614+0000","last_fullsized":"2023-07-18T03:41:12.438614+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:04:44.231921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1232896,"data_stored":869098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85599999999999998,"5min":1.167,"15min":1.167},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.755}]},{"osd":1,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71999999999999997,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.88900000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.873,"15min":0.89500000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.2570000000000001,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":0.70399999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.746,"15min":0.753},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1419999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0249999999999999,"5min":1.365,"15min":1.365},"last":0.90700000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.0010000000000003,"5min":4.0010000000000003,"15min":4.0010000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.72899999999999998,"15min":0.71199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.2989999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.77700000000000002,"15min":0.76000000000000001},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.63500000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.81699999999999995,"15min":0.80000000000000004},"min":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0609999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.86899999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509477,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.73799999999999999,"15min":0.754},"min":{"1min":0.51400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84699999999999998,"5min":2.427,"15min":2.427},"last":0.63100000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.84099999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65000000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95999999999999996,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91300000000000003,"5min":1.034,"15min":1.034},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96399999999999997,"5min":1.113,"15min":1.113},"last":0.81299999999999994}]},{"osd":4,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90400000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82999999999999996}]},{"osd":5,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55900000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.90900000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81000000000000005,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0820000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.755}]}]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":626688,"data_stored":273026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.432,"15min":0.441},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.40899999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.58899999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47399999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.69999999999999996,"15min":0.72799999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.66100000000000003,"5min":3.956,"15min":3.956},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.68400000000000005,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.755,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.497}]},{"osd":4,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.94399999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.86599999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.92900000000000005,"5min":1.22,"15min":1.22},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.88200000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.72499999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91300000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90400000000000003,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.75900000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.94599999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.79900000000000004,"15min":0.81000000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84899999999999998,"5min":1.272,"15min":1.272},"last":0.78400000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1220608,"data_stored":863394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64500000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.46100000000000002,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.434,"15min":0.435},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.60099999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.58399999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.65700000000000003,"5min":0.748,"15min":0.748},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90400000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.59399999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.85099999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":0.73799999999999999},"min":{"1min":0.57199999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84999999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.747,"15min":0.754},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.74199999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88400000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.754,"15min":0.755},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.94999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.75900000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.61499999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.01,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92000000000000004,"5min":1.042,"15min":1.042},"last":0.79200000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":622592,"data_stored":272812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.77800000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.47499999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.81899999999999995}]},{"osd":2,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89400000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.83399999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.81699999999999995,"5min":1.032,"15min":1.032},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.91100000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.503},{"interface":"front","average":{"1min":0.63,"5min":0.70399999999999996,"15min":0.71599999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92700000000000005,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92200000000000004,"5min":0.999,"15min":0.999},"last":0.76200000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":626688,"data_stored":272934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.67500000000000004,"15min":0.63700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.74099999999999999,"15min":0.68700000000000006},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":8.3780000000000001,"5min":8.3780000000000001,"15min":8.3780000000000001},"last":0.628},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.74299999999999999,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.64300000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.77200000000000002,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":4.8769999999999998,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.66100000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70499999999999996,"15min":0.69499999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.52600000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.755,"15min":0.73899999999999999},"min":{"1min":0.41199999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.9690000000000003,"5min":4.9690000000000003,"15min":4.9690000000000003},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.6829999999999998,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71399999999999997,"15min":0.71899999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90800000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.83899999999999997,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.3140000000000001,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.80300000000000005,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":3.7389999999999999,"5min":3.7389999999999999,"15min":3.7389999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94799999999999995,"5min":1.472,"15min":1.472},"last":0.77600000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509478,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31656,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":622592,"data_stored":272812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.47499999999999998,"15min":0.48899999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":1.389,"15min":1.389},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.64500000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.84099999999999997,"15min":0.90300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.74199999999999999,"5min":8.3040000000000003,"15min":8.3040000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.66700000000000004,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.44700000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.77300000000000002,"15min":0.81899999999999995},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63100000000000001,"5min":8.0229999999999997,"15min":8.0229999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.88,"15min":0.93799999999999994},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.70999999999999996,"5min":8.359,"15min":8.359},"last":0.58599999999999997}]},{"osd":4,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97699999999999998,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78400000000000003,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0129999999999999,"5min":1.429,"15min":1.429},"last":0.82699999999999996}]},{"osd":5,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.117,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.747,"5min":0.754,"15min":0.755},"min":{"1min":0.623,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0920000000000001,"5min":1.266,"15min":1.266},"last":0.751}]},{"osd":6,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95199999999999996,"5min":1.752,"15min":1.752},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.626,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0449999999999999,"5min":1.454,"15min":1.454},"last":0.96099999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.58499999999999996,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.94299999999999995,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0249999999999999,"5min":1.222,"15min":1.222},"last":0.84899999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":626688,"data_stored":273026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.503,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.498,"15min":0.497},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72499999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.45500000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.437},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57199999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.622,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78800000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78700000000000003,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0580000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.038,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.86399999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.60799999999999998,"5min":0.504,"15min":0.504},"max":{"1min":1.0209999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.82099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:41:15.901 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:41:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.112+0000 7f51951ff700 1 Processor -- start 2023-07-18T03:41:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.112+0000 7f51951ff700 1 -- start start 2023-07-18T03:41:16.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.113+0000 7f51951ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f519015ca30 0x7f519006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:16.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.113+0000 7f51951ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519006c360 con 0x7f51900d0f40 2023-07-18T03:41:16.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.114+0000 7f51951ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519006ba50 con 0x7f51900ce840 2023-07-18T03:41:16.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.114+0000 7f51951ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51900d1b70 con 0x7f519015ca30 2023-07-18T03:41:16.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.114+0000 7f518f59e700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f51900d0f40 0x7f51900ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49556/0 (socket says 172.21.15.49:49556) 2023-07-18T03:41:16.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.114+0000 7f518f59e700 1 -- 172.21.15.49:0/1351708020 learned_addr learned my addr 172.21.15.49:0/1351708020 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:16.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.114+0000 7f518ed9d700 1 --2- 172.21.15.49:0/1351708020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f519015ca30 0x7f519006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:16.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.114+0000 7f518dd9b700 1 -- 172.21.15.49:0/1351708020 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787428065 0 0) 0x7f519006ba50 con 0x7f51900ce840 2023-07-18T03:41:16.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.114+0000 7f518dd9b700 1 -- 172.21.15.49:0/1351708020 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51780035b0 con 0x7f51900ce840 2023-07-18T03:41:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.114+0000 7f518ed9d700 1 -- 172.21.15.49:0/1351708020 >> v1:172.21.15.107:6789/0 conn(0x7f51900ce840 legacy=0x7f51900d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.114+0000 7f518ed9d700 1 -- 172.21.15.49:0/1351708020 >> v1:172.21.15.49:6789/0 conn(0x7f51900d0f40 legacy=0x7f51900ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.114+0000 7f518dd9b700 1 -- 172.21.15.49:0/1351708020 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241483414 0 0) 0x7f519006c360 con 0x7f51900d0f40 2023-07-18T03:41:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.114+0000 7f518ed9d700 1 -- 172.21.15.49:0/1351708020 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900d14a0 con 0x7f519015ca30 2023-07-18T03:41:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.115+0000 7f518ed9d700 1 --2- 172.21.15.49:0/1351708020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f519015ca30 0x7f519006c7f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f51800098a0 tx=0x7f5180009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=119b8a882ed9f706 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.115+0000 7f518dd9b700 1 -- 172.21.15.49:0/1351708020 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180011830 con 0x7f519015ca30 2023-07-18T03:41:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.115+0000 7f518dd9b700 1 -- 172.21.15.49:0/1351708020 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5180011990 con 0x7f519015ca30 2023-07-18T03:41:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.115+0000 7f518dd9b700 1 -- 172.21.15.49:0/1351708020 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180011d00 con 0x7f519015ca30 2023-07-18T03:41:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.115+0000 7f51951ff700 1 -- 172.21.15.49:0/1351708020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f519015ca30 msgr2=0x7f519006c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.115+0000 7f51951ff700 1 --2- 172.21.15.49:0/1351708020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f519015ca30 0x7f519006c7f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f51800098a0 tx=0x7f5180009bb0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.116+0000 7f51951ff700 1 -- 172.21.15.49:0/1351708020 shutdown_connections 2023-07-18T03:41:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.116+0000 7f51951ff700 1 --2- 172.21.15.49:0/1351708020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f519015ca30 0x7f519006c7f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.116+0000 7f51951ff700 1 -- 172.21.15.49:0/1351708020 >> 172.21.15.49:0/1351708020 conn(0x7f51900664f0 msgr2=0x7f5190099fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.116+0000 7f51951ff700 1 -- 172.21.15.49:0/1351708020 shutdown_connections 2023-07-18T03:41:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.116+0000 7f51951ff700 1 -- 172.21.15.49:0/1351708020 wait complete. 2023-07-18T03:41:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.117+0000 7f51951ff700 1 Processor -- start 2023-07-18T03:41:16.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.118+0000 7f51951ff700 1 -- start start 2023-07-18T03:41:16.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.118+0000 7f51951ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51900ce840 0x7f51900c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:16.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.118+0000 7f51951ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51900d0f40 0x7f51901f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:16.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.118+0000 7f51951ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f519015ca30 0x7f51901f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:16.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.118+0000 7f51951ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51901f6700 con 0x7f51900ce840 2023-07-18T03:41:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.118+0000 7f51951ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51900c51a0 con 0x7f51900d0f40 2023-07-18T03:41:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.118+0000 7f51951ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51901f2050 con 0x7f519015ca30 2023-07-18T03:41:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.119+0000 7f518ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51900ce840 0x7f51900c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.119+0000 7f518ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51900ce840 0x7f51900c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58232/0 (socket says 172.21.15.49:58232) 2023-07-18T03:41:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.119+0000 7f518ed9d700 1 -- 172.21.15.49:0/879732134 learned_addr learned my addr 172.21.15.49:0/879732134 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:16.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.119+0000 7f518ed9d700 1 -- 172.21.15.49:0/879732134 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f519015ca30 msgr2=0x7f51901f1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.119+0000 7f518ed9d700 1 --2- 172.21.15.49:0/879732134 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f519015ca30 0x7f51901f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.119+0000 7f518ed9d700 1 -- 172.21.15.49:0/879732134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51900d0f40 msgr2=0x7f51901f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.119+0000 7f518ed9d700 1 --2- 172.21.15.49:0/879732134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51900d0f40 0x7f51901f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.119+0000 7f518ed9d700 1 -- 172.21.15.49:0/879732134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800094a0 con 0x7f51900ce840 2023-07-18T03:41:16.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.119+0000 7f518ed9d700 1 --2- 172.21.15.49:0/879732134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51900ce840 0x7f51900c5940 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f51800090b0 tx=0x7f5180004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:16.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.120+0000 7f51777fe700 1 -- 172.21.15.49:0/879732134 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800036a0 con 0x7f51900ce840 2023-07-18T03:41:16.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.120+0000 7f51777fe700 1 -- 172.21.15.49:0/879732134 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5180003820 con 0x7f51900ce840 2023-07-18T03:41:16.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.120+0000 7f51777fe700 1 -- 172.21.15.49:0/879732134 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180003bd0 con 0x7f51900ce840 2023-07-18T03:41:16.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.120+0000 7f51951ff700 1 -- 172.21.15.49:0/879732134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51900c4870 con 0x7f51900ce840 2023-07-18T03:41:16.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.120+0000 7f51951ff700 1 -- 172.21.15.49:0/879732134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51900c4e30 con 0x7f51900ce840 2023-07-18T03:41:16.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.121+0000 7f51951ff700 1 -- 172.21.15.49:0/879732134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51900cfc20 con 0x7f51900ce840 2023-07-18T03:41:16.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.122+0000 7f51777fe700 1 -- 172.21.15.49:0/879732134 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f5180003d30 con 0x7f51900ce840 2023-07-18T03:41:16.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.123+0000 7f51777fe700 1 --2- 172.21.15.49:0/879732134 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f517c06ae00 0x7f517c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:16.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.123+0000 7f51777fe700 1 -- 172.21.15.49:0/879732134 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f518008a880 con 0x7f51900ce840 2023-07-18T03:41:16.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.123+0000 7f518e59c700 1 --2- 172.21.15.49:0/879732134 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f517c06ae00 0x7f517c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:16.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.124+0000 7f518e59c700 1 --2- 172.21.15.49:0/879732134 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f517c06ae00 0x7f517c06d2b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f51840010e0 tx=0x7f51840079e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:16.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.127+0000 7f51777fe700 1 -- 172.21.15.49:0/879732134 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5180055b30 con 0x7f51900ce840 2023-07-18T03:41:16.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.332+0000 7f51951ff700 1 -- 172.21.15.49:0/879732134 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f519006bff0 con 0x7f517c06ae00 2023-07-18T03:41:16.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.335+0000 7f51777fe700 1 -- 172.21.15.49:0/879732134 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73363 (secure 0 0 0) 0x7f51840050d0 con 0x7f517c06ae00 2023-07-18T03:41:16.333 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.338+0000 7f51951ff700 1 -- 172.21.15.49:0/879732134 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f517c06ae00 msgr2=0x7f517c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.338+0000 7f51951ff700 1 --2- 172.21.15.49:0/879732134 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f517c06ae00 0x7f517c06d2b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f51840010e0 tx=0x7f51840079e0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.338+0000 7f51951ff700 1 -- 172.21.15.49:0/879732134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51900ce840 msgr2=0x7f51900c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.338+0000 7f51951ff700 1 --2- 172.21.15.49:0/879732134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51900ce840 0x7f51900c5940 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f51800090b0 tx=0x7f5180004750 comp rx=0 tx=0).stop 2023-07-18T03:41:16.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.338+0000 7f51951ff700 1 -- 172.21.15.49:0/879732134 shutdown_connections 2023-07-18T03:41:16.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.338+0000 7f51951ff700 1 --2- 172.21.15.49:0/879732134 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f519015ca30 0x7f51901f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.338+0000 7f51951ff700 1 --2- 172.21.15.49:0/879732134 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f517c06ae00 0x7f517c06d2b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.338+0000 7f51951ff700 1 --2- 172.21.15.49:0/879732134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51900ce840 0x7f51900c5940 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.338+0000 7f51951ff700 1 --2- 172.21.15.49:0/879732134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51900d0f40 0x7f51901f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.338+0000 7f51951ff700 1 -- 172.21.15.49:0/879732134 >> 172.21.15.49:0/879732134 conn(0x7f51900664f0 msgr2=0x7f519015ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:16.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.338+0000 7f51951ff700 1 -- 172.21.15.49:0/879732134 shutdown_connections 2023-07-18T03:41:16.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.338+0000 7f51951ff700 1 -- 172.21.15.49:0/879732134 wait complete. 2023-07-18T03:41:16.340 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:41:16.350 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2023-07-18T03:41:16.136481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238072,"kb_used_data":6072,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754736648,"statfs":{"total":773094113280,"available":772850327552,"internally_reserved":0,"allocated":6217728,"data_stored":3375832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-137598,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":-5,"num_read_kb":-3,"num_write":-107,"num_write_kb":-183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.221284"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.013833+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:41:12.013833+0000","last_peered":"2023-07-18T03:41:12.013833+0000","last_clean":"2023-07-18T03:41:12.013833+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:41:12.013833+0000","last_undegraded":"2023-07-18T03:41:12.013833+0000","last_fullsized":"2023-07-18T03:41:12.013833+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:06:30.033507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-18T03:41:11.646563+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:41:11.646563+0000","last_peered":"2023-07-18T03:41:11.646563+0000","last_clean":"2023-07-18T03:41:11.646563+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:41:11.646563+0000","last_undegraded":"2023-07-18T03:41:11.646563+0000","last_fullsized":"2023-07-18T03:41:11.646563+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:50.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.007265+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:41:12.007265+0000","last_peered":"2023-07-18T03:41:12.007265+0000","last_clean":"2023-07-18T03:41:12.007265+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:41:12.007265+0000","last_undegraded":"2023-07-18T03:41:12.007265+0000","last_fullsized":"2023-07-18T03:41:12.007265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:01:17.117290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.012586+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:41:12.012586+0000","last_peered":"2023-07-18T03:41:12.012586+0000","last_clean":"2023-07-18T03:41:12.012586+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:41:12.012586+0000","last_undegraded":"2023-07-18T03:41:12.012586+0000","last_fullsized":"2023-07-18T03:41:12.012586+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:19:41.650574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.015761+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:41:12.015761+0000","last_peered":"2023-07-18T03:41:12.015761+0000","last_clean":"2023-07-18T03:41:12.015761+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:41:12.015761+0000","last_undegraded":"2023-07-18T03:41:12.015761+0000","last_fullsized":"2023-07-18T03:41:12.015761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:29:32.125624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.012398+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:41:12.012398+0000","last_peered":"2023-07-18T03:41:12.012398+0000","last_clean":"2023-07-18T03:41:12.012398+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:41:12.012398+0000","last_undegraded":"2023-07-18T03:41:12.012398+0000","last_fullsized":"2023-07-18T03:41:12.012398+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:14:41.054472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-18T03:41:11.645503+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:41:11.645503+0000","last_peered":"2023-07-18T03:41:11.645503+0000","last_clean":"2023-07-18T03:41:11.645503+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:41:11.645503+0000","last_undegraded":"2023-07-18T03:41:11.645503+0000","last_fullsized":"2023-07-18T03:41:11.645503+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:39.926749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.438393+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:41:12.438393+0000","last_peered":"2023-07-18T03:41:12.438393+0000","last_clean":"2023-07-18T03:41:12.438393+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:41:12.438393+0000","last_undegraded":"2023-07-18T03:41:12.438393+0000","last_fullsized":"2023-07-18T03:41:12.438393+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:15:38.470063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.438614+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:41:12.438614+0000","last_peered":"2023-07-18T03:41:12.438614+0000","last_clean":"2023-07-18T03:41:12.438614+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:41:12.438614+0000","last_undegraded":"2023-07-18T03:41:12.438614+0000","last_fullsized":"2023-07-18T03:41:12.438614+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:04:44.231921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1232896,"data_stored":869098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85599999999999998,"5min":1.167,"15min":1.167},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.755}]},{"osd":1,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71999999999999997,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.88900000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.873,"15min":0.89500000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.2570000000000001,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":0.70399999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.746,"15min":0.753},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1419999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0249999999999999,"5min":1.365,"15min":1.365},"last":0.90700000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.0010000000000003,"5min":4.0010000000000003,"15min":4.0010000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.72899999999999998,"15min":0.71199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.2989999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.77700000000000002,"15min":0.76000000000000001},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.63500000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.81699999999999995,"15min":0.80000000000000004},"min":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0609999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.86899999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509477,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.73799999999999999,"15min":0.754},"min":{"1min":0.51400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84699999999999998,"5min":2.427,"15min":2.427},"last":0.63100000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.84099999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65000000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95999999999999996,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91300000000000003,"5min":1.034,"15min":1.034},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96399999999999997,"5min":1.113,"15min":1.113},"last":0.81299999999999994}]},{"osd":4,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90400000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82999999999999996}]},{"osd":5,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55900000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.90900000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81000000000000005,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0820000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.755}]}]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":626688,"data_stored":273026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.432,"15min":0.441},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.40899999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.58899999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47399999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.69999999999999996,"15min":0.72799999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.66100000000000003,"5min":3.956,"15min":3.956},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.68400000000000005,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.755,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.497}]},{"osd":4,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.94399999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.86599999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.92900000000000005,"5min":1.22,"15min":1.22},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.88200000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.72499999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91300000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90400000000000003,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.75900000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.94599999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.79900000000000004,"15min":0.81000000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84899999999999998,"5min":1.272,"15min":1.272},"last":0.78400000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1220608,"data_stored":863394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64500000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.46100000000000002,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.434,"15min":0.435},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.60099999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.58399999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.65700000000000003,"5min":0.748,"15min":0.748},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90400000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.59399999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.85099999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":0.73799999999999999},"min":{"1min":0.57199999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84999999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.747,"15min":0.754},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.74199999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88400000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.754,"15min":0.755},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.94999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.75900000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.61499999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.01,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92000000000000004,"5min":1.042,"15min":1.042},"last":0.79200000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":622592,"data_stored":272812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.77800000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.47499999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.81899999999999995}]},{"osd":2,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89400000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.83399999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.81699999999999995,"5min":1.032,"15min":1.032},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.91100000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.503},{"interface":"front","average":{"1min":0.63,"5min":0.70399999999999996,"15min":0.71599999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92700000000000005,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92200000000000004,"5min":0.999,"15min":0.999},"last":0.76200000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":626688,"data_stored":272934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.67500000000000004,"15min":0.63700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.74099999999999999,"15min":0.68700000000000006},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":8.3780000000000001,"5min":8.3780000000000001,"15min":8.3780000000000001},"last":0.628},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.74299999999999999,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.64300000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.77200000000000002,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":4.8769999999999998,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.66100000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70499999999999996,"15min":0.69499999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.52600000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.755,"15min":0.73899999999999999},"min":{"1min":0.41199999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.9690000000000003,"5min":4.9690000000000003,"15min":4.9690000000000003},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.6829999999999998,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71399999999999997,"15min":0.71899999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90800000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.83899999999999997,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.3140000000000001,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.80300000000000005,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":3.7389999999999999,"5min":3.7389999999999999,"15min":3.7389999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94799999999999995,"5min":1.472,"15min":1.472},"last":0.77600000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509478,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31656,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":622592,"data_stored":272812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.47499999999999998,"15min":0.48899999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":1.389,"15min":1.389},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.64500000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.84099999999999997,"15min":0.90300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.74199999999999999,"5min":8.3040000000000003,"15min":8.3040000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.66700000000000004,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.44700000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.77300000000000002,"15min":0.81899999999999995},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63100000000000001,"5min":8.0229999999999997,"15min":8.0229999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.88,"15min":0.93799999999999994},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.70999999999999996,"5min":8.359,"15min":8.359},"last":0.58599999999999997}]},{"osd":4,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97699999999999998,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78400000000000003,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0129999999999999,"5min":1.429,"15min":1.429},"last":0.82699999999999996}]},{"osd":5,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.117,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.747,"5min":0.754,"15min":0.755},"min":{"1min":0.623,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0920000000000001,"5min":1.266,"15min":1.266},"last":0.751}]},{"osd":6,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95199999999999996,"5min":1.752,"15min":1.752},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.626,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0449999999999999,"5min":1.454,"15min":1.454},"last":0.96099999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.58499999999999996,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.94299999999999995,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0249999999999999,"5min":1.222,"15min":1.222},"last":0.84899999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":626688,"data_stored":273026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.503,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.498,"15min":0.497},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72499999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.45500000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.437},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57199999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.622,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78800000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78700000000000003,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0580000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.038,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.86399999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.60799999999999998,"5min":0.504,"15min":0.504},"max":{"1min":1.0209999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.82099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:41:16.353 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-18T03:41:16.353 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:41:16.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.564+0000 7f3f933ff700 1 Processor -- start 2023-07-18T03:41:16.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.564+0000 7f3f933ff700 1 -- start start 2023-07-18T03:41:16.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.565+0000 7f3f933ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f8c0d1220 0x7f3f8c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:16.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.565+0000 7f3f933ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8c0d2ce0 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.565+0000 7f3f933ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8c0b3350 con 0x7f3f8c15c950 2023-07-18T03:41:16.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.565+0000 7f3f933ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c0b3630 con 0x7f3f8c0d1220 2023-07-18T03:41:16.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.565+0000 7f3f9199c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3f8c0b1e00 0x7f3f8c0afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49562/0 (socket says 172.21.15.49:49562) 2023-07-18T03:41:16.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.566+0000 7f3f9199c700 1 -- 172.21.15.49:0/612549759 learned_addr learned my addr 172.21.15.49:0/612549759 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:16.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.566+0000 7f3f9099a700 1 --2- 172.21.15.49:0/612549759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f8c0d1220 0x7f3f8c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:16.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.566+0000 7f3f83fff700 1 -- 172.21.15.49:0/612549759 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176071132 0 0) 0x7f3f8c0d2ce0 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.566+0000 7f3f83fff700 1 -- 172.21.15.49:0/612549759 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f780035b0 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.566+0000 7f3f9099a700 1 -- 172.21.15.49:0/612549759 >> v1:172.21.15.107:6789/0 conn(0x7f3f8c15c950 legacy=0x7f3f8c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.566+0000 7f3f9099a700 1 -- 172.21.15.49:0/612549759 >> v1:172.21.15.49:6789/0 conn(0x7f3f8c0b1e00 legacy=0x7f3f8c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.566+0000 7f3f9099a700 1 -- 172.21.15.49:0/612549759 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8c0b3910 con 0x7f3f8c0d1220 2023-07-18T03:41:16.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.567+0000 7f3f9099a700 1 --2- 172.21.15.49:0/612549759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f8c0d1220 0x7f3f8c0b43f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009870 tx=0x7f3f7c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99914341b988a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:16.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.567+0000 7f3f83fff700 1 -- 172.21.15.49:0/612549759 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c011830 con 0x7f3f8c0d1220 2023-07-18T03:41:16.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.567+0000 7f3f83fff700 1 -- 172.21.15.49:0/612549759 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c011990 con 0x7f3f8c0d1220 2023-07-18T03:41:16.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.567+0000 7f3f83fff700 1 -- 172.21.15.49:0/612549759 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c011d20 con 0x7f3f8c0d1220 2023-07-18T03:41:16.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.568+0000 7f3f933ff700 1 -- 172.21.15.49:0/612549759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f8c0d1220 msgr2=0x7f3f8c0b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.568+0000 7f3f933ff700 1 --2- 172.21.15.49:0/612549759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f8c0d1220 0x7f3f8c0b43f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009870 tx=0x7f3f7c009b80 comp rx=0 tx=0).stop 2023-07-18T03:41:16.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.568+0000 7f3f933ff700 1 -- 172.21.15.49:0/612549759 shutdown_connections 2023-07-18T03:41:16.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.568+0000 7f3f933ff700 1 --2- 172.21.15.49:0/612549759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f8c0d1220 0x7f3f8c0b43f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.568+0000 7f3f933ff700 1 -- 172.21.15.49:0/612549759 >> 172.21.15.49:0/612549759 conn(0x7f3f8c09c1d0 msgr2=0x7f3f8c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:16.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.568+0000 7f3f933ff700 1 -- 172.21.15.49:0/612549759 shutdown_connections 2023-07-18T03:41:16.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.568+0000 7f3f933ff700 1 -- 172.21.15.49:0/612549759 wait complete. 2023-07-18T03:41:16.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.569+0000 7f3f933ff700 1 Processor -- start 2023-07-18T03:41:16.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.570+0000 7f3f933ff700 1 -- start start 2023-07-18T03:41:16.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f933ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c0b1e00 0x7f3f8c0cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:16.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f933ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f8c0d1220 0x7f3f8c157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:16.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f933ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f8c15c950 0x7f3f8c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f933ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c157e90 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f933ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c0cb400 con 0x7f3f8c15c950 2023-07-18T03:41:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f933ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c0ca550 con 0x7f3f8c0d1220 2023-07-18T03:41:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f9119b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c0b1e00 0x7f3f8c0cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f9119b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c0b1e00 0x7f3f8c0cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58248/0 (socket says 172.21.15.49:58248) 2023-07-18T03:41:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f9119b700 1 -- 172.21.15.49:0/1450850952 learned_addr learned my addr 172.21.15.49:0/1450850952 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f9099a700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f8c0d1220 0x7f3f8c157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f9199c700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f8c15c950 0x7f3f8c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f9119b700 1 -- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f8c0d1220 msgr2=0x7f3f8c157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f9119b700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f8c0d1220 0x7f3f8c157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f9119b700 1 -- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f8c15c950 msgr2=0x7f3f8c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f9119b700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f8c15c950 0x7f3f8c0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.571+0000 7f3f9119b700 1 -- 172.21.15.49:0/1450850952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7c0094a0 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.572+0000 7f3f9119b700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c0b1e00 0x7f3f8c0cfd80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800c480 tx=0x7f3f8800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:16.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.572+0000 7f3f81ffb700 1 -- 172.21.15.49:0/1450850952 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f880052d0 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.572+0000 7f3f81ffb700 1 -- 172.21.15.49:0/1450850952 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f88005430 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.572+0000 7f3f933ff700 1 -- 172.21.15.49:0/1450850952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8c0ca7f0 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.572+0000 7f3f81ffb700 1 -- 172.21.15.49:0/1450850952 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f880038f0 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.572+0000 7f3f933ff700 1 -- 172.21.15.49:0/1450850952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8c0cadb0 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.573+0000 7f3f933ff700 1 -- 172.21.15.49:0/1450850952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8c04e280 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.574+0000 7f3f81ffb700 1 -- 172.21.15.49:0/1450850952 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f3f88003a50 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.576+0000 7f3f81ffb700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3f7006ae00 0x7f3f7006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:16.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.576+0000 7f3f81ffb700 1 -- 172.21.15.49:0/1450850952 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3f88088640 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.576+0000 7f3f9099a700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3f7006ae00 0x7f3f7006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:16.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.577+0000 7f3f9099a700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3f7006ae00 0x7f3f7006d2b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0d1610 tx=0x7f3f7c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:16.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.580+0000 7f3f81ffb700 1 -- 172.21.15.49:0/1450850952 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3f880538e0 con 0x7f3f8c0b1e00 2023-07-18T03:41:16.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.784+0000 7f3f933ff700 1 -- 172.21.15.49:0/1450850952 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f8c0b3ed0 con 0x7f3f7006ae00 2023-07-18T03:41:16.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.787+0000 7f3f81ffb700 1 -- 172.21.15.49:0/1450850952 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73363 (secure 0 0 0) 0x7f3f7c011b80 con 0x7f3f7006ae00 2023-07-18T03:41:16.783 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:16.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.789+0000 7f3f933ff700 1 -- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3f7006ae00 msgr2=0x7f3f7006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.789+0000 7f3f933ff700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3f7006ae00 0x7f3f7006d2b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0d1610 tx=0x7f3f7c008030 comp rx=0 tx=0).stop 2023-07-18T03:41:16.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.789+0000 7f3f933ff700 1 -- 172.21.15.49:0/1450850952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c0b1e00 msgr2=0x7f3f8c0cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.789+0000 7f3f933ff700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c0b1e00 0x7f3f8c0cfd80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800c480 tx=0x7f3f8800c790 comp rx=0 tx=0).stop 2023-07-18T03:41:16.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.789+0000 7f3f933ff700 1 -- 172.21.15.49:0/1450850952 shutdown_connections 2023-07-18T03:41:16.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.789+0000 7f3f933ff700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f8c15c950 0x7f3f8c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.789+0000 7f3f933ff700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3f7006ae00 0x7f3f7006d2b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.789+0000 7f3f933ff700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f8c0b1e00 0x7f3f8c0cfd80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.789+0000 7f3f933ff700 1 --2- 172.21.15.49:0/1450850952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f8c0d1220 0x7f3f8c157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:16.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.790+0000 7f3f933ff700 1 -- 172.21.15.49:0/1450850952 >> 172.21.15.49:0/1450850952 conn(0x7f3f8c09c1d0 msgr2=0x7f3f8c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:16.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.790+0000 7f3f933ff700 1 -- 172.21.15.49:0/1450850952 shutdown_connections 2023-07-18T03:41:16.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:16.790+0000 7f3f933ff700 1 -- 172.21.15.49:0/1450850952 wait complete. 2023-07-18T03:41:16.791 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:41:16.802 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2023-07-18T03:41:16.136481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238072,"kb_used_data":6072,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754736648,"statfs":{"total":773094113280,"available":772850327552,"internally_reserved":0,"allocated":6217728,"data_stored":3375832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-137598,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":-5,"num_read_kb":-3,"num_write":-107,"num_write_kb":-183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.221284"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.013833+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:41:12.013833+0000","last_peered":"2023-07-18T03:41:12.013833+0000","last_clean":"2023-07-18T03:41:12.013833+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:41:12.013833+0000","last_undegraded":"2023-07-18T03:41:12.013833+0000","last_fullsized":"2023-07-18T03:41:12.013833+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:06:30.033507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-18T03:41:11.646563+0000","last_change":"2023-07-18T03:38:29.341560+0000","last_active":"2023-07-18T03:41:11.646563+0000","last_peered":"2023-07-18T03:41:11.646563+0000","last_clean":"2023-07-18T03:41:11.646563+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:41:11.646563+0000","last_undegraded":"2023-07-18T03:41:11.646563+0000","last_fullsized":"2023-07-18T03:41:11.646563+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:50.606243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.007265+0000","last_change":"2023-07-18T03:38:29.345503+0000","last_active":"2023-07-18T03:41:12.007265+0000","last_peered":"2023-07-18T03:41:12.007265+0000","last_clean":"2023-07-18T03:41:12.007265+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:41:12.007265+0000","last_undegraded":"2023-07-18T03:41:12.007265+0000","last_fullsized":"2023-07-18T03:41:12.007265+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:01:17.117290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.012586+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:41:12.012586+0000","last_peered":"2023-07-18T03:41:12.012586+0000","last_clean":"2023-07-18T03:41:12.012586+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:41:12.012586+0000","last_undegraded":"2023-07-18T03:41:12.012586+0000","last_fullsized":"2023-07-18T03:41:12.012586+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:19:41.650574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.015761+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:41:12.015761+0000","last_peered":"2023-07-18T03:41:12.015761+0000","last_clean":"2023-07-18T03:41:12.015761+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:41:12.015761+0000","last_undegraded":"2023-07-18T03:41:12.015761+0000","last_fullsized":"2023-07-18T03:41:12.015761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:29:32.125624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.012398+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:41:12.012398+0000","last_peered":"2023-07-18T03:41:12.012398+0000","last_clean":"2023-07-18T03:41:12.012398+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:41:12.012398+0000","last_undegraded":"2023-07-18T03:41:12.012398+0000","last_fullsized":"2023-07-18T03:41:12.012398+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:14:41.054472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-18T03:41:11.645503+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:41:11.645503+0000","last_peered":"2023-07-18T03:41:11.645503+0000","last_clean":"2023-07-18T03:41:11.645503+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:41:11.645503+0000","last_undegraded":"2023-07-18T03:41:11.645503+0000","last_fullsized":"2023-07-18T03:41:11.645503+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:39.926749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.438393+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:41:12.438393+0000","last_peered":"2023-07-18T03:41:12.438393+0000","last_clean":"2023-07-18T03:41:12.438393+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:41:12.438393+0000","last_undegraded":"2023-07-18T03:41:12.438393+0000","last_fullsized":"2023-07-18T03:41:12.438393+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:15:38.470063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.438614+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:41:12.438614+0000","last_peered":"2023-07-18T03:41:12.438614+0000","last_clean":"2023-07-18T03:41:12.438614+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:41:12.438614+0000","last_undegraded":"2023-07-18T03:41:12.438614+0000","last_fullsized":"2023-07-18T03:41:12.438614+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:04:44.231921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1232896,"data_stored":869098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85599999999999998,"5min":1.167,"15min":1.167},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.755}]},{"osd":1,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71999999999999997,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.88900000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.873,"15min":0.89500000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.2570000000000001,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":0.70399999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.746,"15min":0.753},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1419999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0249999999999999,"5min":1.365,"15min":1.365},"last":0.90700000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.0010000000000003,"5min":4.0010000000000003,"15min":4.0010000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.72899999999999998,"15min":0.71199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.2989999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.77700000000000002,"15min":0.76000000000000001},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.63500000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.81699999999999995,"15min":0.80000000000000004},"min":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0609999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.86899999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509477,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.73799999999999999,"15min":0.754},"min":{"1min":0.51400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84699999999999998,"5min":2.427,"15min":2.427},"last":0.63100000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.84099999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65000000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95999999999999996,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91300000000000003,"5min":1.034,"15min":1.034},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96399999999999997,"5min":1.113,"15min":1.113},"last":0.81299999999999994}]},{"osd":4,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90400000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82999999999999996}]},{"osd":5,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55900000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.90900000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81000000000000005,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0820000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.755}]}]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":626688,"data_stored":273026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.432,"15min":0.441},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.40899999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.58899999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47399999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.69999999999999996,"15min":0.72799999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.66100000000000003,"5min":3.956,"15min":3.956},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.68400000000000005,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.755,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.497}]},{"osd":4,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.94399999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.86599999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.92900000000000005,"5min":1.22,"15min":1.22},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.88200000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.72499999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91300000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90400000000000003,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.75900000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.94599999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.79900000000000004,"15min":0.81000000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84899999999999998,"5min":1.272,"15min":1.272},"last":0.78400000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1220608,"data_stored":863394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64500000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.46100000000000002,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.434,"15min":0.435},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.60099999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.58399999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.65700000000000003,"5min":0.748,"15min":0.748},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90400000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.59399999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.85099999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":0.73799999999999999},"min":{"1min":0.57199999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84999999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.747,"15min":0.754},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.74199999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88400000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.754,"15min":0.755},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.94999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.75900000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.61499999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.01,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92000000000000004,"5min":1.042,"15min":1.042},"last":0.79200000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":622592,"data_stored":272812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.77800000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.47499999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.81899999999999995}]},{"osd":2,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89400000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.83399999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.81699999999999995,"5min":1.032,"15min":1.032},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.91100000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.503},{"interface":"front","average":{"1min":0.63,"5min":0.70399999999999996,"15min":0.71599999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92700000000000005,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92200000000000004,"5min":0.999,"15min":0.999},"last":0.76200000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":626688,"data_stored":272934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.67500000000000004,"15min":0.63700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.74099999999999999,"15min":0.68700000000000006},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":8.3780000000000001,"5min":8.3780000000000001,"15min":8.3780000000000001},"last":0.628},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.74299999999999999,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.64300000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.77200000000000002,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":4.8769999999999998,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.66100000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70499999999999996,"15min":0.69499999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.52600000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.755,"15min":0.73899999999999999},"min":{"1min":0.41199999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.9690000000000003,"5min":4.9690000000000003,"15min":4.9690000000000003},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.6829999999999998,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71399999999999997,"15min":0.71899999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90800000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.83899999999999997,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.3140000000000001,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.80300000000000005,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":3.7389999999999999,"5min":3.7389999999999999,"15min":3.7389999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94799999999999995,"5min":1.472,"15min":1.472},"last":0.77600000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509478,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31656,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":622592,"data_stored":272812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.47499999999999998,"15min":0.48899999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":1.389,"15min":1.389},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.64500000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.84099999999999997,"15min":0.90300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.74199999999999999,"5min":8.3040000000000003,"15min":8.3040000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.66700000000000004,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.44700000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.77300000000000002,"15min":0.81899999999999995},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63100000000000001,"5min":8.0229999999999997,"15min":8.0229999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.88,"15min":0.93799999999999994},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.70999999999999996,"5min":8.359,"15min":8.359},"last":0.58599999999999997}]},{"osd":4,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97699999999999998,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78400000000000003,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0129999999999999,"5min":1.429,"15min":1.429},"last":0.82699999999999996}]},{"osd":5,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.117,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.747,"5min":0.754,"15min":0.755},"min":{"1min":0.623,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0920000000000001,"5min":1.266,"15min":1.266},"last":0.751}]},{"osd":6,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95199999999999996,"5min":1.752,"15min":1.752},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.626,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0449999999999999,"5min":1.454,"15min":1.454},"last":0.96099999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.58499999999999996,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.94299999999999995,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0249999999999999,"5min":1.222,"15min":1.222},"last":0.84899999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":626688,"data_stored":273026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.503,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.498,"15min":0.497},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72499999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.45500000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.437},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57199999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.622,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78800000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78700000000000003,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0580000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.038,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.86399999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.60799999999999998,"5min":0.504,"15min":0.504},"max":{"1min":1.0209999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.82099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:41:16.806 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:41:17.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.017+0000 7f2c679ff700 1 Processor -- start 2023-07-18T03:41:17.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.017+0000 7f2c679ff700 1 -- start start 2023-07-18T03:41:17.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.018+0000 7f2c679ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c600cfc60 0x7f2c60160860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:17.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.018+0000 7f2c679ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2c6006ae40 0x7f2c600d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:17.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.018+0000 7f2c679ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c600cd4e0 con 0x7f2c600b87d0 2023-07-18T03:41:17.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.018+0000 7f2c679ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c600cdcf0 con 0x7f2c6006ae40 2023-07-18T03:41:17.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.018+0000 7f2c679ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600d00d0 con 0x7f2c600cfc60 2023-07-18T03:41:17.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.019+0000 7f2c65f9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2c6006ae40 0x7f2c600d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:17.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.019+0000 7f2c64f9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c600cfc60 0x7f2c60160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:17.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.019+0000 7f2c64f9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c600cfc60 0x7f2c60160860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:38134/0 (socket says 172.21.15.49:38134) 2023-07-18T03:41:17.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.019+0000 7f2c6579b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f2c600b87d0 0x7f2c60160150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.49:35494/0 (socket says 172.21.15.49:35494) 2023-07-18T03:41:17.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.019+0000 7f2c65f9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2c6006ae40 0x7f2c600d0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.49:36410/0 (socket says 172.21.15.49:36410) 2023-07-18T03:41:17.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.019+0000 7f2c64f9a700 1 -- 172.21.15.49:0/2757665142 learned_addr learned my addr 172.21.15.49:0/2757665142 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:17.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.019+0000 7f2c64f9a700 1 -- 172.21.15.49:0/2757665142 >> v2:172.21.15.107:3300/0 conn(0x7f2c6006ae40 msgr2=0x7f2c600d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:17.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.019+0000 7f2c64f9a700 1 --2- 172.21.15.49:0/2757665142 >> v2:172.21.15.107:3300/0 conn(0x7f2c6006ae40 0x7f2c600d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:17.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.019+0000 7f2c64f9a700 1 -- 172.21.15.49:0/2757665142 >> v1:172.21.15.107:6789/0 conn(0x7f2c600b87d0 legacy=0x7f2c60160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:17.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.019+0000 7f2c64f9a700 1 -- 172.21.15.49:0/2757665142 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c600cd730 con 0x7f2c600cfc60 2023-07-18T03:41:17.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.020+0000 7f2c64f9a700 1 --2- 172.21.15.49:0/2757665142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c600cfc60 0x7f2c60160860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2c500098f0 tx=0x7f2c50009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f9b3b1d9221dfef server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:17.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.020+0000 7f2c57fff700 1 -- 172.21.15.49:0/2757665142 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50011830 con 0x7f2c600cfc60 2023-07-18T03:41:17.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.020+0000 7f2c57fff700 1 -- 172.21.15.49:0/2757665142 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c50011990 con 0x7f2c600cfc60 2023-07-18T03:41:17.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.020+0000 7f2c57fff700 1 -- 172.21.15.49:0/2757665142 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50011d20 con 0x7f2c600cfc60 2023-07-18T03:41:17.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.020+0000 7f2c679ff700 1 -- 172.21.15.49:0/2757665142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c600cfc60 msgr2=0x7f2c60160860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:17.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.020+0000 7f2c679ff700 1 --2- 172.21.15.49:0/2757665142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c600cfc60 0x7f2c60160860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2c500098f0 tx=0x7f2c50009c00 comp rx=0 tx=0).stop 2023-07-18T03:41:17.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.020+0000 7f2c679ff700 1 -- 172.21.15.49:0/2757665142 shutdown_connections 2023-07-18T03:41:17.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.020+0000 7f2c679ff700 1 --2- 172.21.15.49:0/2757665142 >> v2:172.21.15.107:3300/0 conn(0x7f2c6006ae40 0x7f2c600d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:17.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.020+0000 7f2c679ff700 1 --2- 172.21.15.49:0/2757665142 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c600cfc60 0x7f2c60160860 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:17.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.020+0000 7f2c679ff700 1 -- 172.21.15.49:0/2757665142 >> 172.21.15.49:0/2757665142 conn(0x7f2c6006c140 msgr2=0x7f2c6006c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:17.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.021+0000 7f2c679ff700 1 -- 172.21.15.49:0/2757665142 shutdown_connections 2023-07-18T03:41:17.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.021+0000 7f2c679ff700 1 -- 172.21.15.49:0/2757665142 wait complete. 2023-07-18T03:41:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.022+0000 7f2c679ff700 1 Processor -- start 2023-07-18T03:41:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.023+0000 7f2c679ff700 1 -- start start 2023-07-18T03:41:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.023+0000 7f2c679ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c600b87d0 0x7f2c6014f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:17.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c679ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c600cfc60 0x7f2c600cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:17.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c679ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c600cbaf0 0x7f2c600cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:17.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c679ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600cc4d0 con 0x7f2c600cfc60 2023-07-18T03:41:17.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c679ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6014f930 con 0x7f2c600cbaf0 2023-07-18T03:41:17.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c679ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600c9d60 con 0x7f2c600b87d0 2023-07-18T03:41:17.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c64f9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c600cfc60 0x7f2c600cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:17.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c64f9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c600cfc60 0x7f2c600cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58258/0 (socket says 172.21.15.49:58258) 2023-07-18T03:41:17.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c64f9a700 1 -- 172.21.15.49:0/1274080842 learned_addr learned my addr 172.21.15.49:0/1274080842 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:17.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c6579b700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c600b87d0 0x7f2c6014f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:17.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c65f9c700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c600cbaf0 0x7f2c600cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:17.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c64f9a700 1 -- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c600b87d0 msgr2=0x7f2c6014f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:17.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c64f9a700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c600b87d0 0x7f2c6014f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:17.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c64f9a700 1 -- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c600cbaf0 msgr2=0x7f2c600cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:17.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c64f9a700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c600cbaf0 0x7f2c600cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.024+0000 7f2c64f9a700 1 -- 172.21.15.49:0/1274080842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50009550 con 0x7f2c600cfc60 2023-07-18T03:41:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.025+0000 7f2c64f9a700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c600cfc60 0x7f2c600cc850 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2c50000c00 tx=0x7f2c50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.025+0000 7f2c55ffb700 1 -- 172.21.15.49:0/1274080842 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50004400 con 0x7f2c600cfc60 2023-07-18T03:41:17.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.025+0000 7f2c55ffb700 1 -- 172.21.15.49:0/1274080842 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5001f070 con 0x7f2c600cfc60 2023-07-18T03:41:17.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.025+0000 7f2c55ffb700 1 -- 172.21.15.49:0/1274080842 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c500038e0 con 0x7f2c600cfc60 2023-07-18T03:41:17.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.025+0000 7f2c679ff700 1 -- 172.21.15.49:0/1274080842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c600c97a0 con 0x7f2c600cfc60 2023-07-18T03:41:17.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.025+0000 7f2c679ff700 1 -- 172.21.15.49:0/1274080842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c600c70e0 con 0x7f2c600cfc60 2023-07-18T03:41:17.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.026+0000 7f2c679ff700 1 -- 172.21.15.49:0/1274080842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6004e280 con 0x7f2c600cfc60 2023-07-18T03:41:17.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.027+0000 7f2c55ffb700 1 -- 172.21.15.49:0/1274080842 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f2c50021030 con 0x7f2c600cfc60 2023-07-18T03:41:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.028+0000 7f2c55ffb700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2c4406ae00 0x7f2c4406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.028+0000 7f2c55ffb700 1 -- 172.21.15.49:0/1274080842 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2c5008a6c0 con 0x7f2c600cfc60 2023-07-18T03:41:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.029+0000 7f2c6579b700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2c4406ae00 0x7f2c4406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:17.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.029+0000 7f2c6579b700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2c4406ae00 0x7f2c4406d2b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000f60 tx=0x7f2c5c007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:17.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.032+0000 7f2c55ffb700 1 -- 172.21.15.49:0/1274080842 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2c50055970 con 0x7f2c600cfc60 2023-07-18T03:41:17.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.242+0000 7f2c679ff700 1 -- 172.21.15.49:0/1274080842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c6015f9d0 con 0x7f2c600cfc60 2023-07-18T03:41:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.243+0000 7f2c55ffb700 1 -- 172.21.15.49:0/1274080842 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+14674 (secure 0 0 0) 0x7f2c50003a40 con 0x7f2c600cfc60 2023-07-18T03:41:17.238 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:17.239 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":54,"fsid":"e859a751-7613-40d7-973c-3e47a5e301cb","created":"2023-07-18T03:38:15.104315+0000","modified":"2023-07-18T03:41:11.995776+0000","last_up_change":"2023-07-18T03:38:27.318708+0000","last_in_change":"2023-07-18T03:38:18.650495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-18T03:38:27.914778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-18T03:38:28.163363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a464d08b-7474-4b14-a691-ad484e4a088d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6801","nonce":990566862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6803","nonce":990566862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6807","nonce":990566862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":990566862},{"type":"v1","addr":"172.21.15.49:6805","nonce":990566862}]},"public_addr":"172.21.15.49:6801/990566862","cluster_addr":"172.21.15.49:6803/990566862","heartbeat_back_addr":"172.21.15.49:6807/990566862","heartbeat_front_addr":"172.21.15.49:6805/990566862","state":["exists","up"]},{"osd":1,"uuid":"2f096bc0-34c5-4376-baec-56bb92dfdf8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6815","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6817","nonce":2328014854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6819","nonce":2328014854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6823","nonce":2328014854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2328014854},{"type":"v1","addr":"172.21.15.49:6821","nonce":2328014854}]},"public_addr":"172.21.15.49:6817/2328014854","cluster_addr":"172.21.15.49:6819/2328014854","heartbeat_back_addr":"172.21.15.49:6823/2328014854","heartbeat_front_addr":"172.21.15.49:6821/2328014854","state":["exists","up"]},{"osd":2,"uuid":"4389944f-ae37-4b2a-b9a8-c47911d7036c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6825","nonce":46330117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6827","nonce":46330117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6831","nonce":46330117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":46330117},{"type":"v1","addr":"172.21.15.49:6829","nonce":46330117}]},"public_addr":"172.21.15.49:6825/46330117","cluster_addr":"172.21.15.49:6827/46330117","heartbeat_back_addr":"172.21.15.49:6831/46330117","heartbeat_front_addr":"172.21.15.49:6829/46330117","state":["exists","up"]},{"osd":3,"uuid":"2bc9c5af-bca8-4a1d-8bba-47c25a3e281c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6809","nonce":3802584914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6811","nonce":3802584914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6816","nonce":3802584914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3802584914},{"type":"v1","addr":"172.21.15.49:6813","nonce":3802584914}]},"public_addr":"172.21.15.49:6809/3802584914","cluster_addr":"172.21.15.49:6811/3802584914","heartbeat_back_addr":"172.21.15.49:6816/3802584914","heartbeat_front_addr":"172.21.15.49:6813/3802584914","state":["exists","up"]},{"osd":4,"uuid":"03c48b18-d741-436e-a801-ae2aa956ff28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6801","nonce":4156054429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6803","nonce":4156054429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6807","nonce":4156054429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4156054429},{"type":"v1","addr":"172.21.15.107:6805","nonce":4156054429}]},"public_addr":"172.21.15.107:6801/4156054429","cluster_addr":"172.21.15.107:6803/4156054429","heartbeat_back_addr":"172.21.15.107:6807/4156054429","heartbeat_front_addr":"172.21.15.107:6805/4156054429","state":["exists","up"]},{"osd":5,"uuid":"545a1dd0-15cd-4867-912c-525da6ea7752","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6820","nonce":1123907778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6824","nonce":1123907778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6831","nonce":1123907778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1123907778},{"type":"v1","addr":"172.21.15.107:6828","nonce":1123907778}]},"public_addr":"172.21.15.107:6820/1123907778","cluster_addr":"172.21.15.107:6824/1123907778","heartbeat_back_addr":"172.21.15.107:6831/1123907778","heartbeat_front_addr":"172.21.15.107:6828/1123907778","state":["exists","up"]},{"osd":6,"uuid":"19421f1b-8499-4bea-9a1b-3be1118862fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6809","nonce":447642181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6811","nonce":447642181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6815","nonce":447642181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":447642181},{"type":"v1","addr":"172.21.15.107:6813","nonce":447642181}]},"public_addr":"172.21.15.107:6809/447642181","cluster_addr":"172.21.15.107:6811/447642181","heartbeat_back_addr":"172.21.15.107:6815/447642181","heartbeat_front_addr":"172.21.15.107:6813/447642181","state":["exists","up"]},{"osd":7,"uuid":"c1a3e5ec-b97b-473d-8044-08c46c82698b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6817","nonce":2327910262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6821","nonce":2327910262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6827","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6829","nonce":2327910262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6823","nonce":2327910262},{"type":"v1","addr":"172.21.15.107:6825","nonce":2327910262}]},"public_addr":"172.21.15.107:6817/2327910262","cluster_addr":"172.21.15.107:6821/2327910262","heartbeat_back_addr":"172.21.15.107:6829/2327910262","heartbeat_front_addr":"172.21.15.107:6825/2327910262","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.265502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.042456+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.289439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.076890+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.933048+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:23.993337+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.008989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:38:24.031970+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6835/1453078087":"2023-07-19T03:41:08.518573+0000","172.21.15.49:6833/1453078087":"2023-07-19T03:41:08.518573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:41:17.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.245+0000 7f2c679ff700 1 -- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2c4406ae00 msgr2=0x7f2c4406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:17.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.245+0000 7f2c679ff700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2c4406ae00 0x7f2c4406d2b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000f60 tx=0x7f2c5c007ed0 comp rx=0 tx=0).stop 2023-07-18T03:41:17.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.245+0000 7f2c679ff700 1 -- 172.21.15.49:0/1274080842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c600cfc60 msgr2=0x7f2c600cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:17.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.245+0000 7f2c679ff700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c600cfc60 0x7f2c600cc850 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2c50000c00 tx=0x7f2c50004750 comp rx=0 tx=0).stop 2023-07-18T03:41:17.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.245+0000 7f2c679ff700 1 -- 172.21.15.49:0/1274080842 shutdown_connections 2023-07-18T03:41:17.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.245+0000 7f2c679ff700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c600cbaf0 0x7f2c600cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:17.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.245+0000 7f2c679ff700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c600b87d0 0x7f2c6014f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:17.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.245+0000 7f2c679ff700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2c4406ae00 0x7f2c4406d2b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:17.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.245+0000 7f2c679ff700 1 --2- 172.21.15.49:0/1274080842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2c600cfc60 0x7f2c600cc850 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:17.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.245+0000 7f2c679ff700 1 -- 172.21.15.49:0/1274080842 >> 172.21.15.49:0/1274080842 conn(0x7f2c6006c140 msgr2=0x7f2c6015ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:17.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.245+0000 7f2c679ff700 1 -- 172.21.15.49:0/1274080842 shutdown_connections 2023-07-18T03:41:17.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:17.245+0000 7f2c679ff700 1 -- 172.21.15.49:0/1274080842 wait complete. 2023-07-18T03:41:18.259 INFO:tasks.ceph:Scrubbing osd.0 2023-07-18T03:41:18.260 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-07-18T03:41:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.486+0000 7f75fafff700 1 Processor -- start 2023-07-18T03:41:18.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.486+0000 7f75fafff700 1 -- start start 2023-07-18T03:41:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.487+0000 7f75fafff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f75f415c7b0 0x7f75f40d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.487+0000 7f75fafff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75f40d1140 0x7f75f40ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.487+0000 7f75fafff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f40d0190 con 0x7f75f40b1e50 2023-07-18T03:41:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.487+0000 7f75fafff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f40d3de0 con 0x7f75f415c7b0 2023-07-18T03:41:18.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.487+0000 7f75fafff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f40d0470 con 0x7f75f40d1140 2023-07-18T03:41:18.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.488+0000 7f75f959c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75f40d1140 0x7f75f40ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:18.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.488+0000 7f75f8d9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f75f415c7b0 0x7f75f40d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:18.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.488+0000 7f75f959c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75f40d1140 0x7f75f40ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.49:38160/0 (socket says 172.21.15.49:38160) 2023-07-18T03:41:18.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.488+0000 7f75f959c700 1 -- 172.21.15.49:0/3407212095 learned_addr learned my addr 172.21.15.49:0/3407212095 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:18.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.488+0000 7f75f959c700 1 -- 172.21.15.49:0/3407212095 >> v2:172.21.15.107:3300/0 conn(0x7f75f415c7b0 msgr2=0x7f75f40d28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:18.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.488+0000 7f75f959c700 1 --2- 172.21.15.49:0/3407212095 >> v2:172.21.15.107:3300/0 conn(0x7f75f415c7b0 0x7f75f40d28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.488+0000 7f75f959c700 1 -- 172.21.15.49:0/3407212095 >> v1:172.21.15.107:6789/0 conn(0x7f75f40b1e50 legacy=0x7f75f40d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.488+0000 7f75f959c700 1 -- 172.21.15.49:0/3407212095 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f40d0750 con 0x7f75f40d1140 2023-07-18T03:41:18.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.489+0000 7f75f959c700 1 --2- 172.21.15.49:0/3407212095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75f40d1140 0x7f75f40ce030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009e50 tx=0x7f75ec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeb61ae1b067ea15 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.489+0000 7f75f37fe700 1 -- 172.21.15.49:0/3407212095 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec00e070 con 0x7f75f40d1140 2023-07-18T03:41:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.489+0000 7f75f37fe700 1 -- 172.21.15.49:0/3407212095 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec00b910 con 0x7f75f40d1140 2023-07-18T03:41:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.489+0000 7f75f37fe700 1 -- 172.21.15.49:0/3407212095 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec00bcc0 con 0x7f75f40d1140 2023-07-18T03:41:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.490+0000 7f75fafff700 1 -- 172.21.15.49:0/3407212095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75f40d1140 msgr2=0x7f75f40ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.490+0000 7f75fafff700 1 --2- 172.21.15.49:0/3407212095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75f40d1140 0x7f75f40ce030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009e50 tx=0x7f75ec00b330 comp rx=0 tx=0).stop 2023-07-18T03:41:18.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.490+0000 7f75fafff700 1 -- 172.21.15.49:0/3407212095 shutdown_connections 2023-07-18T03:41:18.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.490+0000 7f75fafff700 1 --2- 172.21.15.49:0/3407212095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75f40d1140 0x7f75f40ce030 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.490+0000 7f75fafff700 1 --2- 172.21.15.49:0/3407212095 >> v2:172.21.15.107:3300/0 conn(0x7f75f415c7b0 0x7f75f40d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.490+0000 7f75fafff700 1 -- 172.21.15.49:0/3407212095 >> 172.21.15.49:0/3407212095 conn(0x7f75f409c1b0 msgr2=0x7f75f40d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:18.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.490+0000 7f75fafff700 1 -- 172.21.15.49:0/3407212095 shutdown_connections 2023-07-18T03:41:18.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.490+0000 7f75fafff700 1 -- 172.21.15.49:0/3407212095 wait complete. 2023-07-18T03:41:18.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.492+0000 7f75fafff700 1 Processor -- start 2023-07-18T03:41:18.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.492+0000 7f75fafff700 1 -- start start 2023-07-18T03:41:18.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.493+0000 7f75fafff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75f40b1e50 0x7f75f40cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.493+0000 7f75fafff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75f40d1140 0x7f75f41f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.493+0000 7f75fafff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75f415c7b0 0x7f75f41f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.493+0000 7f75fafff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41f21a0 con 0x7f75f40d1140 2023-07-18T03:41:18.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.493+0000 7f75fafff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f40cc800 con 0x7f75f40b1e50 2023-07-18T03:41:18.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.493+0000 7f75fafff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f415bc80 con 0x7f75f415c7b0 2023-07-18T03:41:18.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.493+0000 7f75f3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75f40d1140 0x7f75f41f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:18.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.493+0000 7f75f959c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75f415c7b0 0x7f75f41f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:18.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.493+0000 7f75f3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75f40d1140 0x7f75f41f16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58274/0 (socket says 172.21.15.49:58274) 2023-07-18T03:41:18.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.493+0000 7f75f8d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75f40b1e50 0x7f75f40cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:18.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.493+0000 7f75f3fff700 1 -- 172.21.15.49:0/3008858253 learned_addr learned my addr 172.21.15.49:0/3008858253 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:18.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.494+0000 7f75f3fff700 1 -- 172.21.15.49:0/3008858253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75f415c7b0 msgr2=0x7f75f41f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.494+0000 7f75f3fff700 1 --2- 172.21.15.49:0/3008858253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75f415c7b0 0x7f75f41f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.494+0000 7f75f3fff700 1 -- 172.21.15.49:0/3008858253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75f40b1e50 msgr2=0x7f75f40cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.494+0000 7f75f3fff700 1 --2- 172.21.15.49:0/3008858253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75f40b1e50 0x7f75f40cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.494+0000 7f75f3fff700 1 -- 172.21.15.49:0/3008858253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec009b30 con 0x7f75f40d1140 2023-07-18T03:41:18.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.494+0000 7f75f3fff700 1 --2- 172.21.15.49:0/3008858253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75f40d1140 0x7f75f41f16f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f75e000c450 tx=0x7f75e000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:18.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.494+0000 7f75f17fa700 1 -- 172.21.15.49:0/3008858253 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e00052d0 con 0x7f75f40d1140 2023-07-18T03:41:18.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.494+0000 7f75f17fa700 1 -- 172.21.15.49:0/3008858253 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e0005430 con 0x7f75f40d1140 2023-07-18T03:41:18.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.494+0000 7f75f17fa700 1 -- 172.21.15.49:0/3008858253 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e00038c0 con 0x7f75f40d1140 2023-07-18T03:41:18.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.494+0000 7f75fafff700 1 -- 172.21.15.49:0/3008858253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f415a2f0 con 0x7f75f40d1140 2023-07-18T03:41:18.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.495+0000 7f75fafff700 1 -- 172.21.15.49:0/3008858253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f415a8b0 con 0x7f75f40d1140 2023-07-18T03:41:18.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.495+0000 7f75fafff700 1 -- 172.21.15.49:0/3008858253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f75f40cd830 con 0x7f75f40d1140 2023-07-18T03:41:18.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.496+0000 7f75f17fa700 1 -- 172.21.15.49:0/3008858253 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f75e0003a20 con 0x7f75f40d1140 2023-07-18T03:41:18.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.498+0000 7f75f17fa700 1 --2- 172.21.15.49:0/3008858253 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f75d806ae00 0x7f75d806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.498+0000 7f75f17fa700 1 -- 172.21.15.49:0/3008858253 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f75e00885f0 con 0x7f75f40d1140 2023-07-18T03:41:18.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.498+0000 7f75f17fa700 1 --2- 172.21.15.49:0/3008858253 >> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] conn(0x7f75d8071270 0x7f75d8073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.498+0000 7f75f17fa700 1 -- 172.21.15.49:0/3008858253 --> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75d8073d50 con 0x7f75d8071270 2023-07-18T03:41:18.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.498+0000 7f75f17fa700 1 -- 172.21.15.49:0/3008858253 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75e008abd0 con 0x7f75f40d1140 2023-07-18T03:41:18.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.498+0000 7f75f8d9b700 1 --2- 172.21.15.49:0/3008858253 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f75d806ae00 0x7f75d806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:18.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.498+0000 7f75f959c700 1 --2- 172.21.15.49:0/3008858253 >> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] conn(0x7f75d8071270 0x7f75d8073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:18.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.499+0000 7f75f8d9b700 1 --2- 172.21.15.49:0/3008858253 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f75d806ae00 0x7f75d806d2b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009be0 tx=0x7f75e8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:18.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.499+0000 7f75f959c700 1 --2- 172.21.15.49:0/3008858253 >> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] conn(0x7f75d8071270 0x7f75d8073680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:18.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.500+0000 7f75f17fa700 1 -- 172.21.15.49:0/3008858253 <== osd.0 v2:172.21.15.49:6800/990566862 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f75d8073d50 con 0x7f75d8071270 2023-07-18T03:41:18.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.515+0000 7f75fafff700 1 -- 172.21.15.49:0/3008858253 --> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f75f40cd550 con 0x7f75d8071270 2023-07-18T03:41:18.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.515+0000 7f75f17fa700 1 -- 172.21.15.49:0/3008858253 <== osd.0 v2:172.21.15.49:6800/990566862 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f75ec00bc00 con 0x7f75d8071270 2023-07-18T03:41:18.510 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:41:18.511 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-18T03:41:18.511 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:41:18.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.515+0000 7f75fafff700 1 -- 172.21.15.49:0/3008858253 >> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] conn(0x7f75d8071270 msgr2=0x7f75d8073680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.515+0000 7f75fafff700 1 --2- 172.21.15.49:0/3008858253 >> [v2:172.21.15.49:6800/990566862,v1:172.21.15.49:6801/990566862] conn(0x7f75d8071270 0x7f75d8073680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.516+0000 7f75fafff700 1 -- 172.21.15.49:0/3008858253 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f75d806ae00 msgr2=0x7f75d806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.516+0000 7f75fafff700 1 --2- 172.21.15.49:0/3008858253 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f75d806ae00 0x7f75d806d2b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009be0 tx=0x7f75e8009210 comp rx=0 tx=0).stop 2023-07-18T03:41:18.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.516+0000 7f75fafff700 1 -- 172.21.15.49:0/3008858253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75f40d1140 msgr2=0x7f75f41f16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.516+0000 7f75fafff700 1 --2- 172.21.15.49:0/3008858253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75f40d1140 0x7f75f41f16f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f75e000c450 tx=0x7f75e000c760 comp rx=0 tx=0).stop 2023-07-18T03:41:18.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.516+0000 7f75f959c700 1 -- 172.21.15.49:0/3008858253 reap_dead start 2023-07-18T03:41:18.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.516+0000 7f75fafff700 1 -- 172.21.15.49:0/3008858253 shutdown_connections 2023-07-18T03:41:18.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.516+0000 7f75fafff700 1 -- 172.21.15.49:0/3008858253 >> 172.21.15.49:0/3008858253 conn(0x7f75f409c1b0 msgr2=0x7f75f40afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:18.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.516+0000 7f75fafff700 1 -- 172.21.15.49:0/3008858253 shutdown_connections 2023-07-18T03:41:18.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.516+0000 7f75fafff700 1 -- 172.21.15.49:0/3008858253 wait complete. 2023-07-18T03:41:18.529 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-07-18T03:41:18.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.740+0000 7f5d247ff700 1 Processor -- start 2023-07-18T03:41:18.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.741+0000 7f5d247ff700 1 -- start start 2023-07-18T03:41:18.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.742+0000 7f5d247ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d1c15d9d0 0x7f5d1c06b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.742+0000 7f5d247ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d1c071170 0x7f5d1c0715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.742+0000 7f5d247ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1c071b00 con 0x7f5d1c06c9f0 2023-07-18T03:41:18.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.742+0000 7f5d247ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d1c06c4e0 con 0x7f5d1c15d9d0 2023-07-18T03:41:18.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.742+0000 7f5d247ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0b2020 con 0x7f5d1c071170 2023-07-18T03:41:18.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.742+0000 7f5d2259b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d1c15d9d0 0x7f5d1c06b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:18.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.742+0000 7f5d2259b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d1c15d9d0 0x7f5d1c06b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58288/0 (socket says 172.21.15.49:58288) 2023-07-18T03:41:18.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.742+0000 7f5d2259b700 1 -- 172.21.15.49:0/3454576250 learned_addr learned my addr 172.21.15.49:0/3454576250 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:18.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.742+0000 7f5d22d9c700 1 --2- 172.21.15.49:0/3454576250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d1c071170 0x7f5d1c0715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:18.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.742+0000 7f5d2259b700 1 -- 172.21.15.49:0/3454576250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d1c071170 msgr2=0x7f5d1c0715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.742+0000 7f5d2259b700 1 --2- 172.21.15.49:0/3454576250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d1c071170 0x7f5d1c0715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.742+0000 7f5d2259b700 1 -- 172.21.15.49:0/3454576250 >> v1:172.21.15.107:6789/0 conn(0x7f5d1c06c9f0 legacy=0x7f5d1c0b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.743+0000 7f5d2259b700 1 -- 172.21.15.49:0/3454576250 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c15dda0 con 0x7f5d1c15d9d0 2023-07-18T03:41:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.743+0000 7f5d22d9c700 1 --2- 172.21.15.49:0/3454576250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d1c071170 0x7f5d1c0715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.743+0000 7f5d2259b700 1 --2- 172.21.15.49:0/3454576250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1c15d9d0 0x7f5d1c06b5b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5d180098f0 tx=0x7f5d18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53ae6e4e7306239 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.743+0000 7f5d21599700 1 -- 172.21.15.49:0/3454576250 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1800b8a0 con 0x7f5d1c15d9d0 2023-07-18T03:41:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.743+0000 7f5d21599700 1 -- 172.21.15.49:0/3454576250 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1800ba00 con 0x7f5d1c15d9d0 2023-07-18T03:41:18.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.743+0000 7f5d21599700 1 -- 172.21.15.49:0/3454576250 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1800bd90 con 0x7f5d1c15d9d0 2023-07-18T03:41:18.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.744+0000 7f5d247ff700 1 -- 172.21.15.49:0/3454576250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1c15d9d0 msgr2=0x7f5d1c06b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.744+0000 7f5d247ff700 1 --2- 172.21.15.49:0/3454576250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1c15d9d0 0x7f5d1c06b5b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5d180098f0 tx=0x7f5d18009c00 comp rx=0 tx=0).stop 2023-07-18T03:41:18.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.744+0000 7f5d247ff700 1 -- 172.21.15.49:0/3454576250 shutdown_connections 2023-07-18T03:41:18.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.744+0000 7f5d247ff700 1 --2- 172.21.15.49:0/3454576250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d1c071170 0x7f5d1c0715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.744+0000 7f5d247ff700 1 --2- 172.21.15.49:0/3454576250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1c15d9d0 0x7f5d1c06b5b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.744+0000 7f5d247ff700 1 -- 172.21.15.49:0/3454576250 >> 172.21.15.49:0/3454576250 conn(0x7f5d1c0b05d0 msgr2=0x7f5d1c0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:18.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.744+0000 7f5d247ff700 1 -- 172.21.15.49:0/3454576250 shutdown_connections 2023-07-18T03:41:18.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.744+0000 7f5d247ff700 1 -- 172.21.15.49:0/3454576250 wait complete. 2023-07-18T03:41:18.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.746+0000 7f5d247ff700 1 Processor -- start 2023-07-18T03:41:18.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.746+0000 7f5d247ff700 1 -- start start 2023-07-18T03:41:18.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d247ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1c06c9f0 0x7f5d1c0c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d247ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d1c071170 0x7f5d1c158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d247ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d1c15d9d0 0x7f5d1c0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d247ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0c3600 con 0x7f5d1c06c9f0 2023-07-18T03:41:18.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d247ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0c2980 con 0x7f5d1c15d9d0 2023-07-18T03:41:18.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d247ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0c2c60 con 0x7f5d1c071170 2023-07-18T03:41:18.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d2259b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1c06c9f0 0x7f5d1c0c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:18.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d2259b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1c06c9f0 0x7f5d1c0c3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58304/0 (socket says 172.21.15.49:58304) 2023-07-18T03:41:18.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d2259b700 1 -- 172.21.15.49:0/2752540700 learned_addr learned my addr 172.21.15.49:0/2752540700 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:18.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d2259b700 1 -- 172.21.15.49:0/2752540700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d1c071170 msgr2=0x7f5d1c158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d2259b700 1 --2- 172.21.15.49:0/2752540700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d1c071170 0x7f5d1c158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d2259b700 1 -- 172.21.15.49:0/2752540700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d1c15d9d0 msgr2=0x7f5d1c0c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d2259b700 1 --2- 172.21.15.49:0/2752540700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d1c15d9d0 0x7f5d1c0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.747+0000 7f5d2259b700 1 -- 172.21.15.49:0/2752540700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d18009550 con 0x7f5d1c06c9f0 2023-07-18T03:41:18.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.748+0000 7f5d2259b700 1 --2- 172.21.15.49:0/2752540700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1c06c9f0 0x7f5d1c0c3a20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009c40 tx=0x7f5d18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:18.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.748+0000 7f5d12ffd700 1 -- 172.21.15.49:0/2752540700 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d180036a0 con 0x7f5d1c06c9f0 2023-07-18T03:41:18.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.748+0000 7f5d12ffd700 1 -- 172.21.15.49:0/2752540700 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1801e070 con 0x7f5d1c06c9f0 2023-07-18T03:41:18.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.748+0000 7f5d247ff700 1 -- 172.21.15.49:0/2752540700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1c0c2f40 con 0x7f5d1c06c9f0 2023-07-18T03:41:18.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.748+0000 7f5d12ffd700 1 -- 172.21.15.49:0/2752540700 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d18003a60 con 0x7f5d1c06c9f0 2023-07-18T03:41:18.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.748+0000 7f5d247ff700 1 -- 172.21.15.49:0/2752540700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1c0c3f60 con 0x7f5d1c06c9f0 2023-07-18T03:41:18.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.749+0000 7f5d247ff700 1 -- 172.21.15.49:0/2752540700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1c0c3740 con 0x7f5d1c06c9f0 2023-07-18T03:41:18.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.750+0000 7f5d12ffd700 1 -- 172.21.15.49:0/2752540700 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f5d18003bc0 con 0x7f5d1c06c9f0 2023-07-18T03:41:18.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.752+0000 7f5d12ffd700 1 --2- 172.21.15.49:0/2752540700 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5d0006ae00 0x7f5d0006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:18.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.752+0000 7f5d12ffd700 1 -- 172.21.15.49:0/2752540700 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5d18020070 con 0x7f5d1c06c9f0 2023-07-18T03:41:18.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.752+0000 7f5d21d9a700 1 --2- 172.21.15.49:0/2752540700 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5d0006ae00 0x7f5d0006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:18.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.753+0000 7f5d21d9a700 1 --2- 172.21.15.49:0/2752540700 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5d0006ae00 0x7f5d0006d2b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c000f60 tx=0x7f5d0c007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:18.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.756+0000 7f5d12ffd700 1 -- 172.21.15.49:0/2752540700 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5d1c0c3740 con 0x7f5d1c06c9f0 2023-07-18T03:41:18.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.963+0000 7f5d247ff700 1 -- 172.21.15.49:0/2752540700 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d1c06c200 con 0x7f5d0006ae00 2023-07-18T03:41:18.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.964+0000 7f5d12ffd700 1 -- 172.21.15.49:0/2752540700 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5d0c005080 con 0x7f5d0006ae00 2023-07-18T03:41:18.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.966+0000 7f5d247ff700 1 -- 172.21.15.49:0/2752540700 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5d0006ae00 msgr2=0x7f5d0006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.966+0000 7f5d247ff700 1 --2- 172.21.15.49:0/2752540700 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5d0006ae00 0x7f5d0006d2b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c000f60 tx=0x7f5d0c007a40 comp rx=0 tx=0).stop 2023-07-18T03:41:18.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.966+0000 7f5d247ff700 1 -- 172.21.15.49:0/2752540700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1c06c9f0 msgr2=0x7f5d1c0c3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:18.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.966+0000 7f5d247ff700 1 --2- 172.21.15.49:0/2752540700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1c06c9f0 0x7f5d1c0c3a20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009c40 tx=0x7f5d18004750 comp rx=0 tx=0).stop 2023-07-18T03:41:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.966+0000 7f5d247ff700 1 -- 172.21.15.49:0/2752540700 shutdown_connections 2023-07-18T03:41:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.966+0000 7f5d247ff700 1 --2- 172.21.15.49:0/2752540700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d1c15d9d0 0x7f5d1c0c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.966+0000 7f5d247ff700 1 --2- 172.21.15.49:0/2752540700 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f5d0006ae00 0x7f5d0006d2b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.966+0000 7f5d247ff700 1 --2- 172.21.15.49:0/2752540700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1c06c9f0 0x7f5d1c0c3a20 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.966+0000 7f5d247ff700 1 --2- 172.21.15.49:0/2752540700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d1c071170 0x7f5d1c158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:18.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.966+0000 7f5d247ff700 1 -- 172.21.15.49:0/2752540700 >> 172.21.15.49:0/2752540700 conn(0x7f5d1c0b05d0 msgr2=0x7f5d1c0b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:18.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.966+0000 7f5d247ff700 1 -- 172.21.15.49:0/2752540700 shutdown_connections 2023-07-18T03:41:18.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:18.966+0000 7f5d247ff700 1 -- 172.21.15.49:0/2752540700 wait complete. 2023-07-18T03:41:18.963 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 0 to deep-scrub 2023-07-18T03:41:18.978 INFO:tasks.ceph:Scrubbing osd.1 2023-07-18T03:41:18.979 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-07-18T03:41:19.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.194+0000 7f1e84fff700 1 Processor -- start 2023-07-18T03:41:19.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.195+0000 7f1e84fff700 1 -- start start 2023-07-18T03:41:19.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.196+0000 7f1e84fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1e800add00 0x7f1e800c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.196+0000 7f1e84fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e800cd1c0 0x7f1e8006c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.196+0000 7f1e84fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8006fdb0 con 0x7f1e8006c800 2023-07-18T03:41:19.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.196+0000 7f1e84fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e80159880 con 0x7f1e800cd1c0 2023-07-18T03:41:19.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.196+0000 7f1e84fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e8006cc40 con 0x7f1e800add00 2023-07-18T03:41:19.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.196+0000 7f1e77fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e800cd1c0 0x7f1e8006c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.196+0000 7f1e77fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e800cd1c0 0x7f1e8006c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47638/0 (socket says 172.21.15.49:47638) 2023-07-18T03:41:19.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.196+0000 7f1e77fff700 1 -- 172.21.15.49:0/2023958839 learned_addr learned my addr 172.21.15.49:0/2023958839 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:19.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.196+0000 7f1e7e59c700 1 --2- 172.21.15.49:0/2023958839 >> v2:172.21.15.107:3300/0 conn(0x7f1e800add00 0x7f1e800c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.197+0000 7f1e77fff700 1 -- 172.21.15.49:0/2023958839 >> v2:172.21.15.107:3300/0 conn(0x7f1e800add00 msgr2=0x7f1e800c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:19.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.197+0000 7f1e77fff700 1 --2- 172.21.15.49:0/2023958839 >> v2:172.21.15.107:3300/0 conn(0x7f1e800add00 0x7f1e800c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.197+0000 7f1e77fff700 1 -- 172.21.15.49:0/2023958839 >> v1:172.21.15.107:6789/0 conn(0x7f1e8006c800 legacy=0x7f1e800b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:19.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.197+0000 7f1e77fff700 1 -- 172.21.15.49:0/2023958839 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e800ae0d0 con 0x7f1e800cd1c0 2023-07-18T03:41:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.197+0000 7f1e7e59c700 1 --2- 172.21.15.49:0/2023958839 >> v2:172.21.15.107:3300/0 conn(0x7f1e800add00 0x7f1e800c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.197+0000 7f1e77fff700 1 --2- 172.21.15.49:0/2023958839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e800cd1c0 0x7f1e8006c2c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1e80073b80 tx=0x7f1e68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6b0cc46a7c250 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:19.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.197+0000 7f1e7dd9b700 1 -- 172.21.15.49:0/2023958839 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68004230 con 0x7f1e800cd1c0 2023-07-18T03:41:19.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.197+0000 7f1e7dd9b700 1 -- 172.21.15.49:0/2023958839 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68004390 con 0x7f1e800cd1c0 2023-07-18T03:41:19.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.197+0000 7f1e7dd9b700 1 -- 172.21.15.49:0/2023958839 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680036a0 con 0x7f1e800cd1c0 2023-07-18T03:41:19.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.198+0000 7f1e84fff700 1 -- 172.21.15.49:0/2023958839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e800cd1c0 msgr2=0x7f1e8006c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.198+0000 7f1e84fff700 1 --2- 172.21.15.49:0/2023958839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e800cd1c0 0x7f1e8006c2c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1e80073b80 tx=0x7f1e68009bd0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.198+0000 7f1e84fff700 1 -- 172.21.15.49:0/2023958839 shutdown_connections 2023-07-18T03:41:19.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.198+0000 7f1e84fff700 1 --2- 172.21.15.49:0/2023958839 >> v2:172.21.15.107:3300/0 conn(0x7f1e800add00 0x7f1e800c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.198+0000 7f1e84fff700 1 --2- 172.21.15.49:0/2023958839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e800cd1c0 0x7f1e8006c2c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.198+0000 7f1e84fff700 1 -- 172.21.15.49:0/2023958839 >> 172.21.15.49:0/2023958839 conn(0x7f1e800cea00 msgr2=0x7f1e800cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:19.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.198+0000 7f1e84fff700 1 -- 172.21.15.49:0/2023958839 shutdown_connections 2023-07-18T03:41:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.198+0000 7f1e84fff700 1 -- 172.21.15.49:0/2023958839 wait complete. 2023-07-18T03:41:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.200+0000 7f1e84fff700 1 Processor -- start 2023-07-18T03:41:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.200+0000 7f1e84fff700 1 -- start start 2023-07-18T03:41:19.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e84fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e8006c800 0x7f1e801f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e84fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e800add00 0x7f1e800aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e84fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e800cd1c0 0x7f1e80158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e84fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e800aefe0 con 0x7f1e800add00 2023-07-18T03:41:19.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e84fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e80158970 con 0x7f1e800cd1c0 2023-07-18T03:41:19.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e84fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e80157860 con 0x7f1e8006c800 2023-07-18T03:41:19.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e77fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e800add00 0x7f1e800aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e77fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e800add00 0x7f1e800aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47640/0 (socket says 172.21.15.49:47640) 2023-07-18T03:41:19.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e77fff700 1 -- 172.21.15.49:0/2926683340 learned_addr learned my addr 172.21.15.49:0/2926683340 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:19.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e7e59c700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e8006c800 0x7f1e801f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e7ed9d700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e800cd1c0 0x7f1e80158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e77fff700 1 -- 172.21.15.49:0/2926683340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e8006c800 msgr2=0x7f1e801f19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e77fff700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e8006c800 0x7f1e801f19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e77fff700 1 -- 172.21.15.49:0/2926683340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e800cd1c0 msgr2=0x7f1e80158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e77fff700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e800cd1c0 0x7f1e80158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e77fff700 1 -- 172.21.15.49:0/2926683340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68009550 con 0x7f1e800add00 2023-07-18T03:41:19.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.201+0000 7f1e7ed9d700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e800cd1c0 0x7f1e80158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:19.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.202+0000 7f1e77fff700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e800add00 0x7f1e800aeaa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1e800673b0 tx=0x7f1e68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:19.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.202+0000 7f1e76ffd700 1 -- 172.21.15.49:0/2926683340 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68003e90 con 0x7f1e800add00 2023-07-18T03:41:19.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.202+0000 7f1e76ffd700 1 -- 172.21.15.49:0/2926683340 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68011630 con 0x7f1e800add00 2023-07-18T03:41:19.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.202+0000 7f1e76ffd700 1 -- 172.21.15.49:0/2926683340 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680119c0 con 0x7f1e800add00 2023-07-18T03:41:19.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.202+0000 7f1e84fff700 1 -- 172.21.15.49:0/2926683340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e80158e60 con 0x7f1e800add00 2023-07-18T03:41:19.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.202+0000 7f1e84fff700 1 -- 172.21.15.49:0/2926683340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e800b27a0 con 0x7f1e800add00 2023-07-18T03:41:19.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.203+0000 7f1e84fff700 1 -- 172.21.15.49:0/2926683340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e8006f760 con 0x7f1e800add00 2023-07-18T03:41:19.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.204+0000 7f1e76ffd700 1 -- 172.21.15.49:0/2926683340 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f1e68011b20 con 0x7f1e800add00 2023-07-18T03:41:19.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.205+0000 7f1e76ffd700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e6006ae00 0x7f1e6006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.205+0000 7f1e76ffd700 1 -- 172.21.15.49:0/2926683340 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1e6808acb0 con 0x7f1e800add00 2023-07-18T03:41:19.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.205+0000 7f1e76ffd700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] conn(0x7f1e60071100 0x7f1e60073510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.205+0000 7f1e76ffd700 1 -- 172.21.15.49:0/2926683340 --> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e60073be0 con 0x7f1e60071100 2023-07-18T03:41:19.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.205+0000 7f1e76ffd700 1 -- 172.21.15.49:0/2926683340 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e6808b020 con 0x7f1e800add00 2023-07-18T03:41:19.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.205+0000 7f1e7e59c700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e6006ae00 0x7f1e6006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.206+0000 7f1e7ed9d700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] conn(0x7f1e60071100 0x7f1e60073510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.206+0000 7f1e7e59c700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e6006ae00 0x7f1e6006d2b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009990 tx=0x7f1e70008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:19.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.206+0000 7f1e7ed9d700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] conn(0x7f1e60071100 0x7f1e60073510 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:19.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.207+0000 7f1e76ffd700 1 -- 172.21.15.49:0/2926683340 <== osd.1 v2:172.21.15.49:6815/2328014854 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1e60073be0 con 0x7f1e60071100 2023-07-18T03:41:19.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.222+0000 7f1e84fff700 1 -- 172.21.15.49:0/2926683340 --> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1e800cd9a0 con 0x7f1e60071100 2023-07-18T03:41:19.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.223+0000 7f1e76ffd700 1 -- 172.21.15.49:0/2926683340 <== osd.1 v2:172.21.15.49:6815/2328014854 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1e78004660 con 0x7f1e60071100 2023-07-18T03:41:19.219 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:41:19.220 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-18T03:41:19.220 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:41:19.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.223+0000 7f1e84fff700 1 -- 172.21.15.49:0/2926683340 >> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] conn(0x7f1e60071100 msgr2=0x7f1e60073510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.223+0000 7f1e84fff700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.49:6815/2328014854,v1:172.21.15.49:6817/2328014854] conn(0x7f1e60071100 0x7f1e60073510 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.223+0000 7f1e84fff700 1 -- 172.21.15.49:0/2926683340 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e6006ae00 msgr2=0x7f1e6006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.223+0000 7f1e84fff700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f1e6006ae00 0x7f1e6006d2b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009990 tx=0x7f1e70008040 comp rx=0 tx=0).stop 2023-07-18T03:41:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.223+0000 7f1e84fff700 1 -- 172.21.15.49:0/2926683340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e800add00 msgr2=0x7f1e800aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.223+0000 7f1e84fff700 1 --2- 172.21.15.49:0/2926683340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e800add00 0x7f1e800aeaa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1e800673b0 tx=0x7f1e68004750 comp rx=0 tx=0).stop 2023-07-18T03:41:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.223+0000 7f1e7ed9d700 1 -- 172.21.15.49:0/2926683340 reap_dead start 2023-07-18T03:41:19.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.223+0000 7f1e84fff700 1 -- 172.21.15.49:0/2926683340 shutdown_connections 2023-07-18T03:41:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.223+0000 7f1e84fff700 1 -- 172.21.15.49:0/2926683340 >> 172.21.15.49:0/2926683340 conn(0x7f1e800cea00 msgr2=0x7f1e80070660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.223+0000 7f1e84fff700 1 -- 172.21.15.49:0/2926683340 shutdown_connections 2023-07-18T03:41:19.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.223+0000 7f1e84fff700 1 -- 172.21.15.49:0/2926683340 wait complete. 2023-07-18T03:41:19.235 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-07-18T03:41:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.446+0000 7fd12fdff700 1 Processor -- start 2023-07-18T03:41:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.446+0000 7fd12fdff700 1 -- start start 2023-07-18T03:41:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.447+0000 7fd12fdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd1280d0f90 0x7fd1280b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.447+0000 7fd12fdff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1280b8e30 con 0x7fd1280b8a60 2023-07-18T03:41:19.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.447+0000 7fd12fdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1280afaa0 con 0x7fd1280d0b10 2023-07-18T03:41:19.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.447+0000 7fd12fdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1280afd80 con 0x7fd1280d0f90 2023-07-18T03:41:19.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.448+0000 7fd12db9b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd1280b8a60 0x7fd1280b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44458/0 (socket says 172.21.15.49:44458) 2023-07-18T03:41:19.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.448+0000 7fd12db9b700 1 -- 172.21.15.49:0/2526841901 learned_addr learned my addr 172.21.15.49:0/2526841901 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:19.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.448+0000 7fd12e39c700 1 --2- 172.21.15.49:0/2526841901 >> v2:172.21.15.107:3300/0 conn(0x7fd1280d0f90 0x7fd1280b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.448+0000 7fd12cb99700 1 -- 172.21.15.49:0/2526841901 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910389722 0 0) 0x7fd1280b8e30 con 0x7fd1280b8a60 2023-07-18T03:41:19.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.448+0000 7fd12cb99700 1 -- 172.21.15.49:0/2526841901 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1140035b0 con 0x7fd1280b8a60 2023-07-18T03:41:19.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.448+0000 7fd12cb99700 1 -- 172.21.15.49:0/2526841901 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921432823 0 0) 0x7fd1280afaa0 con 0x7fd1280d0b10 2023-07-18T03:41:19.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.448+0000 7fd12cb99700 1 -- 172.21.15.49:0/2526841901 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1280b8e30 con 0x7fd1280d0b10 2023-07-18T03:41:19.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.449+0000 7fd12cb99700 1 -- 172.21.15.49:0/2526841901 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3330538132 0 0) 0x7fd1140035b0 con 0x7fd1280b8a60 2023-07-18T03:41:19.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.449+0000 7fd12e39c700 1 -- 172.21.15.49:0/2526841901 >> v1:172.21.15.107:6789/0 conn(0x7fd1280d0b10 legacy=0x7fd1280b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.449+0000 7fd12e39c700 1 -- 172.21.15.49:0/2526841901 >> v1:172.21.15.49:6789/0 conn(0x7fd1280b8a60 legacy=0x7fd1280b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.449+0000 7fd12e39c700 1 -- 172.21.15.49:0/2526841901 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1280b0060 con 0x7fd1280d0f90 2023-07-18T03:41:19.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.449+0000 7fd12e39c700 1 --2- 172.21.15.49:0/2526841901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1280d0f90 0x7fd1280b1530 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd120009d90 tx=0x7fd12000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d89b3ce80d31254 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:19.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.449+0000 7fd12cb99700 1 -- 172.21.15.49:0/2526841901 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12000e070 con 0x7fd1280d0f90 2023-07-18T03:41:19.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.449+0000 7fd12cb99700 1 -- 172.21.15.49:0/2526841901 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd120003a20 con 0x7fd1280d0f90 2023-07-18T03:41:19.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.449+0000 7fd12cb99700 1 -- 172.21.15.49:0/2526841901 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124001670 con 0x7fd1280d0f90 2023-07-18T03:41:19.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.450+0000 7fd12fdff700 1 -- 172.21.15.49:0/2526841901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1280d0f90 msgr2=0x7fd1280b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.450+0000 7fd12fdff700 1 --2- 172.21.15.49:0/2526841901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1280d0f90 0x7fd1280b1530 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd120009d90 tx=0x7fd12000b330 comp rx=0 tx=0).stop 2023-07-18T03:41:19.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.450+0000 7fd12fdff700 1 -- 172.21.15.49:0/2526841901 shutdown_connections 2023-07-18T03:41:19.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.450+0000 7fd12fdff700 1 --2- 172.21.15.49:0/2526841901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1280d0f90 0x7fd1280b1530 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.450+0000 7fd12fdff700 1 -- 172.21.15.49:0/2526841901 >> 172.21.15.49:0/2526841901 conn(0x7fd12809c1b0 msgr2=0x7fd1280d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:19.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.450+0000 7fd12fdff700 1 -- 172.21.15.49:0/2526841901 shutdown_connections 2023-07-18T03:41:19.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.450+0000 7fd12fdff700 1 -- 172.21.15.49:0/2526841901 wait complete. 2023-07-18T03:41:19.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.452+0000 7fd12fdff700 1 Processor -- start 2023-07-18T03:41:19.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.452+0000 7fd12fdff700 1 -- start start 2023-07-18T03:41:19.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.453+0000 7fd12fdff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1280b8a60 0x7fd1280c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.453+0000 7fd12fdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1280d0b10 0x7fd128143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.453+0000 7fd12fdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1280d0f90 0x7fd1280ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.453+0000 7fd12fdff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12815e220 con 0x7fd1280b8a60 2023-07-18T03:41:19.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.453+0000 7fd12fdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12815e500 con 0x7fd1280d0b10 2023-07-18T03:41:19.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.453+0000 7fd12fdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12815e7e0 con 0x7fd1280d0f90 2023-07-18T03:41:19.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.453+0000 7fd12db9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1280b8a60 0x7fd1280c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.454+0000 7fd12db9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1280b8a60 0x7fd1280c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47648/0 (socket says 172.21.15.49:47648) 2023-07-18T03:41:19.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.454+0000 7fd12db9b700 1 -- 172.21.15.49:0/2640985295 learned_addr learned my addr 172.21.15.49:0/2640985295 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:19.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.454+0000 7fd12d39a700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1280d0b10 0x7fd128143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.454+0000 7fd12e39c700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1280d0f90 0x7fd1280ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.454+0000 7fd12db9b700 1 -- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1280d0f90 msgr2=0x7fd1280ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.454+0000 7fd12db9b700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1280d0f90 0x7fd1280ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.454+0000 7fd12db9b700 1 -- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1280d0b10 msgr2=0x7fd128143190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.454+0000 7fd12db9b700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1280d0b10 0x7fd128143190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.454+0000 7fd12db9b700 1 -- 172.21.15.49:0/2640985295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd120009a70 con 0x7fd1280b8a60 2023-07-18T03:41:19.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.454+0000 7fd12db9b700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1280b8a60 0x7fd1280c6e40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd12400c750 tx=0x7fd12400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:19.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.454+0000 7fd11e7fc700 1 -- 172.21.15.49:0/2640985295 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124004c80 con 0x7fd1280b8a60 2023-07-18T03:41:19.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.455+0000 7fd11e7fc700 1 -- 172.21.15.49:0/2640985295 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd124004de0 con 0x7fd1280b8a60 2023-07-18T03:41:19.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.455+0000 7fd12fdff700 1 -- 172.21.15.49:0/2640985295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12815eac0 con 0x7fd1280b8a60 2023-07-18T03:41:19.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.455+0000 7fd11e7fc700 1 -- 172.21.15.49:0/2640985295 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124007310 con 0x7fd1280b8a60 2023-07-18T03:41:19.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.455+0000 7fd12fdff700 1 -- 172.21.15.49:0/2640985295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1281438b0 con 0x7fd1280b8a60 2023-07-18T03:41:19.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.456+0000 7fd12fdff700 1 -- 172.21.15.49:0/2640985295 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd128074350 con 0x7fd1280b8a60 2023-07-18T03:41:19.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.457+0000 7fd11e7fc700 1 -- 172.21.15.49:0/2640985295 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fd124007470 con 0x7fd1280b8a60 2023-07-18T03:41:19.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.458+0000 7fd11e7fc700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd10c06ae00 0x7fd10c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.458+0000 7fd11e7fc700 1 -- 172.21.15.49:0/2640985295 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd124088ff0 con 0x7fd1280b8a60 2023-07-18T03:41:19.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.458+0000 7fd12d39a700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd10c06ae00 0x7fd10c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.459+0000 7fd12d39a700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd10c06ae00 0x7fd10c06d2b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd12815fc50 tx=0x7fd118009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:19.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.462+0000 7fd11e7fc700 1 -- 172.21.15.49:0/2640985295 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd124054290 con 0x7fd1280b8a60 2023-07-18T03:41:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.668+0000 7fd12fdff700 1 -- 172.21.15.49:0/2640985295 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1280b0620 con 0x7fd10c06ae00 2023-07-18T03:41:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.669+0000 7fd11e7fc700 1 -- 172.21.15.49:0/2640985295 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd118001cd0 con 0x7fd10c06ae00 2023-07-18T03:41:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.671+0000 7fd12fdff700 1 -- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd10c06ae00 msgr2=0x7fd10c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.671+0000 7fd12fdff700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd10c06ae00 0x7fd10c06d2b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd12815fc50 tx=0x7fd118009040 comp rx=0 tx=0).stop 2023-07-18T03:41:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.671+0000 7fd12fdff700 1 -- 172.21.15.49:0/2640985295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1280b8a60 msgr2=0x7fd1280c6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.671+0000 7fd12fdff700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1280b8a60 0x7fd1280c6e40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd12400c750 tx=0x7fd12400ca60 comp rx=0 tx=0).stop 2023-07-18T03:41:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.671+0000 7fd12fdff700 1 -- 172.21.15.49:0/2640985295 shutdown_connections 2023-07-18T03:41:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.671+0000 7fd12fdff700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1280d0f90 0x7fd1280ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.671+0000 7fd12fdff700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fd10c06ae00 0x7fd10c06d2b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.671+0000 7fd12fdff700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1280b8a60 0x7fd1280c6e40 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.671+0000 7fd12fdff700 1 --2- 172.21.15.49:0/2640985295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1280d0b10 0x7fd128143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.671+0000 7fd12fdff700 1 -- 172.21.15.49:0/2640985295 >> 172.21.15.49:0/2640985295 conn(0x7fd12809c1b0 msgr2=0x7fd12806af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.671+0000 7fd12fdff700 1 -- 172.21.15.49:0/2640985295 shutdown_connections 2023-07-18T03:41:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.671+0000 7fd12fdff700 1 -- 172.21.15.49:0/2640985295 wait complete. 2023-07-18T03:41:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 1 to deep-scrub 2023-07-18T03:41:19.683 INFO:tasks.ceph:Scrubbing osd.2 2023-07-18T03:41:19.684 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-07-18T03:41:19.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.898+0000 7f9d709ff700 1 Processor -- start 2023-07-18T03:41:19.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.898+0000 7f9d709ff700 1 -- start start 2023-07-18T03:41:19.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.900+0000 7f9d709ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9d6c0d1140 0x7f9d6c0d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.900+0000 7f9d709ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6c0d0190 con 0x7f9d6c15c7b0 2023-07-18T03:41:19.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.900+0000 7f9d709ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6c0d0470 con 0x7f9d6c0b1e50 2023-07-18T03:41:19.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.900+0000 7f9d709ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6c0d3de0 con 0x7f9d6c0d1140 2023-07-18T03:41:19.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.900+0000 7f9d6a59c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9d6c15c7b0 0x7f9d6c0d0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44462/0 (socket says 172.21.15.49:44462) 2023-07-18T03:41:19.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.900+0000 7f9d6a59c700 1 -- 172.21.15.49:0/3050978579 learned_addr learned my addr 172.21.15.49:0/3050978579 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:19.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d63fff700 1 --2- 172.21.15.49:0/3050978579 >> v2:172.21.15.107:3300/0 conn(0x7f9d6c0d1140 0x7f9d6c0d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548096902 0 0) 0x7f9d6c0d0190 con 0x7f9d6c15c7b0 2023-07-18T03:41:19.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d580035b0 con 0x7f9d6c15c7b0 2023-07-18T03:41:19.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2393700992 0 0) 0x7f9d580035b0 con 0x7f9d6c15c7b0 2023-07-18T03:41:19.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 >> v2:172.21.15.107:3300/0 conn(0x7f9d6c0d1140 msgr2=0x7f9d6c0d28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:19.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 --2- 172.21.15.49:0/3050978579 >> v2:172.21.15.107:3300/0 conn(0x7f9d6c0d1140 0x7f9d6c0d28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 >> v1:172.21.15.107:6789/0 conn(0x7f9d6c0b1e50 legacy=0x7f9d6c0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0d0750 con 0x7f9d6c15c7b0 2023-07-18T03:41:19.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f9d5c0019f0 con 0x7f9d6c15c7b0 2023-07-18T03:41:19.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 >> v1:172.21.15.49:6789/0 conn(0x7f9d6c15c7b0 legacy=0x7f9d6c0d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 --2- 172.21.15.49:0/3050978579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d58003b10 0x7f9d58003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 --2- 172.21.15.49:0/3050978579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d58004850 0x7f9d58004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 --2- 172.21.15.49:0/3050978579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d58005250 0x7f9d5800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0d3de0 con 0x7f9d58005250 2023-07-18T03:41:19.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5800c190 con 0x7f9d58004850 2023-07-18T03:41:19.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5800c350 con 0x7f9d58003b10 2023-07-18T03:41:19.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d6ad9d700 1 --2- 172.21.15.49:0/3050978579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d58005250 0x7f9d5800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d63fff700 1 --2- 172.21.15.49:0/3050978579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d58004850 0x7f9d58004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.901+0000 7f9d6a59c700 1 --2- 172.21.15.49:0/3050978579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d58003b10 0x7f9d58003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.902+0000 7f9d6ad9d700 1 -- 172.21.15.49:0/3050978579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d58003b10 msgr2=0x7f9d58003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:19.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.902+0000 7f9d6ad9d700 1 --2- 172.21.15.49:0/3050978579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d58003b10 0x7f9d58003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.902+0000 7f9d6ad9d700 1 -- 172.21.15.49:0/3050978579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d58004850 msgr2=0x7f9d58004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:19.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.902+0000 7f9d6ad9d700 1 --2- 172.21.15.49:0/3050978579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d58004850 0x7f9d58004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.902+0000 7f9d6ad9d700 1 -- 172.21.15.49:0/3050978579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d580035b0 con 0x7f9d58005250 2023-07-18T03:41:19.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.902+0000 7f9d6ad9d700 1 -- 172.21.15.49:0/3050978579 reap_dead start 2023-07-18T03:41:19.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.902+0000 7f9d6ad9d700 1 --2- 172.21.15.49:0/3050978579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d58005250 0x7f9d5800ba10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9d64007fd0 tx=0x7f9d58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10db90c7ba8e431c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:19.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.902+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d640052d0 con 0x7f9d58005250 2023-07-18T03:41:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.902+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d64005450 con 0x7f9d58005250 2023-07-18T03:41:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.902+0000 7f9d69d9b700 1 -- 172.21.15.49:0/3050978579 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d640038c0 con 0x7f9d58005250 2023-07-18T03:41:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.903+0000 7f9d709ff700 1 -- 172.21.15.49:0/3050978579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d58005250 msgr2=0x7f9d5800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.903+0000 7f9d709ff700 1 --2- 172.21.15.49:0/3050978579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d58005250 0x7f9d5800ba10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9d64007fd0 tx=0x7f9d58005880 comp rx=0 tx=0).stop 2023-07-18T03:41:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.903+0000 7f9d709ff700 1 -- 172.21.15.49:0/3050978579 shutdown_connections 2023-07-18T03:41:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.903+0000 7f9d709ff700 1 --2- 172.21.15.49:0/3050978579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d58005250 0x7f9d5800ba10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.903+0000 7f9d709ff700 1 -- 172.21.15.49:0/3050978579 >> 172.21.15.49:0/3050978579 conn(0x7f9d6c09c1b0 msgr2=0x7f9d6c0d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:19.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.903+0000 7f9d709ff700 1 -- 172.21.15.49:0/3050978579 shutdown_connections 2023-07-18T03:41:19.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.903+0000 7f9d709ff700 1 -- 172.21.15.49:0/3050978579 wait complete. 2023-07-18T03:41:19.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.905+0000 7f9d709ff700 1 Processor -- start 2023-07-18T03:41:19.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.905+0000 7f9d709ff700 1 -- start start 2023-07-18T03:41:19.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.906+0000 7f9d709ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d58005250 0x7f9d6c14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.906+0000 7f9d709ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d6c0c85c0 0x7f9d6c156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.906+0000 7f9d709ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c0c8a30 0x7f9d6c1574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.906+0000 7f9d709ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c155310 con 0x7f9d58005250 2023-07-18T03:41:19.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.906+0000 7f9d709ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0c7520 con 0x7f9d6c0c8a30 2023-07-18T03:41:19.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.906+0000 7f9d709ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0c7800 con 0x7f9d6c0c85c0 2023-07-18T03:41:19.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.906+0000 7f9d6a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d58005250 0x7f9d6c14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.906+0000 7f9d6a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d58005250 0x7f9d6c14de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47662/0 (socket says 172.21.15.49:47662) 2023-07-18T03:41:19.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.906+0000 7f9d6a59c700 1 -- 172.21.15.49:0/4045389094 learned_addr learned my addr 172.21.15.49:0/4045389094 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:19.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d63fff700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d6c0c85c0 0x7f9d6c156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d6ad9d700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c0c8a30 0x7f9d6c1574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d6a59c700 1 -- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d6c0c85c0 msgr2=0x7f9d6c156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d6a59c700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d6c0c85c0 0x7f9d6c156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d6a59c700 1 -- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c0c8a30 msgr2=0x7f9d6c1574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d6a59c700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c0c8a30 0x7f9d6c1574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d6a59c700 1 -- 172.21.15.49:0/4045389094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0c7ae0 con 0x7f9d58005250 2023-07-18T03:41:19.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d6ad9d700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c0c8a30 0x7f9d6c1574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:19.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d63fff700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d6c0c85c0 0x7f9d6c156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:19.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d6a59c700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d58005250 0x7f9d6c14de60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c000f80 tx=0x7f9d5c00a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:19.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d62ffd700 1 -- 172.21.15.49:0/4045389094 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0133f0 con 0x7f9d58005250 2023-07-18T03:41:19.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d62ffd700 1 -- 172.21.15.49:0/4045389094 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c013590 con 0x7f9d58005250 2023-07-18T03:41:19.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d62ffd700 1 -- 172.21.15.49:0/4045389094 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c013940 con 0x7f9d58005250 2023-07-18T03:41:19.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d709ff700 1 -- 172.21.15.49:0/4045389094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c0c7dc0 con 0x7f9d58005250 2023-07-18T03:41:19.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.907+0000 7f9d709ff700 1 -- 172.21.15.49:0/4045389094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c14e3a0 con 0x7f9d58005250 2023-07-18T03:41:19.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.908+0000 7f9d709ff700 1 -- 172.21.15.49:0/4045389094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d6c0c7970 con 0x7f9d58005250 2023-07-18T03:41:19.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.909+0000 7f9d62ffd700 1 -- 172.21.15.49:0/4045389094 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f9d5c013aa0 con 0x7f9d58005250 2023-07-18T03:41:19.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.910+0000 7f9d62ffd700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9d4c06ae00 0x7f9d4c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.910+0000 7f9d62ffd700 1 -- 172.21.15.49:0/4045389094 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9d5c08a120 con 0x7f9d58005250 2023-07-18T03:41:19.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.911+0000 7f9d62ffd700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] conn(0x7f9d4c071270 0x7f9d4c073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:19.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.911+0000 7f9d62ffd700 1 -- 172.21.15.49:0/4045389094 --> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d4c073d50 con 0x7f9d4c071270 2023-07-18T03:41:19.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.911+0000 7f9d62ffd700 1 -- 172.21.15.49:0/4045389094 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d5c08c700 con 0x7f9d58005250 2023-07-18T03:41:19.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.911+0000 7f9d63fff700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9d4c06ae00 0x7f9d4c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.911+0000 7f9d6ad9d700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] conn(0x7f9d4c071270 0x7f9d4c073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:19.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.911+0000 7f9d6ad9d700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] conn(0x7f9d4c071270 0x7f9d4c073680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:19.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.912+0000 7f9d63fff700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9d4c06ae00 0x7f9d4c06d2b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009be0 tx=0x7f9d54008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:19.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.912+0000 7f9d62ffd700 1 -- 172.21.15.49:0/4045389094 <== osd.2 v2:172.21.15.49:6824/46330117 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9d4c073d50 con 0x7f9d4c071270 2023-07-18T03:41:19.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.928+0000 7f9d709ff700 1 -- 172.21.15.49:0/4045389094 --> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9d6c0cd830 con 0x7f9d4c071270 2023-07-18T03:41:19.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.928+0000 7f9d62ffd700 1 -- 172.21.15.49:0/4045389094 <== osd.2 v2:172.21.15.49:6824/46330117 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9d64001290 con 0x7f9d4c071270 2023-07-18T03:41:19.922 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:41:19.923 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-18T03:41:19.923 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:41:19.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.928+0000 7f9d709ff700 1 -- 172.21.15.49:0/4045389094 >> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] conn(0x7f9d4c071270 msgr2=0x7f9d4c073680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.928+0000 7f9d709ff700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.49:6824/46330117,v1:172.21.15.49:6825/46330117] conn(0x7f9d4c071270 0x7f9d4c073680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:19.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.929+0000 7f9d709ff700 1 -- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9d4c06ae00 msgr2=0x7f9d4c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.929+0000 7f9d709ff700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9d4c06ae00 0x7f9d4c06d2b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009be0 tx=0x7f9d54008040 comp rx=0 tx=0).stop 2023-07-18T03:41:19.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.929+0000 7f9d709ff700 1 -- 172.21.15.49:0/4045389094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d58005250 msgr2=0x7f9d6c14de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:19.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.929+0000 7f9d709ff700 1 --2- 172.21.15.49:0/4045389094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d58005250 0x7f9d6c14de60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c000f80 tx=0x7f9d5c00a600 comp rx=0 tx=0).stop 2023-07-18T03:41:19.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.929+0000 7f9d6ad9d700 1 -- 172.21.15.49:0/4045389094 reap_dead start 2023-07-18T03:41:19.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.929+0000 7f9d709ff700 1 -- 172.21.15.49:0/4045389094 shutdown_connections 2023-07-18T03:41:19.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.929+0000 7f9d709ff700 1 -- 172.21.15.49:0/4045389094 >> 172.21.15.49:0/4045389094 conn(0x7f9d6c09c1b0 msgr2=0x7f9d6c0afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:19.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.929+0000 7f9d709ff700 1 -- 172.21.15.49:0/4045389094 shutdown_connections 2023-07-18T03:41:19.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:19.929+0000 7f9d709ff700 1 -- 172.21.15.49:0/4045389094 wait complete. 2023-07-18T03:41:19.940 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-07-18T03:41:20.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.153+0000 7f2d19fff700 1 Processor -- start 2023-07-18T03:41:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.154+0000 7f2d19fff700 1 -- start start 2023-07-18T03:41:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.155+0000 7f2d19fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2d140b4600 0x7f2d140cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.155+0000 7f2d19fff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d1415cd60 con 0x7f2d140cfe40 2023-07-18T03:41:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.155+0000 7f2d19fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d140ce360 con 0x7f2d140b6f00 2023-07-18T03:41:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.155+0000 7f2d19fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d140d1550 con 0x7f2d140b4600 2023-07-18T03:41:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.155+0000 7f2d137fe700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2d140b4600 0x7f2d140cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.155+0000 7f2d12ffd700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2d140cfe40 0x7f2d140cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44476/0 (socket says 172.21.15.49:44476) 2023-07-18T03:41:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.155+0000 7f2d12ffd700 1 -- 172.21.15.49:0/319924713 learned_addr learned my addr 172.21.15.49:0/319924713 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.156+0000 7f2d137fe700 1 -- 172.21.15.49:0/319924713 >> v1:172.21.15.107:6789/0 conn(0x7f2d140b6f00 legacy=0x7f2d140cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.156+0000 7f2d137fe700 1 -- 172.21.15.49:0/319924713 >> v1:172.21.15.49:6789/0 conn(0x7f2d140cfe40 legacy=0x7f2d140cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.156+0000 7f2d137fe700 1 -- 172.21.15.49:0/319924713 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d140cf340 con 0x7f2d140b4600 2023-07-18T03:41:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.156+0000 7f2d127fc700 1 -- 172.21.15.49:0/319924713 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636763825 0 0) 0x7f2d1415cd60 con 0x7f2d140cfe40 2023-07-18T03:41:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.156+0000 7f2d137fe700 1 --2- 172.21.15.49:0/319924713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d140b4600 0x7f2d140cf900 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009840 tx=0x7f2d04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f288cf121da6f0d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.156+0000 7f2d127fc700 1 -- 172.21.15.49:0/319924713 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04004230 con 0x7f2d140b4600 2023-07-18T03:41:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.156+0000 7f2d127fc700 1 -- 172.21.15.49:0/319924713 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d04004390 con 0x7f2d140b4600 2023-07-18T03:41:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.156+0000 7f2d127fc700 1 -- 172.21.15.49:0/319924713 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040036a0 con 0x7f2d140b4600 2023-07-18T03:41:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.157+0000 7f2d19fff700 1 -- 172.21.15.49:0/319924713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d140b4600 msgr2=0x7f2d140cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.157+0000 7f2d19fff700 1 --2- 172.21.15.49:0/319924713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d140b4600 0x7f2d140cf900 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009840 tx=0x7f2d04009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:20.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.157+0000 7f2d19fff700 1 -- 172.21.15.49:0/319924713 shutdown_connections 2023-07-18T03:41:20.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.157+0000 7f2d19fff700 1 --2- 172.21.15.49:0/319924713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d140b4600 0x7f2d140cf900 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.157+0000 7f2d19fff700 1 -- 172.21.15.49:0/319924713 >> 172.21.15.49:0/319924713 conn(0x7f2d1409c200 msgr2=0x7f2d140d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:20.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.157+0000 7f2d19fff700 1 -- 172.21.15.49:0/319924713 shutdown_connections 2023-07-18T03:41:20.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.157+0000 7f2d19fff700 1 -- 172.21.15.49:0/319924713 wait complete. 2023-07-18T03:41:20.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.159+0000 7f2d19fff700 1 Processor -- start 2023-07-18T03:41:20.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.159+0000 7f2d19fff700 1 -- start start 2023-07-18T03:41:20.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d19fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d140b6f00 0x7f2d140c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d19fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d140cfe40 0x7f2d14159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d19fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d14152430 0x7f2d14151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d19fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d14153ba0 con 0x7f2d140cfe40 2023-07-18T03:41:20.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d19fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d14152d30 con 0x7f2d140b6f00 2023-07-18T03:41:20.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d19fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d14152ff0 con 0x7f2d14152430 2023-07-18T03:41:20.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d12ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d140cfe40 0x7f2d14159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d12ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d140cfe40 0x7f2d14159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47680/0 (socket says 172.21.15.49:47680) 2023-07-18T03:41:20.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d12ffd700 1 -- 172.21.15.49:0/2718808841 learned_addr learned my addr 172.21.15.49:0/2718808841 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:20.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d137fe700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d140b6f00 0x7f2d140c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d13fff700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d14152430 0x7f2d14151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d12ffd700 1 -- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d14152430 msgr2=0x7f2d14151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d12ffd700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d14152430 0x7f2d14151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d12ffd700 1 -- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d140b6f00 msgr2=0x7f2d140c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d12ffd700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d140b6f00 0x7f2d140c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.160+0000 7f2d12ffd700 1 -- 172.21.15.49:0/2718808841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d040094a0 con 0x7f2d140cfe40 2023-07-18T03:41:20.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.161+0000 7f2d12ffd700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d140cfe40 0x7f2d14159140 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800c580 tx=0x7f2d0800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:20.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.161+0000 7f2cfbfff700 1 -- 172.21.15.49:0/2718808841 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08004c80 con 0x7f2d140cfe40 2023-07-18T03:41:20.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.161+0000 7f2cfbfff700 1 -- 172.21.15.49:0/2718808841 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d08004e00 con 0x7f2d140cfe40 2023-07-18T03:41:20.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.161+0000 7f2cfbfff700 1 -- 172.21.15.49:0/2718808841 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08007310 con 0x7f2d140cfe40 2023-07-18T03:41:20.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.161+0000 7f2d19fff700 1 -- 172.21.15.49:0/2718808841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d141566e0 con 0x7f2d140cfe40 2023-07-18T03:41:20.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.161+0000 7f2d19fff700 1 -- 172.21.15.49:0/2718808841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d141534e0 con 0x7f2d140cfe40 2023-07-18T03:41:20.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.162+0000 7f2d19fff700 1 -- 172.21.15.49:0/2718808841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d14153ce0 con 0x7f2d140cfe40 2023-07-18T03:41:20.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.163+0000 7f2cfbfff700 1 -- 172.21.15.49:0/2718808841 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f2d08007470 con 0x7f2d140cfe40 2023-07-18T03:41:20.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.164+0000 7f2cfbfff700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2d0006ae00 0x7f2d0006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.164+0000 7f2cfbfff700 1 -- 172.21.15.49:0/2718808841 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2d08088e50 con 0x7f2d140cfe40 2023-07-18T03:41:20.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.165+0000 7f2d137fe700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2d0006ae00 0x7f2d0006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.165+0000 7f2d137fe700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2d0006ae00 0x7f2d0006d2b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2d040090b0 tx=0x7f2d0401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:20.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.169+0000 7f2cfbfff700 1 -- 172.21.15.49:0/2718808841 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2d14153ce0 con 0x7f2d140cfe40 2023-07-18T03:41:20.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.378+0000 7f2d19fff700 1 -- 172.21.15.49:0/2718808841 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d140d1170 con 0x7f2d0006ae00 2023-07-18T03:41:20.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.379+0000 7f2cfbfff700 1 -- 172.21.15.49:0/2718808841 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2d04004130 con 0x7f2d0006ae00 2023-07-18T03:41:20.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.381+0000 7f2d19fff700 1 -- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2d0006ae00 msgr2=0x7f2d0006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.381+0000 7f2d19fff700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2d0006ae00 0x7f2d0006d2b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2d040090b0 tx=0x7f2d0401c040 comp rx=0 tx=0).stop 2023-07-18T03:41:20.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.381+0000 7f2d19fff700 1 -- 172.21.15.49:0/2718808841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d140cfe40 msgr2=0x7f2d14159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.381+0000 7f2d19fff700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d140cfe40 0x7f2d14159140 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800c580 tx=0x7f2d0800c890 comp rx=0 tx=0).stop 2023-07-18T03:41:20.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.382+0000 7f2d19fff700 1 -- 172.21.15.49:0/2718808841 shutdown_connections 2023-07-18T03:41:20.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.382+0000 7f2d19fff700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d14152430 0x7f2d14151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.382+0000 7f2d19fff700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d140b6f00 0x7f2d140c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.382+0000 7f2d19fff700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2d0006ae00 0x7f2d0006d2b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.382+0000 7f2d19fff700 1 --2- 172.21.15.49:0/2718808841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d140cfe40 0x7f2d14159140 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.382+0000 7f2d19fff700 1 -- 172.21.15.49:0/2718808841 >> 172.21.15.49:0/2718808841 conn(0x7f2d1409c200 msgr2=0x7f2d1406baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.382+0000 7f2d19fff700 1 -- 172.21.15.49:0/2718808841 shutdown_connections 2023-07-18T03:41:20.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.382+0000 7f2d19fff700 1 -- 172.21.15.49:0/2718808841 wait complete. 2023-07-18T03:41:20.378 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 2 to deep-scrub 2023-07-18T03:41:20.395 INFO:tasks.ceph:Scrubbing osd.3 2023-07-18T03:41:20.396 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-07-18T03:41:20.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.613+0000 7f16f11ff700 1 Processor -- start 2023-07-18T03:41:20.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.613+0000 7f16f11ff700 1 -- start start 2023-07-18T03:41:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16f11ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f16ec0b0480 0x7f16ec0b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16f11ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16ec070520 0x7f16ec0709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16f11ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ec15b530 con 0x7f16ec0ca1e0 2023-07-18T03:41:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16f11ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ec15aa10 con 0x7f16ec0b0480 2023-07-18T03:41:20.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16f11ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec15eef0 con 0x7f16ec070520 2023-07-18T03:41:20.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16e3fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f16ec0b0480 0x7f16ec0b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16e3fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f16ec0b0480 0x7f16ec0b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47688/0 (socket says 172.21.15.49:47688) 2023-07-18T03:41:20.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16e3fff700 1 -- 172.21.15.49:0/2768697640 learned_addr learned my addr 172.21.15.49:0/2768697640 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:20.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16eb59e700 1 --2- 172.21.15.49:0/2768697640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16ec070520 0x7f16ec0709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16e3fff700 1 -- 172.21.15.49:0/2768697640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16ec070520 msgr2=0x7f16ec0709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:20.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16e3fff700 1 --2- 172.21.15.49:0/2768697640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16ec070520 0x7f16ec0709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16e3fff700 1 -- 172.21.15.49:0/2768697640 >> v1:172.21.15.107:6789/0 conn(0x7f16ec0ca1e0 legacy=0x7f16ec0ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:20.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.615+0000 7f16e3fff700 1 -- 172.21.15.49:0/2768697640 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec15ac00 con 0x7f16ec0b0480 2023-07-18T03:41:20.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.616+0000 7f16e3fff700 1 --2- 172.21.15.49:0/2768697640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16ec0b0480 0x7f16ec0b08d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009920 tx=0x7f16d4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad40b94db38af7ce server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:20.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.616+0000 7f16ea59c700 1 -- 172.21.15.49:0/2768697640 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d400ba20 con 0x7f16ec0b0480 2023-07-18T03:41:20.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.616+0000 7f16ea59c700 1 -- 172.21.15.49:0/2768697640 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d400bb80 con 0x7f16ec0b0480 2023-07-18T03:41:20.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.616+0000 7f16ea59c700 1 -- 172.21.15.49:0/2768697640 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4009e70 con 0x7f16ec0b0480 2023-07-18T03:41:20.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.617+0000 7f16f11ff700 1 -- 172.21.15.49:0/2768697640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16ec0b0480 msgr2=0x7f16ec0b08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.617+0000 7f16f11ff700 1 --2- 172.21.15.49:0/2768697640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16ec0b0480 0x7f16ec0b08d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009920 tx=0x7f16d4009c30 comp rx=0 tx=0).stop 2023-07-18T03:41:20.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.617+0000 7f16f11ff700 1 -- 172.21.15.49:0/2768697640 shutdown_connections 2023-07-18T03:41:20.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.617+0000 7f16f11ff700 1 --2- 172.21.15.49:0/2768697640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16ec070520 0x7f16ec0709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.617+0000 7f16f11ff700 1 --2- 172.21.15.49:0/2768697640 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16ec0b0480 0x7f16ec0b08d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.617+0000 7f16f11ff700 1 -- 172.21.15.49:0/2768697640 >> 172.21.15.49:0/2768697640 conn(0x7f16ec0cea00 msgr2=0x7f16ec0cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:20.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.617+0000 7f16f11ff700 1 -- 172.21.15.49:0/2768697640 shutdown_connections 2023-07-18T03:41:20.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.617+0000 7f16f11ff700 1 -- 172.21.15.49:0/2768697640 wait complete. 2023-07-18T03:41:20.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.619+0000 7f16f11ff700 1 Processor -- start 2023-07-18T03:41:20.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.619+0000 7f16f11ff700 1 -- start start 2023-07-18T03:41:20.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16f11ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16ec070520 0x7f16ec0b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16f11ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16ec0b0480 0x7f16ec0b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16f11ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16ec0ca1e0 0x7f16ec0b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16f11ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec0c28e0 con 0x7f16ec070520 2023-07-18T03:41:20.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16f11ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec0b3cc0 con 0x7f16ec0b0480 2023-07-18T03:41:20.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16f11ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec140a40 con 0x7f16ec0ca1e0 2023-07-18T03:41:20.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16ead9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16ec070520 0x7f16ec0b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16ead9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16ec070520 0x7f16ec0b2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47698/0 (socket says 172.21.15.49:47698) 2023-07-18T03:41:20.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16ead9d700 1 -- 172.21.15.49:0/1716260744 learned_addr learned my addr 172.21.15.49:0/1716260744 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:20.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16e3fff700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16ec0b0480 0x7f16ec0b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16eb59e700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16ec0ca1e0 0x7f16ec0b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16ead9d700 1 -- 172.21.15.49:0/1716260744 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16ec0ca1e0 msgr2=0x7f16ec0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16ead9d700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16ec0ca1e0 0x7f16ec0b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16ead9d700 1 -- 172.21.15.49:0/1716260744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16ec0b0480 msgr2=0x7f16ec0b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16ead9d700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16ec0b0480 0x7f16ec0b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.620+0000 7f16ead9d700 1 -- 172.21.15.49:0/1716260744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d4009550 con 0x7f16ec070520 2023-07-18T03:41:20.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.621+0000 7f16ead9d700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16ec070520 0x7f16ec0b2d00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00c450 tx=0x7f16dc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:20.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.621+0000 7f16e37fe700 1 -- 172.21.15.49:0/1716260744 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0052d0 con 0x7f16ec070520 2023-07-18T03:41:20.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.621+0000 7f16e37fe700 1 -- 172.21.15.49:0/1716260744 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16dc005430 con 0x7f16ec070520 2023-07-18T03:41:20.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.621+0000 7f16e37fe700 1 -- 172.21.15.49:0/1716260744 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0038c0 con 0x7f16ec070520 2023-07-18T03:41:20.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.621+0000 7f16f11ff700 1 -- 172.21.15.49:0/1716260744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec0c2540 con 0x7f16ec070520 2023-07-18T03:41:20.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.621+0000 7f16f11ff700 1 -- 172.21.15.49:0/1716260744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec157800 con 0x7f16ec070520 2023-07-18T03:41:20.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.622+0000 7f16f11ff700 1 -- 172.21.15.49:0/1716260744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16ec15b1c0 con 0x7f16ec070520 2023-07-18T03:41:20.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.623+0000 7f16e37fe700 1 -- 172.21.15.49:0/1716260744 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f16dc003a20 con 0x7f16ec070520 2023-07-18T03:41:20.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.624+0000 7f16e37fe700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f16cc06ae00 0x7f16cc06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.624+0000 7f16e37fe700 1 -- 172.21.15.49:0/1716260744 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f16dc088720 con 0x7f16ec070520 2023-07-18T03:41:20.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.624+0000 7f16e37fe700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] conn(0x7f16cc071270 0x7f16cc073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.624+0000 7f16e37fe700 1 -- 172.21.15.49:0/1716260744 --> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16cc073d50 con 0x7f16cc071270 2023-07-18T03:41:20.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.624+0000 7f16e37fe700 1 -- 172.21.15.49:0/1716260744 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16dc08ad00 con 0x7f16ec070520 2023-07-18T03:41:20.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.624+0000 7f16e3fff700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f16cc06ae00 0x7f16cc06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.625+0000 7f16eb59e700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] conn(0x7f16cc071270 0x7f16cc073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.625+0000 7f16e3fff700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f16cc06ae00 0x7f16cc06d2b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f16d4000c00 tx=0x7f16d4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:20.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.625+0000 7f16eb59e700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] conn(0x7f16cc071270 0x7f16cc073680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:20.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.626+0000 7f16e37fe700 1 -- 172.21.15.49:0/1716260744 <== osd.3 v2:172.21.15.49:6808/3802584914 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f16cc073d50 con 0x7f16cc071270 2023-07-18T03:41:20.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.641+0000 7f16f11ff700 1 -- 172.21.15.49:0/1716260744 --> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f16ec15aee0 con 0x7f16cc071270 2023-07-18T03:41:20.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.641+0000 7f16e37fe700 1 -- 172.21.15.49:0/1716260744 <== osd.3 v2:172.21.15.49:6808/3802584914 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f16e4009b70 con 0x7f16cc071270 2023-07-18T03:41:20.637 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:41:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-18T03:41:20.638 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:41:20.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.642+0000 7f16f11ff700 1 -- 172.21.15.49:0/1716260744 >> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] conn(0x7f16cc071270 msgr2=0x7f16cc073680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.642+0000 7f16f11ff700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.49:6808/3802584914,v1:172.21.15.49:6809/3802584914] conn(0x7f16cc071270 0x7f16cc073680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.642+0000 7f16f11ff700 1 -- 172.21.15.49:0/1716260744 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f16cc06ae00 msgr2=0x7f16cc06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.642+0000 7f16f11ff700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f16cc06ae00 0x7f16cc06d2b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f16d4000c00 tx=0x7f16d4008030 comp rx=0 tx=0).stop 2023-07-18T03:41:20.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.642+0000 7f16f11ff700 1 -- 172.21.15.49:0/1716260744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16ec070520 msgr2=0x7f16ec0b2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.642+0000 7f16f11ff700 1 --2- 172.21.15.49:0/1716260744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16ec070520 0x7f16ec0b2d00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00c450 tx=0x7f16dc00c810 comp rx=0 tx=0).stop 2023-07-18T03:41:20.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.642+0000 7f16eb59e700 1 -- 172.21.15.49:0/1716260744 reap_dead start 2023-07-18T03:41:20.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.642+0000 7f16f11ff700 1 -- 172.21.15.49:0/1716260744 shutdown_connections 2023-07-18T03:41:20.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.642+0000 7f16f11ff700 1 -- 172.21.15.49:0/1716260744 >> 172.21.15.49:0/1716260744 conn(0x7f16ec0cea00 msgr2=0x7f16ec0cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:20.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.642+0000 7f16f11ff700 1 -- 172.21.15.49:0/1716260744 shutdown_connections 2023-07-18T03:41:20.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.642+0000 7f16f11ff700 1 -- 172.21.15.49:0/1716260744 wait complete. 2023-07-18T03:41:20.653 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-07-18T03:41:20.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.865+0000 7f8361fff700 1 Processor -- start 2023-07-18T03:41:20.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.865+0000 7f8361fff700 1 -- start start 2023-07-18T03:41:20.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.866+0000 7f8361fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f835c15d9d0 0x7f835c06b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.866+0000 7f8361fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f835c071170 0x7f835c0715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.866+0000 7f8361fff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835c071b30 con 0x7f835c06c9f0 2023-07-18T03:41:20.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.866+0000 7f8361fff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835c06c4e0 con 0x7f835c071170 2023-07-18T03:41:20.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.866+0000 7f8361fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835c0b2020 con 0x7f835c15d9d0 2023-07-18T03:41:20.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.866+0000 7f835bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f835c071170 0x7f835c0715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.866+0000 7f8353fff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f835c06c9f0 0x7f835c0b1910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44482/0 (socket says 172.21.15.49:44482) 2023-07-18T03:41:20.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.867+0000 7f835bfff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f835c071170 0x7f835c0715f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47714/0 (socket says 172.21.15.49:47714) 2023-07-18T03:41:20.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.867+0000 7f8353fff700 1 -- 172.21.15.49:0/124630940 learned_addr learned my addr 172.21.15.49:0/124630940 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:20.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.867+0000 7f835b7fe700 1 --2- 172.21.15.49:0/124630940 >> v2:172.21.15.107:3300/0 conn(0x7f835c15d9d0 0x7f835c06b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.867+0000 7f835bfff700 1 -- 172.21.15.49:0/124630940 >> v2:172.21.15.107:3300/0 conn(0x7f835c15d9d0 msgr2=0x7f835c06b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:20.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.867+0000 7f835bfff700 1 --2- 172.21.15.49:0/124630940 >> v2:172.21.15.107:3300/0 conn(0x7f835c15d9d0 0x7f835c06b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.867+0000 7f835bfff700 1 -- 172.21.15.49:0/124630940 >> v1:172.21.15.49:6789/0 conn(0x7f835c06c9f0 legacy=0x7f835c0b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.867+0000 7f835bfff700 1 -- 172.21.15.49:0/124630940 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c15dda0 con 0x7f835c071170 2023-07-18T03:41:20.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.867+0000 7f835b7fe700 1 --2- 172.21.15.49:0/124630940 >> v2:172.21.15.107:3300/0 conn(0x7f835c15d9d0 0x7f835c06b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:20.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.867+0000 7f835affd700 1 -- 172.21.15.49:0/124630940 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119205971 0 0) 0x7f835c071b30 con 0x7f835c06c9f0 2023-07-18T03:41:20.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.867+0000 7f835bfff700 1 --2- 172.21.15.49:0/124630940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c071170 0x7f835c0715f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f835c06dcc0 tx=0x7f835400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a46e5741fe9b041 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:20.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.867+0000 7f835affd700 1 -- 172.21.15.49:0/124630940 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835400e070 con 0x7f835c071170 2023-07-18T03:41:20.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.868+0000 7f835affd700 1 -- 172.21.15.49:0/124630940 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835400bbd0 con 0x7f835c071170 2023-07-18T03:41:20.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.868+0000 7f835affd700 1 -- 172.21.15.49:0/124630940 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354012430 con 0x7f835c071170 2023-07-18T03:41:20.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.868+0000 7f8361fff700 1 -- 172.21.15.49:0/124630940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c071170 msgr2=0x7f835c0715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.868+0000 7f8361fff700 1 --2- 172.21.15.49:0/124630940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c071170 0x7f835c0715f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f835c06dcc0 tx=0x7f835400b330 comp rx=0 tx=0).stop 2023-07-18T03:41:20.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.868+0000 7f8361fff700 1 -- 172.21.15.49:0/124630940 shutdown_connections 2023-07-18T03:41:20.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.868+0000 7f8361fff700 1 --2- 172.21.15.49:0/124630940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c071170 0x7f835c0715f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.868+0000 7f8361fff700 1 --2- 172.21.15.49:0/124630940 >> v2:172.21.15.107:3300/0 conn(0x7f835c15d9d0 0x7f835c06b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.868+0000 7f8361fff700 1 -- 172.21.15.49:0/124630940 >> 172.21.15.49:0/124630940 conn(0x7f835c0b05d0 msgr2=0x7f835c0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:20.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.868+0000 7f8361fff700 1 -- 172.21.15.49:0/124630940 shutdown_connections 2023-07-18T03:41:20.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.868+0000 7f8361fff700 1 -- 172.21.15.49:0/124630940 wait complete. 2023-07-18T03:41:20.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.870+0000 7f8361fff700 1 Processor -- start 2023-07-18T03:41:20.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.870+0000 7f8361fff700 1 -- start start 2023-07-18T03:41:20.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.871+0000 7f8361fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f835c06c9f0 0x7f835c1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.871+0000 7f8361fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c071170 0x7f835c0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.871+0000 7f8361fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835c15d9d0 0x7f835c0acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.871+0000 7f8361fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c1f5950 con 0x7f835c071170 2023-07-18T03:41:20.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.871+0000 7f8361fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c1f5bc0 con 0x7f835c06c9f0 2023-07-18T03:41:20.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.871+0000 7f8361fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835c1f5ea0 con 0x7f835c15d9d0 2023-07-18T03:41:20.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.871+0000 7f8353fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c071170 0x7f835c0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.871+0000 7f835b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f835c06c9f0 0x7f835c1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.871+0000 7f8353fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c071170 0x7f835c0ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47730/0 (socket says 172.21.15.49:47730) 2023-07-18T03:41:20.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.871+0000 7f8353fff700 1 -- 172.21.15.49:0/2138239470 learned_addr learned my addr 172.21.15.49:0/2138239470 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:20.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.871+0000 7f835bfff700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835c15d9d0 0x7f835c0acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.872+0000 7f8353fff700 1 -- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835c15d9d0 msgr2=0x7f835c0acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.872+0000 7f8353fff700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835c15d9d0 0x7f835c0acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.872+0000 7f8353fff700 1 -- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f835c06c9f0 msgr2=0x7f835c1521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:20.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.872+0000 7f8353fff700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f835c06c9f0 0x7f835c1521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:20.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.872+0000 7f8353fff700 1 -- 172.21.15.49:0/2138239470 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8354009a70 con 0x7f835c071170 2023-07-18T03:41:20.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.872+0000 7f8353fff700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c071170 0x7f835c0ac860 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f834400c540 tx=0x7f834400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:20.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.872+0000 7f8358ff9700 1 -- 172.21.15.49:0/2138239470 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344004c80 con 0x7f835c071170 2023-07-18T03:41:20.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.872+0000 7f8358ff9700 1 -- 172.21.15.49:0/2138239470 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8344004de0 con 0x7f835c071170 2023-07-18T03:41:20.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.872+0000 7f8361fff700 1 -- 172.21.15.49:0/2138239470 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835c1f6180 con 0x7f835c071170 2023-07-18T03:41:20.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.872+0000 7f8358ff9700 1 -- 172.21.15.49:0/2138239470 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344007310 con 0x7f835c071170 2023-07-18T03:41:20.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.872+0000 7f8361fff700 1 -- 172.21.15.49:0/2138239470 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835c1526f0 con 0x7f835c071170 2023-07-18T03:41:20.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.873+0000 7f8361fff700 1 -- 172.21.15.49:0/2138239470 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835c1f66a0 con 0x7f835c071170 2023-07-18T03:41:20.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.874+0000 7f8358ff9700 1 -- 172.21.15.49:0/2138239470 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f8344007470 con 0x7f835c071170 2023-07-18T03:41:20.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.875+0000 7f8358ff9700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f833c06ae00 0x7f833c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:20.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.875+0000 7f8358ff9700 1 -- 172.21.15.49:0/2138239470 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8344088fe0 con 0x7f835c071170 2023-07-18T03:41:20.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.876+0000 7f835b7fe700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f833c06ae00 0x7f833c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:20.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.876+0000 7f835b7fe700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f833c06ae00 0x7f833c06d2b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f834c00c940 tx=0x7f834c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:20.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:20.879+0000 7f8358ff9700 1 -- 172.21.15.49:0/2138239470 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f835c1f66a0 con 0x7f835c071170 2023-07-18T03:41:21.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.087+0000 7f8361fff700 1 -- 172.21.15.49:0/2138239470 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f835c06c200 con 0x7f833c06ae00 2023-07-18T03:41:21.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.088+0000 7f8358ff9700 1 -- 172.21.15.49:0/2138239470 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f834c00ce60 con 0x7f833c06ae00 2023-07-18T03:41:21.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.090+0000 7f8361fff700 1 -- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f833c06ae00 msgr2=0x7f833c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.091+0000 7f8361fff700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f833c06ae00 0x7f833c06d2b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f834c00c940 tx=0x7f834c008040 comp rx=0 tx=0).stop 2023-07-18T03:41:21.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.091+0000 7f8361fff700 1 -- 172.21.15.49:0/2138239470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c071170 msgr2=0x7f835c0ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.091+0000 7f8361fff700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c071170 0x7f835c0ac860 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f834400c540 tx=0x7f834400c850 comp rx=0 tx=0).stop 2023-07-18T03:41:21.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.091+0000 7f8361fff700 1 -- 172.21.15.49:0/2138239470 shutdown_connections 2023-07-18T03:41:21.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.091+0000 7f8361fff700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f835c15d9d0 0x7f835c0acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.091+0000 7f8361fff700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f835c06c9f0 0x7f835c1521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.091+0000 7f8361fff700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f833c06ae00 0x7f833c06d2b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.091+0000 7f8361fff700 1 --2- 172.21.15.49:0/2138239470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f835c071170 0x7f835c0ac860 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.091+0000 7f8361fff700 1 -- 172.21.15.49:0/2138239470 >> 172.21.15.49:0/2138239470 conn(0x7f835c0b05d0 msgr2=0x7f835c0cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:21.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.091+0000 7f8361fff700 1 -- 172.21.15.49:0/2138239470 shutdown_connections 2023-07-18T03:41:21.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.091+0000 7f8361fff700 1 -- 172.21.15.49:0/2138239470 wait complete. 2023-07-18T03:41:21.087 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 3 to deep-scrub 2023-07-18T03:41:21.103 INFO:tasks.ceph:Scrubbing osd.4 2023-07-18T03:41:21.103 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-07-18T03:41:21.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.317+0000 7f3067bff700 1 Processor -- start 2023-07-18T03:41:21.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.318+0000 7f3067bff700 1 -- start start 2023-07-18T03:41:21.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.319+0000 7f3067bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f306015c8f0 0x7f30600d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:21.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.319+0000 7f3067bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f30600b1740 0x7f30600d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:21.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.319+0000 7f3067bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30600af9b0 con 0x7f30600b1e10 2023-07-18T03:41:21.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.319+0000 7f3067bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30600b45b0 con 0x7f306015c8f0 2023-07-18T03:41:21.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.319+0000 7f3067bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30600afc50 con 0x7f30600b1740 2023-07-18T03:41:21.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.319+0000 7f306599b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f306015c8f0 0x7f30600d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:21.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.319+0000 7f306519a700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f30600b1e10 0x7f30600b2740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44492/0 (socket says 172.21.15.49:44492) 2023-07-18T03:41:21.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.319+0000 7f306599b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f306015c8f0 0x7f30600d09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47746/0 (socket says 172.21.15.49:47746) 2023-07-18T03:41:21.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.319+0000 7f306519a700 1 -- 172.21.15.49:0/1305860699 learned_addr learned my addr 172.21.15.49:0/1305860699 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:21.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.320+0000 7f306619c700 1 --2- 172.21.15.49:0/1305860699 >> v2:172.21.15.107:3300/0 conn(0x7f30600b1740 0x7f30600d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:21.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.320+0000 7f306599b700 1 -- 172.21.15.49:0/1305860699 >> v2:172.21.15.107:3300/0 conn(0x7f30600b1740 msgr2=0x7f30600d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:21.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.320+0000 7f306599b700 1 --2- 172.21.15.49:0/1305860699 >> v2:172.21.15.107:3300/0 conn(0x7f30600b1740 0x7f30600d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.320+0000 7f306599b700 1 -- 172.21.15.49:0/1305860699 >> v1:172.21.15.49:6789/0 conn(0x7f30600b1e10 legacy=0x7f30600b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.320+0000 7f306599b700 1 -- 172.21.15.49:0/1305860699 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30600aff30 con 0x7f306015c8f0 2023-07-18T03:41:21.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.320+0000 7f306619c700 1 --2- 172.21.15.49:0/1305860699 >> v2:172.21.15.107:3300/0 conn(0x7f30600b1740 0x7f30600d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:21.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.320+0000 7f3064999700 1 -- 172.21.15.49:0/1305860699 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905166105 0 0) 0x7f30600af9b0 con 0x7f30600b1e10 2023-07-18T03:41:21.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.320+0000 7f306599b700 1 --2- 172.21.15.49:0/1305860699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306015c8f0 0x7f30600d09b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f305c009840 tx=0x7f305c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6591c73cf55645c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.320+0000 7f3064999700 1 -- 172.21.15.49:0/1305860699 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c011830 con 0x7f306015c8f0 2023-07-18T03:41:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.320+0000 7f3064999700 1 -- 172.21.15.49:0/1305860699 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c011990 con 0x7f306015c8f0 2023-07-18T03:41:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.320+0000 7f3064999700 1 -- 172.21.15.49:0/1305860699 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c011d20 con 0x7f306015c8f0 2023-07-18T03:41:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.321+0000 7f3067bff700 1 -- 172.21.15.49:0/1305860699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306015c8f0 msgr2=0x7f30600d09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.321+0000 7f3067bff700 1 --2- 172.21.15.49:0/1305860699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306015c8f0 0x7f30600d09b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f305c009840 tx=0x7f305c009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.321+0000 7f3067bff700 1 -- 172.21.15.49:0/1305860699 shutdown_connections 2023-07-18T03:41:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.321+0000 7f3067bff700 1 --2- 172.21.15.49:0/1305860699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306015c8f0 0x7f30600d09b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.321+0000 7f3067bff700 1 --2- 172.21.15.49:0/1305860699 >> v2:172.21.15.107:3300/0 conn(0x7f30600b1740 0x7f30600d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.321+0000 7f3067bff700 1 -- 172.21.15.49:0/1305860699 >> 172.21.15.49:0/1305860699 conn(0x7f306009c150 msgr2=0x7f30600d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:21.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.321+0000 7f3067bff700 1 -- 172.21.15.49:0/1305860699 shutdown_connections 2023-07-18T03:41:21.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.321+0000 7f3067bff700 1 -- 172.21.15.49:0/1305860699 wait complete. 2023-07-18T03:41:21.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.323+0000 7f3067bff700 1 Processor -- start 2023-07-18T03:41:21.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.323+0000 7f3067bff700 1 -- start start 2023-07-18T03:41:21.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.324+0000 7f3067bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30600b1740 0x7f30600c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:21.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.324+0000 7f3067bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30600b1e10 0x7f30600b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:21.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.324+0000 7f3067bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306015c8f0 0x7f306006b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:21.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.324+0000 7f3067bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30600b7ff0 con 0x7f306015c8f0 2023-07-18T03:41:21.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.324+0000 7f3067bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306009e2d0 con 0x7f30600b1e10 2023-07-18T03:41:21.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.324+0000 7f3067bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30600c9140 con 0x7f30600b1740 2023-07-18T03:41:21.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.324+0000 7f306619c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306015c8f0 0x7f306006b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:21.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.324+0000 7f306619c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306015c8f0 0x7f306006b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47762/0 (socket says 172.21.15.49:47762) 2023-07-18T03:41:21.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.324+0000 7f306619c700 1 -- 172.21.15.49:0/2024884534 learned_addr learned my addr 172.21.15.49:0/2024884534 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:21.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.325+0000 7f306599b700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30600b1740 0x7f30600c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:21.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.324+0000 7f306519a700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30600b1e10 0x7f30600b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:21.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.325+0000 7f306619c700 1 -- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30600b1740 msgr2=0x7f30600c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.325+0000 7f306619c700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30600b1740 0x7f30600c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.325+0000 7f306619c700 1 -- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30600b1e10 msgr2=0x7f30600b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.325+0000 7f306619c700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30600b1e10 0x7f30600b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.325+0000 7f306619c700 1 -- 172.21.15.49:0/2024884534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c0094a0 con 0x7f306015c8f0 2023-07-18T03:41:21.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.325+0000 7f306619c700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306015c8f0 0x7f306006b700 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f305800cef0 tx=0x7f305800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:21.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.325+0000 7f30567fc700 1 -- 172.21.15.49:0/2024884534 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305800ad00 con 0x7f306015c8f0 2023-07-18T03:41:21.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.325+0000 7f30567fc700 1 -- 172.21.15.49:0/2024884534 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305800ae60 con 0x7f306015c8f0 2023-07-18T03:41:21.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.325+0000 7f30567fc700 1 -- 172.21.15.49:0/2024884534 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058015680 con 0x7f306015c8f0 2023-07-18T03:41:21.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.325+0000 7f3067bff700 1 -- 172.21.15.49:0/2024884534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30600b7470 con 0x7f306015c8f0 2023-07-18T03:41:21.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.326+0000 7f3067bff700 1 -- 172.21.15.49:0/2024884534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30600b7a30 con 0x7f306015c8f0 2023-07-18T03:41:21.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.326+0000 7f3067bff700 1 -- 172.21.15.49:0/2024884534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f30600b0210 con 0x7f306015c8f0 2023-07-18T03:41:21.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.327+0000 7f30567fc700 1 -- 172.21.15.49:0/2024884534 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f305800d430 con 0x7f306015c8f0 2023-07-18T03:41:21.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.328+0000 7f30567fc700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f304406ae00 0x7f304406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:21.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.328+0000 7f30567fc700 1 -- 172.21.15.49:0/2024884534 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f30580897b0 con 0x7f306015c8f0 2023-07-18T03:41:21.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.329+0000 7f30567fc700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] conn(0x7f3044071270 0x7f3044073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:21.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.329+0000 7f30567fc700 1 -- 172.21.15.49:0/2024884534 --> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3044073d50 con 0x7f3044071270 2023-07-18T03:41:21.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.329+0000 7f30567fc700 1 -- 172.21.15.49:0/2024884534 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f305808be50 con 0x7f306015c8f0 2023-07-18T03:41:21.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.329+0000 7f306599b700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f304406ae00 0x7f304406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:21.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.329+0000 7f306519a700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] conn(0x7f3044071270 0x7f3044073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:21.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.330+0000 7f306599b700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f304406ae00 0x7f304406d2b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f305c0090b0 tx=0x7f305c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:21.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.330+0000 7f306519a700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] conn(0x7f3044071270 0x7f3044073680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:21.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.331+0000 7f30567fc700 1 -- 172.21.15.49:0/2024884534 <== osd.4 v2:172.21.15.107:6800/4156054429 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3044073d50 con 0x7f3044071270 2023-07-18T03:41:21.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.346+0000 7f3067bff700 1 -- 172.21.15.49:0/2024884534 --> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f306009e4d0 con 0x7f3044071270 2023-07-18T03:41:21.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.346+0000 7f30567fc700 1 -- 172.21.15.49:0/2024884534 <== osd.4 v2:172.21.15.107:6800/4156054429 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f306009e4d0 con 0x7f3044071270 2023-07-18T03:41:21.341 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:41:21.341 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-18T03:41:21.341 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:41:21.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.347+0000 7f3067bff700 1 -- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] conn(0x7f3044071270 msgr2=0x7f3044073680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.347+0000 7f3067bff700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:6800/4156054429,v1:172.21.15.107:6801/4156054429] conn(0x7f3044071270 0x7f3044073680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.347+0000 7f3067bff700 1 -- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f304406ae00 msgr2=0x7f304406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.347+0000 7f3067bff700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f304406ae00 0x7f304406d2b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f305c0090b0 tx=0x7f305c01c040 comp rx=0 tx=0).stop 2023-07-18T03:41:21.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.347+0000 7f3067bff700 1 -- 172.21.15.49:0/2024884534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306015c8f0 msgr2=0x7f306006b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.347+0000 7f3067bff700 1 --2- 172.21.15.49:0/2024884534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306015c8f0 0x7f306006b700 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f305800cef0 tx=0x7f305800cf20 comp rx=0 tx=0).stop 2023-07-18T03:41:21.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.347+0000 7f306619c700 1 -- 172.21.15.49:0/2024884534 reap_dead start 2023-07-18T03:41:21.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.347+0000 7f3067bff700 1 -- 172.21.15.49:0/2024884534 shutdown_connections 2023-07-18T03:41:21.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.347+0000 7f3067bff700 1 -- 172.21.15.49:0/2024884534 >> 172.21.15.49:0/2024884534 conn(0x7f306009c150 msgr2=0x7f30600b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:21.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.347+0000 7f3067bff700 1 -- 172.21.15.49:0/2024884534 shutdown_connections 2023-07-18T03:41:21.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.347+0000 7f3067bff700 1 -- 172.21.15.49:0/2024884534 wait complete. 2023-07-18T03:41:21.358 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-07-18T03:41:21.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.569+0000 7fc2f2bff700 1 Processor -- start 2023-07-18T03:41:21.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.569+0000 7fc2f2bff700 1 -- start start 2023-07-18T03:41:21.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.570+0000 7fc2f2bff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2ec0b8a60 0x7fc2ec0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:21.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.570+0000 7fc2f2bff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ec0b8e30 con 0x7fc2ec0b4530 2023-07-18T03:41:21.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.570+0000 7fc2f2bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ec0afaa0 con 0x7fc2ec0d1220 2023-07-18T03:41:21.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.570+0000 7fc2f2bff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2ec0afd80 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.571+0000 7fc2f099b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2ec0b8a60 0x7fc2ec0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:21.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.571+0000 7fc2ebfff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc2ec0b4530 0x7fc2ec0d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44500/0 (socket says 172.21.15.49:44500) 2023-07-18T03:41:21.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.571+0000 7fc2ebfff700 1 -- 172.21.15.49:0/1200274702 learned_addr learned my addr 172.21.15.49:0/1200274702 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:21.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.571+0000 7fc2f099b700 1 -- 172.21.15.49:0/1200274702 >> v1:172.21.15.107:6789/0 conn(0x7fc2ec0d1220 legacy=0x7fc2ec0b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.571+0000 7fc2f099b700 1 -- 172.21.15.49:0/1200274702 >> v1:172.21.15.49:6789/0 conn(0x7fc2ec0b4530 legacy=0x7fc2ec0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.571+0000 7fc2eb7fe700 1 -- 172.21.15.49:0/1200274702 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304440163 0 0) 0x7fc2ec0b8e30 con 0x7fc2ec0b4530 2023-07-18T03:41:21.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.571+0000 7fc2f099b700 1 -- 172.21.15.49:0/1200274702 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0b0060 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.571+0000 7fc2f099b700 1 --2- 172.21.15.49:0/1200274702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ec0b8a60 0x7fc2ec0b3ff0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009840 tx=0x7fc2e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ef02357d385803 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:21.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.572+0000 7fc2eb7fe700 1 -- 172.21.15.49:0/1200274702 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0011830 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.572+0000 7fc2eb7fe700 1 -- 172.21.15.49:0/1200274702 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e0011990 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.572+0000 7fc2eb7fe700 1 -- 172.21.15.49:0/1200274702 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0011d20 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.572+0000 7fc2f2bff700 1 -- 172.21.15.49:0/1200274702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ec0b8a60 msgr2=0x7fc2ec0b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.572+0000 7fc2f2bff700 1 --2- 172.21.15.49:0/1200274702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ec0b8a60 0x7fc2ec0b3ff0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009840 tx=0x7fc2e0009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:21.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.572+0000 7fc2f2bff700 1 -- 172.21.15.49:0/1200274702 shutdown_connections 2023-07-18T03:41:21.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.572+0000 7fc2f2bff700 1 --2- 172.21.15.49:0/1200274702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ec0b8a60 0x7fc2ec0b3ff0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.572+0000 7fc2f2bff700 1 -- 172.21.15.49:0/1200274702 >> 172.21.15.49:0/1200274702 conn(0x7fc2ec09c1b0 msgr2=0x7fc2ec0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:21.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.572+0000 7fc2f2bff700 1 -- 172.21.15.49:0/1200274702 shutdown_connections 2023-07-18T03:41:21.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.572+0000 7fc2f2bff700 1 -- 172.21.15.49:0/1200274702 wait complete. 2023-07-18T03:41:21.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.574+0000 7fc2f2bff700 1 Processor -- start 2023-07-18T03:41:21.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.574+0000 7fc2f2bff700 1 -- start start 2023-07-18T03:41:21.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.575+0000 7fc2f2bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2ec0b4530 0x7fc2ec1ef770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:21.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.575+0000 7fc2f2bff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ec0b8a60 0x7fc2ec0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:21.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.575+0000 7fc2f2bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2ec0d1220 0x7fc2ec0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:21.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.575+0000 7fc2f2bff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0ce2e0 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.575+0000 7fc2f2bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec15de00 con 0x7fc2ec0b4530 2023-07-18T03:41:21.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.575+0000 7fc2f2bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0ca980 con 0x7fc2ec0d1220 2023-07-18T03:41:21.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.576+0000 7fc2ebfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ec0b8a60 0x7fc2ec0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:21.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.576+0000 7fc2ebfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ec0b8a60 0x7fc2ec0ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47770/0 (socket says 172.21.15.49:47770) 2023-07-18T03:41:21.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.576+0000 7fc2ebfff700 1 -- 172.21.15.49:0/749005760 learned_addr learned my addr 172.21.15.49:0/749005760 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:21.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.576+0000 7fc2f119c700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2ec0d1220 0x7fc2ec0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:21.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.576+0000 7fc2f099b700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2ec0b4530 0x7fc2ec1ef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:21.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.576+0000 7fc2ebfff700 1 -- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2ec0d1220 msgr2=0x7fc2ec0cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.576+0000 7fc2ebfff700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2ec0d1220 0x7fc2ec0cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.576+0000 7fc2ebfff700 1 -- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2ec0b4530 msgr2=0x7fc2ec1ef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.576+0000 7fc2ebfff700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2ec0b4530 0x7fc2ec1ef770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.576+0000 7fc2ebfff700 1 -- 172.21.15.49:0/749005760 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e00094a0 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.577+0000 7fc2ebfff700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ec0b8a60 0x7fc2ec0ca010 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800c520 tx=0x7fc2d800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:21.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.577+0000 7fc2e97fa700 1 -- 172.21.15.49:0/749005760 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8004c80 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.577+0000 7fc2e97fa700 1 -- 172.21.15.49:0/749005760 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d8004e00 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.577+0000 7fc2f2bff700 1 -- 172.21.15.49:0/749005760 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ec0cac60 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.577+0000 7fc2e97fa700 1 -- 172.21.15.49:0/749005760 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8007310 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.577+0000 7fc2f2bff700 1 -- 172.21.15.49:0/749005760 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ec15e0a0 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.578+0000 7fc2f2bff700 1 -- 172.21.15.49:0/749005760 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ec15e2c0 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.579+0000 7fc2e97fa700 1 -- 172.21.15.49:0/749005760 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7fc2d8007470 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.581+0000 7fc2e97fa700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc2d006ae00 0x7fc2d006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:21.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.581+0000 7fc2e97fa700 1 -- 172.21.15.49:0/749005760 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc2d8088fe0 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.581+0000 7fc2f099b700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc2d006ae00 0x7fc2d006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:21.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.582+0000 7fc2f099b700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc2d006ae00 0x7fc2d006d2b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00090b0 tx=0x7fc2e0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:21.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.585+0000 7fc2e97fa700 1 -- 172.21.15.49:0/749005760 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc2d8054200 con 0x7fc2ec0b8a60 2023-07-18T03:41:21.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.792+0000 7fc2f2bff700 1 -- 172.21.15.49:0/749005760 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2ec0cb180 con 0x7fc2d006ae00 2023-07-18T03:41:21.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.793+0000 7fc2e97fa700 1 -- 172.21.15.49:0/749005760 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc2e0011750 con 0x7fc2d006ae00 2023-07-18T03:41:21.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.795+0000 7fc2f2bff700 1 -- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc2d006ae00 msgr2=0x7fc2d006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.795+0000 7fc2f2bff700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc2d006ae00 0x7fc2d006d2b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00090b0 tx=0x7fc2e0008030 comp rx=0 tx=0).stop 2023-07-18T03:41:21.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.795+0000 7fc2f2bff700 1 -- 172.21.15.49:0/749005760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ec0b8a60 msgr2=0x7fc2ec0ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:21.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.795+0000 7fc2f2bff700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ec0b8a60 0x7fc2ec0ca010 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800c520 tx=0x7fc2d800c830 comp rx=0 tx=0).stop 2023-07-18T03:41:21.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.795+0000 7fc2f2bff700 1 -- 172.21.15.49:0/749005760 shutdown_connections 2023-07-18T03:41:21.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.795+0000 7fc2f2bff700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2ec0d1220 0x7fc2ec0cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.795+0000 7fc2f2bff700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2ec0b4530 0x7fc2ec1ef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.795+0000 7fc2f2bff700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7fc2d006ae00 0x7fc2d006d2b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.795+0000 7fc2f2bff700 1 --2- 172.21.15.49:0/749005760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ec0b8a60 0x7fc2ec0ca010 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:21.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.796+0000 7fc2f2bff700 1 -- 172.21.15.49:0/749005760 >> 172.21.15.49:0/749005760 conn(0x7fc2ec09c1b0 msgr2=0x7fc2ec06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:21.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.796+0000 7fc2f2bff700 1 -- 172.21.15.49:0/749005760 shutdown_connections 2023-07-18T03:41:21.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:21.796+0000 7fc2f2bff700 1 -- 172.21.15.49:0/749005760 wait complete. 2023-07-18T03:41:21.792 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 4 to deep-scrub 2023-07-18T03:41:21.808 INFO:tasks.ceph:Scrubbing osd.5 2023-07-18T03:41:21.809 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-07-18T03:41:22.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.025+0000 7f4bac9ff700 1 Processor -- start 2023-07-18T03:41:22.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.025+0000 7f4bac9ff700 1 -- start start 2023-07-18T03:41:22.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.026+0000 7f4bac9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba806c9d0 0x7f4ba8163f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.026+0000 7f4bac9ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba806fdb0 con 0x7f4ba80c9860 2023-07-18T03:41:22.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.026+0000 7f4bac9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba80c9cd0 con 0x7f4ba80add00 2023-07-18T03:41:22.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.026+0000 7f4bac9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8159880 con 0x7f4ba806c9d0 2023-07-18T03:41:22.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.026+0000 7f4b9ffff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4ba80c9860 0x7f4ba806c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44510/0 (socket says 172.21.15.49:44510) 2023-07-18T03:41:22.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.026+0000 7f4b9ffff700 1 -- 172.21.15.49:0/2259489732 learned_addr learned my addr 172.21.15.49:0/2259489732 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:22.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba6d9d700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba806c9d0 0x7f4ba8163f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342780737 0 0) 0x7f4ba806fdb0 con 0x7f4ba80c9860 2023-07-18T03:41:22.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b940057c0 con 0x7f4ba80c9860 2023-07-18T03:41:22.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717850801 0 0) 0x7f4ba80c9cd0 con 0x7f4ba80add00 2023-07-18T03:41:22.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ba806fdb0 con 0x7f4ba80add00 2023-07-18T03:41:22.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4176629902 0 0) 0x7f4b940057c0 con 0x7f4ba80c9860 2023-07-18T03:41:22.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba806c9d0 msgr2=0x7f4ba8163f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:22.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba806c9d0 0x7f4ba8163f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 >> v1:172.21.15.107:6789/0 conn(0x7f4ba80add00 legacy=0x7f4ba80cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba80ae0d0 con 0x7f4ba80c9860 2023-07-18T03:41:22.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7f4b90001670 con 0x7f4ba80c9860 2023-07-18T03:41:22.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba6d9d700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba806c9d0 0x7f4ba8163f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:22.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 >> v1:172.21.15.49:6789/0 conn(0x7f4ba80c9860 legacy=0x7f4ba806c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba806c9d0 0x7f4b94005e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b940063d0 0x7f4b94006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b94006fa0 0x7f4b94007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8159880 con 0x7f4b94006fa0 2023-07-18T03:41:22.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9400dd60 con 0x7f4b940063d0 2023-07-18T03:41:22.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.027+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9400df20 con 0x7f4ba806c9d0 2023-07-18T03:41:22.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.028+0000 7f4ba6d9d700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b94006fa0 0x7f4b94007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.028+0000 7f4ba659c700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba806c9d0 0x7f4b94005e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.028+0000 7f4b9ffff700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b940063d0 0x7f4b94006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.028+0000 7f4ba6d9d700 1 -- 172.21.15.49:0/2259489732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba806c9d0 msgr2=0x7f4b94005e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:22.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.028+0000 7f4ba6d9d700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba806c9d0 0x7f4b94005e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.028+0000 7f4ba6d9d700 1 -- 172.21.15.49:0/2259489732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b940063d0 msgr2=0x7f4b94006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:22.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.028+0000 7f4ba6d9d700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b940063d0 0x7f4b94006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.028+0000 7f4ba6d9d700 1 -- 172.21.15.49:0/2259489732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba0009a70 con 0x7f4b94006fa0 2023-07-18T03:41:22.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.028+0000 7f4ba6d9d700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b94006fa0 0x7f4b94007410 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4ba806cdc0 tx=0x7f4b94007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e817786d1a2efa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.028+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000edb0 con 0x7f4b94006fa0 2023-07-18T03:41:22.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.029+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba000f040 con 0x7f4b94006fa0 2023-07-18T03:41:22.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.029+0000 7f4ba5d9b700 1 -- 172.21.15.49:0/2259489732 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0003c40 con 0x7f4b94006fa0 2023-07-18T03:41:22.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.029+0000 7f4bac9ff700 1 -- 172.21.15.49:0/2259489732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b94006fa0 msgr2=0x7f4b94007410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.029+0000 7f4bac9ff700 1 --2- 172.21.15.49:0/2259489732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b94006fa0 0x7f4b94007410 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4ba806cdc0 tx=0x7f4b94007a90 comp rx=0 tx=0).stop 2023-07-18T03:41:22.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.029+0000 7f4ba6d9d700 1 -- 172.21.15.49:0/2259489732 reap_dead start 2023-07-18T03:41:22.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.029+0000 7f4bac9ff700 1 -- 172.21.15.49:0/2259489732 shutdown_connections 2023-07-18T03:41:22.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.029+0000 7f4bac9ff700 1 -- 172.21.15.49:0/2259489732 >> 172.21.15.49:0/2259489732 conn(0x7f4ba80cea00 msgr2=0x7f4ba80cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:22.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.029+0000 7f4bac9ff700 1 -- 172.21.15.49:0/2259489732 shutdown_connections 2023-07-18T03:41:22.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.029+0000 7f4bac9ff700 1 -- 172.21.15.49:0/2259489732 wait complete. 2023-07-18T03:41:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.032+0000 7f4bac9ff700 1 Processor -- start 2023-07-18T03:41:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.032+0000 7f4bac9ff700 1 -- start start 2023-07-18T03:41:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.033+0000 7f4bac9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b94006fa0 0x7f4ba80af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.033+0000 7f4bac9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ba81f1ad0 0x7f4ba80c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.033+0000 7f4bac9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba80c6670 0x7f4ba8151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.033+0000 7f4bac9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80c6ae0 con 0x7f4ba81f1ad0 2023-07-18T03:41:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.033+0000 7f4bac9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8152860 con 0x7f4ba80c6670 2023-07-18T03:41:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.033+0000 7f4bac9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80b2a30 con 0x7f4b94006fa0 2023-07-18T03:41:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.033+0000 7f4b9ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ba81f1ad0 0x7f4ba80c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.033+0000 7f4b9ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ba81f1ad0 0x7f4ba80c6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47780/0 (socket says 172.21.15.49:47780) 2023-07-18T03:41:22.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4b9ffff700 1 -- 172.21.15.49:0/3602217664 learned_addr learned my addr 172.21.15.49:0/3602217664 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:22.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4ba659c700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b94006fa0 0x7f4ba80af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4ba6d9d700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba80c6670 0x7f4ba8151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4b9ffff700 1 -- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b94006fa0 msgr2=0x7f4ba80af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4b9ffff700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b94006fa0 0x7f4ba80af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4b9ffff700 1 -- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba80c6670 msgr2=0x7f4ba8151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4b9ffff700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba80c6670 0x7f4ba8151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4b9ffff700 1 -- 172.21.15.49:0/3602217664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba80b2d10 con 0x7f4ba81f1ad0 2023-07-18T03:41:22.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4ba6d9d700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba80c6670 0x7f4ba8151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:22.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4b9ffff700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ba81f1ad0 0x7f4ba80c6130 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4b90007340 tx=0x7f4b90007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4b9effd700 1 -- 172.21.15.49:0/3602217664 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90007780 con 0x7f4ba81f1ad0 2023-07-18T03:41:22.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4b9effd700 1 -- 172.21.15.49:0/3602217664 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9000f870 con 0x7f4ba81f1ad0 2023-07-18T03:41:22.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.034+0000 7f4b9effd700 1 -- 172.21.15.49:0/3602217664 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000fc20 con 0x7f4ba81f1ad0 2023-07-18T03:41:22.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.035+0000 7f4bac9ff700 1 -- 172.21.15.49:0/3602217664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba80b2ff0 con 0x7f4ba81f1ad0 2023-07-18T03:41:22.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.035+0000 7f4bac9ff700 1 -- 172.21.15.49:0/3602217664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba8151ff0 con 0x7f4ba81f1ad0 2023-07-18T03:41:22.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.035+0000 7f4bac9ff700 1 -- 172.21.15.49:0/3602217664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4ba806f760 con 0x7f4ba81f1ad0 2023-07-18T03:41:22.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.037+0000 7f4b9effd700 1 -- 172.21.15.49:0/3602217664 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f4b9000fd80 con 0x7f4ba81f1ad0 2023-07-18T03:41:22.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.038+0000 7f4b9effd700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4b8806ae00 0x7f4b8806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.038+0000 7f4b9effd700 1 -- 172.21.15.49:0/3602217664 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4b900892a0 con 0x7f4ba81f1ad0 2023-07-18T03:41:22.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.038+0000 7f4b9effd700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] conn(0x7f4b88071270 0x7f4b88073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.038+0000 7f4b9effd700 1 -- 172.21.15.49:0/3602217664 --> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b88073d50 con 0x7f4b88071270 2023-07-18T03:41:22.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.038+0000 7f4b9effd700 1 -- 172.21.15.49:0/3602217664 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b9008b940 con 0x7f4ba81f1ad0 2023-07-18T03:41:22.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.038+0000 7f4ba659c700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4b8806ae00 0x7f4b8806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.038+0000 7f4ba6d9d700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] conn(0x7f4b88071270 0x7f4b88073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.039+0000 7f4ba6d9d700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] conn(0x7f4b88071270 0x7f4b88073680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.039+0000 7f4ba659c700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4b8806ae00 0x7f4b8806d2b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4b98007fd0 tx=0x7f4b98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.040+0000 7f4b9effd700 1 -- 172.21.15.49:0/3602217664 <== osd.5 v2:172.21.15.107:6818/1123907778 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4b88073d50 con 0x7f4b88071270 2023-07-18T03:41:22.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.055+0000 7f4bac9ff700 1 -- 172.21.15.49:0/3602217664 --> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4ba80cd9a0 con 0x7f4b88071270 2023-07-18T03:41:22.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.057+0000 7f4b9effd700 1 -- 172.21.15.49:0/3602217664 <== osd.5 v2:172.21.15.107:6818/1123907778 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4ba000ed70 con 0x7f4b88071270 2023-07-18T03:41:22.050 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:41:22.050 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-18T03:41:22.051 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:41:22.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.057+0000 7f4bac9ff700 1 -- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] conn(0x7f4b88071270 msgr2=0x7f4b88073680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.057+0000 7f4bac9ff700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:6818/1123907778,v1:172.21.15.107:6820/1123907778] conn(0x7f4b88071270 0x7f4b88073680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.057+0000 7f4bac9ff700 1 -- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4b8806ae00 msgr2=0x7f4b8806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.057+0000 7f4bac9ff700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f4b8806ae00 0x7f4b8806d2b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4b98007fd0 tx=0x7f4b98009040 comp rx=0 tx=0).stop 2023-07-18T03:41:22.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.057+0000 7f4bac9ff700 1 -- 172.21.15.49:0/3602217664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ba81f1ad0 msgr2=0x7f4ba80c6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.057+0000 7f4bac9ff700 1 --2- 172.21.15.49:0/3602217664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ba81f1ad0 0x7f4ba80c6130 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4b90007340 tx=0x7f4b90007650 comp rx=0 tx=0).stop 2023-07-18T03:41:22.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.057+0000 7f4ba6d9d700 1 -- 172.21.15.49:0/3602217664 reap_dead start 2023-07-18T03:41:22.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.057+0000 7f4bac9ff700 1 -- 172.21.15.49:0/3602217664 shutdown_connections 2023-07-18T03:41:22.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.057+0000 7f4bac9ff700 1 -- 172.21.15.49:0/3602217664 >> 172.21.15.49:0/3602217664 conn(0x7f4ba80cea00 msgr2=0x7f4ba8070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:22.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.057+0000 7f4bac9ff700 1 -- 172.21.15.49:0/3602217664 shutdown_connections 2023-07-18T03:41:22.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.057+0000 7f4bac9ff700 1 -- 172.21.15.49:0/3602217664 wait complete. 2023-07-18T03:41:22.068 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-07-18T03:41:22.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.279+0000 7f07971ff700 1 Processor -- start 2023-07-18T03:41:22.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.279+0000 7f07971ff700 1 -- start start 2023-07-18T03:41:22.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.280+0000 7f07971ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f07900d0b10 0x7f07900d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.280+0000 7f07971ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07900b8e30 con 0x7f07900b8a60 2023-07-18T03:41:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.280+0000 7f07971ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07900afaa0 con 0x7f07900b0e20 2023-07-18T03:41:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.280+0000 7f07971ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07900d14d0 con 0x7f07900d0b10 2023-07-18T03:41:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.281+0000 7f078ffff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f07900d0b10 0x7f07900d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.281+0000 7f0794f9b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f07900b8a60 0x7f07900b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44522/0 (socket says 172.21.15.49:44522) 2023-07-18T03:41:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.281+0000 7f0794f9b700 1 -- 172.21.15.49:0/2331424813 learned_addr learned my addr 172.21.15.49:0/2331424813 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:22.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.281+0000 7f078f7fe700 1 -- 172.21.15.49:0/2331424813 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021974024 0 0) 0x7f07900b8e30 con 0x7f07900b8a60 2023-07-18T03:41:22.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.281+0000 7f078ffff700 1 -- 172.21.15.49:0/2331424813 >> v1:172.21.15.107:6789/0 conn(0x7f07900b0e20 legacy=0x7f07900b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.281+0000 7f078ffff700 1 -- 172.21.15.49:0/2331424813 >> v1:172.21.15.49:6789/0 conn(0x7f07900b8a60 legacy=0x7f07900b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.281+0000 7f078ffff700 1 -- 172.21.15.49:0/2331424813 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07900afd80 con 0x7f07900d0b10 2023-07-18T03:41:22.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.281+0000 7f078ffff700 1 --2- 172.21.15.49:0/2331424813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07900d0b10 0x7f07900d0f90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f077c009840 tx=0x7f077c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c09abf6cdcce1e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.282+0000 7f078f7fe700 1 -- 172.21.15.49:0/2331424813 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c011830 con 0x7f07900d0b10 2023-07-18T03:41:22.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.282+0000 7f078f7fe700 1 -- 172.21.15.49:0/2331424813 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c011990 con 0x7f07900d0b10 2023-07-18T03:41:22.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.282+0000 7f078f7fe700 1 -- 172.21.15.49:0/2331424813 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c011d20 con 0x7f07900d0b10 2023-07-18T03:41:22.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.282+0000 7f07971ff700 1 -- 172.21.15.49:0/2331424813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07900d0b10 msgr2=0x7f07900d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.282+0000 7f07971ff700 1 --2- 172.21.15.49:0/2331424813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07900d0b10 0x7f07900d0f90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f077c009840 tx=0x7f077c009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:22.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.282+0000 7f07971ff700 1 -- 172.21.15.49:0/2331424813 shutdown_connections 2023-07-18T03:41:22.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.282+0000 7f07971ff700 1 --2- 172.21.15.49:0/2331424813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07900d0b10 0x7f07900d0f90 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.282+0000 7f07971ff700 1 -- 172.21.15.49:0/2331424813 >> 172.21.15.49:0/2331424813 conn(0x7f079009c1b0 msgr2=0x7f07900d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:22.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.282+0000 7f07971ff700 1 -- 172.21.15.49:0/2331424813 shutdown_connections 2023-07-18T03:41:22.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.283+0000 7f07971ff700 1 -- 172.21.15.49:0/2331424813 wait complete. 2023-07-18T03:41:22.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.284+0000 7f07971ff700 1 Processor -- start 2023-07-18T03:41:22.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.284+0000 7f07971ff700 1 -- start start 2023-07-18T03:41:22.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.285+0000 7f07971ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07900b0e20 0x7f07900d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.285+0000 7f07971ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07900b8a60 0x7f07900cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.285+0000 7f07971ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07900d0b10 0x7f079015f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.285+0000 7f07971ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079015e190 con 0x7f07900d0b10 2023-07-18T03:41:22.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.285+0000 7f07971ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079015e470 con 0x7f07900b8a60 2023-07-18T03:41:22.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.285+0000 7f07971ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079015e750 con 0x7f07900b0e20 2023-07-18T03:41:22.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.286+0000 7f079579c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07900d0b10 0x7f079015f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.286+0000 7f079579c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07900d0b10 0x7f079015f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47796/0 (socket says 172.21.15.49:47796) 2023-07-18T03:41:22.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.286+0000 7f079579c700 1 -- 172.21.15.49:0/3626458959 learned_addr learned my addr 172.21.15.49:0/3626458959 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:22.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.286+0000 7f0794f9b700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07900b0e20 0x7f07900d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.286+0000 7f078ffff700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07900b8a60 0x7f07900cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.286+0000 7f079579c700 1 -- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07900b0e20 msgr2=0x7f07900d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.286+0000 7f079579c700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07900b0e20 0x7f07900d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.286+0000 7f079579c700 1 -- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07900b8a60 msgr2=0x7f07900cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.286+0000 7f079579c700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07900b8a60 0x7f07900cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.286+0000 7f079579c700 1 -- 172.21.15.49:0/3626458959 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c0094a0 con 0x7f07900d0b10 2023-07-18T03:41:22.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.286+0000 7f079579c700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07900d0b10 0x7f079015f230 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f078800cc50 tx=0x7f078800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.287+0000 7f078d7fa700 1 -- 172.21.15.49:0/3626458959 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788003c00 con 0x7f07900d0b10 2023-07-18T03:41:22.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.287+0000 7f078d7fa700 1 -- 172.21.15.49:0/3626458959 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788003d60 con 0x7f07900d0b10 2023-07-18T03:41:22.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.287+0000 7f078d7fa700 1 -- 172.21.15.49:0/3626458959 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078800aaf0 con 0x7f07900d0b10 2023-07-18T03:41:22.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.287+0000 7f07971ff700 1 -- 172.21.15.49:0/3626458959 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079015ea30 con 0x7f07900d0b10 2023-07-18T03:41:22.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.287+0000 7f07971ff700 1 -- 172.21.15.49:0/3626458959 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07900d07e0 con 0x7f07900d0b10 2023-07-18T03:41:22.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.288+0000 7f07971ff700 1 -- 172.21.15.49:0/3626458959 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079015ef50 con 0x7f07900d0b10 2023-07-18T03:41:22.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.289+0000 7f078d7fa700 1 -- 172.21.15.49:0/3626458959 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f078800ac50 con 0x7f07900d0b10 2023-07-18T03:41:22.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.290+0000 7f078d7fa700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f077406ae00 0x7f077406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.290+0000 7f078d7fa700 1 -- 172.21.15.49:0/3626458959 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f07880893d0 con 0x7f07900d0b10 2023-07-18T03:41:22.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.290+0000 7f0794f9b700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f077406ae00 0x7f077406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.291+0000 7f0794f9b700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f077406ae00 0x7f077406d2b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0784007b10 tx=0x7f0784007a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.294+0000 7f078d7fa700 1 -- 172.21.15.49:0/3626458959 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f079015ef50 con 0x7f07900d0b10 2023-07-18T03:41:22.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.499+0000 7f07971ff700 1 -- 172.21.15.49:0/3626458959 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f07900b0340 con 0x7f077406ae00 2023-07-18T03:41:22.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.500+0000 7f078d7fa700 1 -- 172.21.15.49:0/3626458959 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0784001330 con 0x7f077406ae00 2023-07-18T03:41:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.502+0000 7f07971ff700 1 -- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f077406ae00 msgr2=0x7f077406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.502+0000 7f07971ff700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f077406ae00 0x7f077406d2b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0784007b10 tx=0x7f0784007a60 comp rx=0 tx=0).stop 2023-07-18T03:41:22.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.502+0000 7f07971ff700 1 -- 172.21.15.49:0/3626458959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07900d0b10 msgr2=0x7f079015f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.502+0000 7f07971ff700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07900d0b10 0x7f079015f230 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f078800cc50 tx=0x7f078800cf60 comp rx=0 tx=0).stop 2023-07-18T03:41:22.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.502+0000 7f07971ff700 1 -- 172.21.15.49:0/3626458959 shutdown_connections 2023-07-18T03:41:22.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.502+0000 7f07971ff700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f077406ae00 0x7f077406d2b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.502+0000 7f07971ff700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07900d0b10 0x7f079015f230 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.502+0000 7f07971ff700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07900b0e20 0x7f07900d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.502+0000 7f07971ff700 1 --2- 172.21.15.49:0/3626458959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07900b8a60 0x7f07900cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.502+0000 7f07971ff700 1 -- 172.21.15.49:0/3626458959 >> 172.21.15.49:0/3626458959 conn(0x7f079009c1b0 msgr2=0x7f079006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:22.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.502+0000 7f07971ff700 1 -- 172.21.15.49:0/3626458959 shutdown_connections 2023-07-18T03:41:22.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.502+0000 7f07971ff700 1 -- 172.21.15.49:0/3626458959 wait complete. 2023-07-18T03:41:22.501 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 5 to deep-scrub 2023-07-18T03:41:22.515 INFO:tasks.ceph:Scrubbing osd.6 2023-07-18T03:41:22.515 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-07-18T03:41:22.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.730+0000 7f9fa75ff700 1 Processor -- start 2023-07-18T03:41:22.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.730+0000 7f9fa75ff700 1 -- start start 2023-07-18T03:41:22.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.730+0000 7f9fa75ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fa015c7b0 0x7f9fa00d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.730+0000 7f9fa75ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9fa00b1e50 0x7f9fa00d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.730+0000 7f9fa75ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa00d0190 con 0x7f9fa00d0f70 2023-07-18T03:41:22.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.730+0000 7f9fa75ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa00d3de0 con 0x7f9fa00b1e50 2023-07-18T03:41:22.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.730+0000 7f9fa75ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00d13f0 con 0x7f9fa015c7b0 2023-07-18T03:41:22.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.730+0000 7f9fa4b9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9fa00b1e50 0x7f9fa00d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.731+0000 7f9fa4b9a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9fa00b1e50 0x7f9fa00d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47812/0 (socket says 172.21.15.49:47812) 2023-07-18T03:41:22.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.731+0000 7f9fa5b9c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9fa00d0f70 0x7f9fa0160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44538/0 (socket says 172.21.15.49:44538) 2023-07-18T03:41:22.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.731+0000 7f9fa4b9a700 1 -- 172.21.15.49:0/2166657347 learned_addr learned my addr 172.21.15.49:0/2166657347 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:22.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.731+0000 7f9fa539b700 1 --2- 172.21.15.49:0/2166657347 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fa015c7b0 0x7f9fa00d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.731+0000 7f9fa4b9a700 1 -- 172.21.15.49:0/2166657347 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fa015c7b0 msgr2=0x7f9fa00d28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:22.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.731+0000 7f9fa4b9a700 1 --2- 172.21.15.49:0/2166657347 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fa015c7b0 0x7f9fa00d28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.731+0000 7f9fa4b9a700 1 -- 172.21.15.49:0/2166657347 >> v1:172.21.15.49:6789/0 conn(0x7f9fa00d0f70 legacy=0x7f9fa0160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.731+0000 7f9fa4b9a700 1 -- 172.21.15.49:0/2166657347 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa00d0470 con 0x7f9fa00b1e50 2023-07-18T03:41:22.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.731+0000 7f9fa4b9a700 1 --2- 172.21.15.49:0/2166657347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa00b1e50 0x7f9fa00d0a30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9f900098f0 tx=0x7f9f90009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ec49f12a2771ab server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.731+0000 7f9f97fff700 1 -- 172.21.15.49:0/2166657347 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90011830 con 0x7f9fa00b1e50 2023-07-18T03:41:22.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.731+0000 7f9f97fff700 1 -- 172.21.15.49:0/2166657347 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f90011990 con 0x7f9fa00b1e50 2023-07-18T03:41:22.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.732+0000 7f9f97fff700 1 -- 172.21.15.49:0/2166657347 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90011d20 con 0x7f9fa00b1e50 2023-07-18T03:41:22.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.732+0000 7f9fa75ff700 1 -- 172.21.15.49:0/2166657347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa00b1e50 msgr2=0x7f9fa00d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.732+0000 7f9fa75ff700 1 --2- 172.21.15.49:0/2166657347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa00b1e50 0x7f9fa00d0a30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9f900098f0 tx=0x7f9f90009c00 comp rx=0 tx=0).stop 2023-07-18T03:41:22.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.732+0000 7f9fa75ff700 1 -- 172.21.15.49:0/2166657347 shutdown_connections 2023-07-18T03:41:22.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.732+0000 7f9fa75ff700 1 --2- 172.21.15.49:0/2166657347 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fa015c7b0 0x7f9fa00d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.732+0000 7f9fa75ff700 1 --2- 172.21.15.49:0/2166657347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa00b1e50 0x7f9fa00d0a30 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.732+0000 7f9fa75ff700 1 -- 172.21.15.49:0/2166657347 >> 172.21.15.49:0/2166657347 conn(0x7f9fa009c1b0 msgr2=0x7f9fa00d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:22.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.732+0000 7f9fa75ff700 1 -- 172.21.15.49:0/2166657347 shutdown_connections 2023-07-18T03:41:22.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.732+0000 7f9fa75ff700 1 -- 172.21.15.49:0/2166657347 wait complete. 2023-07-18T03:41:22.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.735+0000 7f9fa75ff700 1 Processor -- start 2023-07-18T03:41:22.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.735+0000 7f9fa75ff700 1 -- start start 2023-07-18T03:41:22.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.736+0000 7f9fa75ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fa00b1e50 0x7f9fa00b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.736+0000 7f9fa75ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa00d0f70 0x7f9fa006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.736+0000 7f9fa75ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fa015c7b0 0x7f9fa00cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.736+0000 7f9fa75ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00b7f60 con 0x7f9fa00d0f70 2023-07-18T03:41:22.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.736+0000 7f9fa75ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa015f8c0 con 0x7f9fa00b1e50 2023-07-18T03:41:22.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.736+0000 7f9fa75ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00b8190 con 0x7f9fa015c7b0 2023-07-18T03:41:22.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.736+0000 7f9fa4b9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa00d0f70 0x7f9fa006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.737+0000 7f9fa4b9a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa00d0f70 0x7f9fa006bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47822/0 (socket says 172.21.15.49:47822) 2023-07-18T03:41:22.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.737+0000 7f9fa4b9a700 1 -- 172.21.15.49:0/547523004 learned_addr learned my addr 172.21.15.49:0/547523004 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:22.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.737+0000 7f9fa5b9c700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fa015c7b0 0x7f9fa00cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.737+0000 7f9fa539b700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fa00b1e50 0x7f9fa00b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.737+0000 7f9fa4b9a700 1 -- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fa015c7b0 msgr2=0x7f9fa00cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.737+0000 7f9fa4b9a700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fa015c7b0 0x7f9fa00cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.737+0000 7f9fa4b9a700 1 -- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fa00b1e50 msgr2=0x7f9fa00b7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.737+0000 7f9fa4b9a700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fa00b1e50 0x7f9fa00b7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.737+0000 7f9fa4b9a700 1 -- 172.21.15.49:0/547523004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f90009550 con 0x7f9fa00d0f70 2023-07-18T03:41:22.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.737+0000 7f9fa5b9c700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fa015c7b0 0x7f9fa00cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:22.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.737+0000 7f9fa4b9a700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa00d0f70 0x7f9fa006bab0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009c40 tx=0x7f9f90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.737+0000 7f9f95ffb700 1 -- 172.21.15.49:0/547523004 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f900045b0 con 0x7f9fa00d0f70 2023-07-18T03:41:22.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.738+0000 7f9f95ffb700 1 -- 172.21.15.49:0/547523004 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9001f070 con 0x7f9fa00d0f70 2023-07-18T03:41:22.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.738+0000 7f9fa75ff700 1 -- 172.21.15.49:0/547523004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa00ca0d0 con 0x7f9fa00d0f70 2023-07-18T03:41:22.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.738+0000 7f9f95ffb700 1 -- 172.21.15.49:0/547523004 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f900038e0 con 0x7f9fa00d0f70 2023-07-18T03:41:22.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.738+0000 7f9fa75ff700 1 -- 172.21.15.49:0/547523004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa00ca690 con 0x7f9fa00d0f70 2023-07-18T03:41:22.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.738+0000 7f9fa75ff700 1 -- 172.21.15.49:0/547523004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9fa00cd550 con 0x7f9fa00d0f70 2023-07-18T03:41:22.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.740+0000 7f9f95ffb700 1 -- 172.21.15.49:0/547523004 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f9f90021030 con 0x7f9fa00d0f70 2023-07-18T03:41:22.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.741+0000 7f9f95ffb700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9f8406ae00 0x7f9f8406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.741+0000 7f9f95ffb700 1 -- 172.21.15.49:0/547523004 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9f9008ad60 con 0x7f9fa00d0f70 2023-07-18T03:41:22.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.741+0000 7f9f95ffb700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] conn(0x7f9f84071270 0x7f9f84073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.741+0000 7f9f95ffb700 1 -- 172.21.15.49:0/547523004 --> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f84073d50 con 0x7f9f84071270 2023-07-18T03:41:22.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.741+0000 7f9f95ffb700 1 -- 172.21.15.49:0/547523004 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f9008b190 con 0x7f9fa00d0f70 2023-07-18T03:41:22.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.741+0000 7f9fa539b700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9f8406ae00 0x7f9f8406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.741+0000 7f9fa5b9c700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] conn(0x7f9f84071270 0x7f9f84073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.742+0000 7f9fa5b9c700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] conn(0x7f9f84071270 0x7f9f84073680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.742+0000 7f9fa539b700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9f8406ae00 0x7f9f8406d2b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009ad0 tx=0x7f9f9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.743+0000 7f9f95ffb700 1 -- 172.21.15.49:0/547523004 <== osd.6 v2:172.21.15.107:6808/447642181 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9f84073d50 con 0x7f9f84071270 2023-07-18T03:41:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.758+0000 7f9fa75ff700 1 -- 172.21.15.49:0/547523004 --> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9fa00d0750 con 0x7f9f84071270 2023-07-18T03:41:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.759+0000 7f9f95ffb700 1 -- 172.21.15.49:0/547523004 <== osd.6 v2:172.21.15.107:6808/447642181 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9f980070f0 con 0x7f9f84071270 2023-07-18T03:41:22.754 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:41:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-18T03:41:22.754 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:41:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.759+0000 7f9fa75ff700 1 -- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] conn(0x7f9f84071270 msgr2=0x7f9f84073680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.759+0000 7f9fa75ff700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:6808/447642181,v1:172.21.15.107:6809/447642181] conn(0x7f9f84071270 0x7f9f84073680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.759+0000 7f9fa75ff700 1 -- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9f8406ae00 msgr2=0x7f9f8406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.759+0000 7f9fa75ff700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f9f8406ae00 0x7f9f8406d2b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009ad0 tx=0x7f9f9c008040 comp rx=0 tx=0).stop 2023-07-18T03:41:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.759+0000 7f9fa75ff700 1 -- 172.21.15.49:0/547523004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa00d0f70 msgr2=0x7f9fa006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.759+0000 7f9fa75ff700 1 --2- 172.21.15.49:0/547523004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9fa00d0f70 0x7f9fa006bab0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009c40 tx=0x7f9f90004750 comp rx=0 tx=0).stop 2023-07-18T03:41:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.759+0000 7f9fa5b9c700 1 -- 172.21.15.49:0/547523004 reap_dead start 2023-07-18T03:41:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.759+0000 7f9fa75ff700 1 -- 172.21.15.49:0/547523004 shutdown_connections 2023-07-18T03:41:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.759+0000 7f9fa75ff700 1 -- 172.21.15.49:0/547523004 >> 172.21.15.49:0/547523004 conn(0x7f9fa009c1b0 msgr2=0x7f9fa00b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.759+0000 7f9fa75ff700 1 -- 172.21.15.49:0/547523004 shutdown_connections 2023-07-18T03:41:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.759+0000 7f9fa75ff700 1 -- 172.21.15.49:0/547523004 wait complete. 2023-07-18T03:41:22.771 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-07-18T03:41:22.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.982+0000 7f2b1d9ff700 1 Processor -- start 2023-07-18T03:41:22.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.982+0000 7f2b1d9ff700 1 -- start start 2023-07-18T03:41:22.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b1d9ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b180b8a60 0x7f2b180b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b1d9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b180b4530 0x7f2b180d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b1d9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2b180d1050 0x7f2b180b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b1d9ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b180d14d0 con 0x7f2b180b8a60 2023-07-18T03:41:22.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b1d9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b180b8e30 con 0x7f2b180d1050 2023-07-18T03:41:22.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b1d9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b180afaa0 con 0x7f2b180b4530 2023-07-18T03:41:22.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b16ffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b180b8a60 0x7f2b180b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b16ffd700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b180b8a60 0x7f2b180b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47830/0 (socket says 172.21.15.49:47830) 2023-07-18T03:41:22.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b16ffd700 1 -- 172.21.15.49:0/374656420 learned_addr learned my addr 172.21.15.49:0/374656420 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:22.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b167fc700 1 --2- 172.21.15.49:0/374656420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b180b4530 0x7f2b180d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b177fe700 1 --2- 172.21.15.49:0/374656420 >> v2:172.21.15.107:3300/0 conn(0x7f2b180d1050 0x7f2b180b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b16ffd700 1 -- 172.21.15.49:0/374656420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b180b4530 msgr2=0x7f2b180d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:22.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b16ffd700 1 --2- 172.21.15.49:0/374656420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b180b4530 0x7f2b180d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b16ffd700 1 -- 172.21.15.49:0/374656420 >> v2:172.21.15.107:3300/0 conn(0x7f2b180d1050 msgr2=0x7f2b180b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:22.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b16ffd700 1 --2- 172.21.15.49:0/374656420 >> v2:172.21.15.107:3300/0 conn(0x7f2b180d1050 0x7f2b180b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.984+0000 7f2b16ffd700 1 -- 172.21.15.49:0/374656420 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b180afd80 con 0x7f2b180b8a60 2023-07-18T03:41:22.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.985+0000 7f2b16ffd700 1 --2- 172.21.15.49:0/374656420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b180b8a60 0x7f2b180b3ff0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2b1808c2e0 tx=0x7f2b08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e10883ba6ac9e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.985+0000 7f2b15ffb700 1 -- 172.21.15.49:0/374656420 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08011830 con 0x7f2b180b8a60 2023-07-18T03:41:22.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.985+0000 7f2b15ffb700 1 -- 172.21.15.49:0/374656420 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b08011990 con 0x7f2b180b8a60 2023-07-18T03:41:22.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.985+0000 7f2b15ffb700 1 -- 172.21.15.49:0/374656420 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08011d20 con 0x7f2b180b8a60 2023-07-18T03:41:22.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.985+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/374656420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b180b8a60 msgr2=0x7f2b180b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.985+0000 7f2b1d9ff700 1 --2- 172.21.15.49:0/374656420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b180b8a60 0x7f2b180b3ff0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2b1808c2e0 tx=0x7f2b08009bd0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.986+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/374656420 shutdown_connections 2023-07-18T03:41:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.986+0000 7f2b1d9ff700 1 --2- 172.21.15.49:0/374656420 >> v2:172.21.15.107:3300/0 conn(0x7f2b180d1050 0x7f2b180b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.986+0000 7f2b1d9ff700 1 --2- 172.21.15.49:0/374656420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b180b8a60 0x7f2b180b3ff0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.986+0000 7f2b1d9ff700 1 --2- 172.21.15.49:0/374656420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b180b4530 0x7f2b180d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.986+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/374656420 >> 172.21.15.49:0/374656420 conn(0x7f2b1809c1b0 msgr2=0x7f2b180d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:22.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.986+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/374656420 shutdown_connections 2023-07-18T03:41:22.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.986+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/374656420 wait complete. 2023-07-18T03:41:22.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.988+0000 7f2b1d9ff700 1 Processor -- start 2023-07-18T03:41:22.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.988+0000 7f2b1d9ff700 1 -- start start 2023-07-18T03:41:22.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b1d9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b180b4530 0x7f2b18151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b1d9ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b180b8a60 0x7f2b181f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b1d9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b180d1050 0x7f2b181f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b1d9ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1815cdb0 con 0x7f2b180b8a60 2023-07-18T03:41:22.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b1d9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1815d090 con 0x7f2b180d1050 2023-07-18T03:41:22.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b1d9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b18152290 con 0x7f2b180b4530 2023-07-18T03:41:22.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b167fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b180b8a60 0x7f2b181f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b167fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b180b8a60 0x7f2b181f5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47844/0 (socket says 172.21.15.49:47844) 2023-07-18T03:41:22.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b167fc700 1 -- 172.21.15.49:0/934857718 learned_addr learned my addr 172.21.15.49:0/934857718 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:22.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b16ffd700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b180b4530 0x7f2b18151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b177fe700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b180d1050 0x7f2b181f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b167fc700 1 -- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b180b4530 msgr2=0x7f2b18151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b167fc700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b180b4530 0x7f2b18151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b167fc700 1 -- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b180d1050 msgr2=0x7f2b181f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:22.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b167fc700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b180d1050 0x7f2b181f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:22.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.989+0000 7f2b167fc700 1 -- 172.21.15.49:0/934857718 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b08009550 con 0x7f2b180b8a60 2023-07-18T03:41:22.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.990+0000 7f2b167fc700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b180b8a60 0x7f2b181f5d80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00ea10 tx=0x7f2b0c00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.990+0000 7f2afffff700 1 -- 172.21.15.49:0/934857718 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c009710 con 0x7f2b180b8a60 2023-07-18T03:41:22.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.990+0000 7f2afffff700 1 -- 172.21.15.49:0/934857718 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c00eeb0 con 0x7f2b180b8a60 2023-07-18T03:41:22.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.990+0000 7f2afffff700 1 -- 172.21.15.49:0/934857718 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c00ca50 con 0x7f2b180b8a60 2023-07-18T03:41:22.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.990+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/934857718 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1815f6e0 con 0x7f2b180b8a60 2023-07-18T03:41:22.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.990+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/934857718 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1815fca0 con 0x7f2b180b8a60 2023-07-18T03:41:22.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.991+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/934857718 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b1815d220 con 0x7f2b180b8a60 2023-07-18T03:41:22.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.992+0000 7f2afffff700 1 -- 172.21.15.49:0/934857718 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f2b0c00cbb0 con 0x7f2b180b8a60 2023-07-18T03:41:22.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.993+0000 7f2afffff700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2b0406ae00 0x7f2b0406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:22.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.993+0000 7f2afffff700 1 -- 172.21.15.49:0/934857718 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2b0c089230 con 0x7f2b180b8a60 2023-07-18T03:41:22.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.994+0000 7f2b16ffd700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2b0406ae00 0x7f2b0406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:22.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.994+0000 7f2b16ffd700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2b0406ae00 0x7f2b0406d2b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009c10 tx=0x7f2b08008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:22.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:22.998+0000 7f2afffff700 1 -- 172.21.15.49:0/934857718 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2b0c057f70 con 0x7f2b180b8a60 2023-07-18T03:41:23.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.206+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/934857718 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b180b0340 con 0x7f2b0406ae00 2023-07-18T03:41:23.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.206+0000 7f2afffff700 1 -- 172.21.15.49:0/934857718 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2b08011810 con 0x7f2b0406ae00 2023-07-18T03:41:23.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.209+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2b0406ae00 msgr2=0x7f2b0406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.209+0000 7f2b1d9ff700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2b0406ae00 0x7f2b0406d2b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009c10 tx=0x7f2b08008030 comp rx=0 tx=0).stop 2023-07-18T03:41:23.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.209+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/934857718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b180b8a60 msgr2=0x7f2b181f5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.209+0000 7f2b1d9ff700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b180b8a60 0x7f2b181f5d80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00ea10 tx=0x7f2b0c00ed20 comp rx=0 tx=0).stop 2023-07-18T03:41:23.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.209+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/934857718 shutdown_connections 2023-07-18T03:41:23.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.209+0000 7f2b1d9ff700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b180d1050 0x7f2b181f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.209+0000 7f2b1d9ff700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b180b4530 0x7f2b18151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.209+0000 7f2b1d9ff700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f2b0406ae00 0x7f2b0406d2b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.209+0000 7f2b1d9ff700 1 --2- 172.21.15.49:0/934857718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b180b8a60 0x7f2b181f5d80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.209+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/934857718 >> 172.21.15.49:0/934857718 conn(0x7f2b1809c1b0 msgr2=0x7f2b1806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:23.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.209+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/934857718 shutdown_connections 2023-07-18T03:41:23.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.209+0000 7f2b1d9ff700 1 -- 172.21.15.49:0/934857718 wait complete. 2023-07-18T03:41:23.207 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 6 to deep-scrub 2023-07-18T03:41:23.222 INFO:tasks.ceph:Scrubbing osd.7 2023-07-18T03:41:23.223 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-07-18T03:41:23.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.437+0000 7f50701ff700 1 Processor -- start 2023-07-18T03:41:23.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.437+0000 7f50701ff700 1 -- start start 2023-07-18T03:41:23.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.438+0000 7f50701ff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5068160c50 0x7f506806c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.438+0000 7f50701ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50680af170 0x7f506806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.438+0000 7f50701ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506806dfb0 con 0x7f506806dbe0 2023-07-18T03:41:23.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.438+0000 7f50701ff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506815cb50 con 0x7f5068160c50 2023-07-18T03:41:23.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.438+0000 7f50701ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50681610c0 con 0x7f50680af170 2023-07-18T03:41:23.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.438+0000 7f506d79a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5068160c50 0x7f506806c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.438+0000 7f506d79a700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5068160c50 0x7f506806c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47846/0 (socket says 172.21.15.49:47846) 2023-07-18T03:41:23.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.439+0000 7f506d79a700 1 -- 172.21.15.49:0/4003982349 learned_addr learned my addr 172.21.15.49:0/4003982349 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:23.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.439+0000 7f506e79c700 1 --2- 172.21.15.49:0/4003982349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50680af170 0x7f506806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.439+0000 7f506d79a700 1 -- 172.21.15.49:0/4003982349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50680af170 msgr2=0x7f506806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:23.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.439+0000 7f506d79a700 1 --2- 172.21.15.49:0/4003982349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50680af170 0x7f506806c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.439+0000 7f506d79a700 1 -- 172.21.15.49:0/4003982349 >> v1:172.21.15.107:6789/0 conn(0x7f506806dbe0 legacy=0x7f5068069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.439+0000 7f506d79a700 1 -- 172.21.15.49:0/4003982349 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50680ce830 con 0x7f5068160c50 2023-07-18T03:41:23.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.439+0000 7f506e79c700 1 --2- 172.21.15.49:0/4003982349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50680af170 0x7f506806c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:23.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.439+0000 7f506d79a700 1 --2- 172.21.15.49:0/4003982349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5068160c50 0x7f506806c2c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f50580098f0 tx=0x7f5058009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388d4080a11806 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.439+0000 7f506cf99700 1 -- 172.21.15.49:0/4003982349 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505800b8a0 con 0x7f5068160c50 2023-07-18T03:41:23.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.439+0000 7f506cf99700 1 -- 172.21.15.49:0/4003982349 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505800ba00 con 0x7f5068160c50 2023-07-18T03:41:23.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.439+0000 7f506cf99700 1 -- 172.21.15.49:0/4003982349 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505800bd90 con 0x7f5068160c50 2023-07-18T03:41:23.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.440+0000 7f50701ff700 1 -- 172.21.15.49:0/4003982349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5068160c50 msgr2=0x7f506806c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.440+0000 7f50701ff700 1 --2- 172.21.15.49:0/4003982349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5068160c50 0x7f506806c2c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f50580098f0 tx=0x7f5058009c00 comp rx=0 tx=0).stop 2023-07-18T03:41:23.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.440+0000 7f50701ff700 1 -- 172.21.15.49:0/4003982349 shutdown_connections 2023-07-18T03:41:23.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.440+0000 7f50701ff700 1 --2- 172.21.15.49:0/4003982349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50680af170 0x7f506806c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.440+0000 7f50701ff700 1 --2- 172.21.15.49:0/4003982349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5068160c50 0x7f506806c2c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.440+0000 7f50701ff700 1 -- 172.21.15.49:0/4003982349 >> 172.21.15.49:0/4003982349 conn(0x7f50680cfb10 msgr2=0x7f50680cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:23.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.440+0000 7f50701ff700 1 -- 172.21.15.49:0/4003982349 shutdown_connections 2023-07-18T03:41:23.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.440+0000 7f50701ff700 1 -- 172.21.15.49:0/4003982349 wait complete. 2023-07-18T03:41:23.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.442+0000 7f50701ff700 1 Processor -- start 2023-07-18T03:41:23.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.442+0000 7f50701ff700 1 -- start start 2023-07-18T03:41:23.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.443+0000 7f50701ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f506806dbe0 0x7f5068150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.443+0000 7f50701ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50680af170 0x7f50680c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.443+0000 7f50701ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5068160c50 0x7f50680c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.443+0000 7f50701ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50680c9b00 con 0x7f5068160c50 2023-07-18T03:41:23.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.443+0000 7f50701ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5068151260 con 0x7f506806dbe0 2023-07-18T03:41:23.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.443+0000 7f50701ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50681e8200 con 0x7f50680af170 2023-07-18T03:41:23.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.443+0000 7f506e79c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5068160c50 0x7f50680c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.443+0000 7f506e79c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5068160c50 0x7f50680c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47850/0 (socket says 172.21.15.49:47850) 2023-07-18T03:41:23.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.443+0000 7f506e79c700 1 -- 172.21.15.49:0/694319602 learned_addr learned my addr 172.21.15.49:0/694319602 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:23.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.443+0000 7f506d79a700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50680af170 0x7f50680c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.443+0000 7f506df9b700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f506806dbe0 0x7f5068150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.444+0000 7f506e79c700 1 -- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50680af170 msgr2=0x7f50680c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.444+0000 7f506e79c700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50680af170 0x7f50680c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.444+0000 7f506e79c700 1 -- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f506806dbe0 msgr2=0x7f5068150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.444+0000 7f506e79c700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f506806dbe0 0x7f5068150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.444+0000 7f506e79c700 1 -- 172.21.15.49:0/694319602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5058009550 con 0x7f5068160c50 2023-07-18T03:41:23.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.444+0000 7f506e79c700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5068160c50 0x7f50680c8ce0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5060009fa0 tx=0x7f506000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.444+0000 7f505e7fc700 1 -- 172.21.15.49:0/694319602 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506000ddb0 con 0x7f5068160c50 2023-07-18T03:41:23.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.444+0000 7f505e7fc700 1 -- 172.21.15.49:0/694319602 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506000f040 con 0x7f5068160c50 2023-07-18T03:41:23.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.444+0000 7f505e7fc700 1 -- 172.21.15.49:0/694319602 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060003c60 con 0x7f5068160c50 2023-07-18T03:41:23.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.444+0000 7f50701ff700 1 -- 172.21.15.49:0/694319602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50681e84e0 con 0x7f5068160c50 2023-07-18T03:41:23.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.445+0000 7f50701ff700 1 -- 172.21.15.49:0/694319602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50681e8aa0 con 0x7f5068160c50 2023-07-18T03:41:23.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.445+0000 7f50701ff700 1 -- 172.21.15.49:0/694319602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f50680cedf0 con 0x7f5068160c50 2023-07-18T03:41:23.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.446+0000 7f505e7fc700 1 -- 172.21.15.49:0/694319602 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f5060004030 con 0x7f5068160c50 2023-07-18T03:41:23.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.447+0000 7f505e7fc700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f504c06ae00 0x7f504c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.447+0000 7f505e7fc700 1 -- 172.21.15.49:0/694319602 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f50600894f0 con 0x7f5068160c50 2023-07-18T03:41:23.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.448+0000 7f505e7fc700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] conn(0x7f504c071270 0x7f504c073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.448+0000 7f505e7fc700 1 -- 172.21.15.49:0/694319602 --> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f504c073d50 con 0x7f504c071270 2023-07-18T03:41:23.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.448+0000 7f505e7fc700 1 -- 172.21.15.49:0/694319602 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f506008bad0 con 0x7f5068160c50 2023-07-18T03:41:23.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.448+0000 7f506df9b700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f504c06ae00 0x7f504c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.448+0000 7f506d79a700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] conn(0x7f504c071270 0x7f504c073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.448+0000 7f506df9b700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f504c06ae00 0x7f504c06d2b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5068151720 tx=0x7f5064009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.449+0000 7f506d79a700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] conn(0x7f504c071270 0x7f504c073680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.450+0000 7f505e7fc700 1 -- 172.21.15.49:0/694319602 <== osd.7 v2:172.21.15.107:6816/2327910262 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f504c073d50 con 0x7f504c071270 2023-07-18T03:41:23.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.465+0000 7f50701ff700 1 -- 172.21.15.49:0/694319602 --> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f50680ceb10 con 0x7f504c071270 2023-07-18T03:41:23.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.465+0000 7f505e7fc700 1 -- 172.21.15.49:0/694319602 <== osd.7 v2:172.21.15.107:6816/2327910262 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5058021040 con 0x7f504c071270 2023-07-18T03:41:23.461 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-07-18T03:41:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-18T03:41:23.462 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-07-18T03:41:23.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.466+0000 7f50701ff700 1 -- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] conn(0x7f504c071270 msgr2=0x7f504c073680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.466+0000 7f50701ff700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:6816/2327910262,v1:172.21.15.107:6817/2327910262] conn(0x7f504c071270 0x7f504c073680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.466+0000 7f50701ff700 1 -- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f504c06ae00 msgr2=0x7f504c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.466+0000 7f50701ff700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f504c06ae00 0x7f504c06d2b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5068151720 tx=0x7f5064009040 comp rx=0 tx=0).stop 2023-07-18T03:41:23.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.466+0000 7f50701ff700 1 -- 172.21.15.49:0/694319602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5068160c50 msgr2=0x7f50680c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.466+0000 7f50701ff700 1 --2- 172.21.15.49:0/694319602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5068160c50 0x7f50680c8ce0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5060009fa0 tx=0x7f506000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.466+0000 7f506e79c700 1 -- 172.21.15.49:0/694319602 reap_dead start 2023-07-18T03:41:23.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.466+0000 7f50701ff700 1 -- 172.21.15.49:0/694319602 shutdown_connections 2023-07-18T03:41:23.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.466+0000 7f50701ff700 1 -- 172.21.15.49:0/694319602 >> 172.21.15.49:0/694319602 conn(0x7f50680cfb10 msgr2=0x7f506806c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:23.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.466+0000 7f50701ff700 1 -- 172.21.15.49:0/694319602 shutdown_connections 2023-07-18T03:41:23.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.466+0000 7f50701ff700 1 -- 172.21.15.49:0/694319602 wait complete. 2023-07-18T03:41:23.479 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-07-18T03:41:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.690+0000 7f9888fff700 1 Processor -- start 2023-07-18T03:41:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.690+0000 7f9888fff700 1 -- start start 2023-07-18T03:41:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.692+0000 7f9888fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f98840d0010 0x7f98840b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.692+0000 7f9888fff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988415cd60 con 0x7f98840d0480 2023-07-18T03:41:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.692+0000 7f9888fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98840cf340 con 0x7f98840b4600 2023-07-18T03:41:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.692+0000 7f9888fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98840d1550 con 0x7f98840d0010 2023-07-18T03:41:23.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.692+0000 7f9882d9d700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f98840d0480 0x7f98840b6430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44548/0 (socket says 172.21.15.49:44548) 2023-07-18T03:41:23.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.692+0000 7f9882d9d700 1 -- 172.21.15.49:0/3282535649 learned_addr learned my addr 172.21.15.49:0/3282535649 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:23.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.692+0000 7f9881d9b700 1 --2- 172.21.15.49:0/3282535649 >> v2:172.21.15.107:3300/0 conn(0x7f98840d0010 0x7f98840b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.692+0000 7f988159a700 1 -- 172.21.15.49:0/3282535649 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194032321 0 0) 0x7f988415cd60 con 0x7f98840d0480 2023-07-18T03:41:23.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.693+0000 7f988159a700 1 -- 172.21.15.49:0/3282535649 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f986c0035b0 con 0x7f98840d0480 2023-07-18T03:41:23.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.693+0000 7f9881d9b700 1 -- 172.21.15.49:0/3282535649 >> v1:172.21.15.107:6789/0 conn(0x7f98840b4600 legacy=0x7f98840cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.693+0000 7f9881d9b700 1 -- 172.21.15.49:0/3282535649 >> v1:172.21.15.49:6789/0 conn(0x7f98840d0480 legacy=0x7f98840b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.693+0000 7f9881d9b700 1 -- 172.21.15.49:0/3282535649 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98840d0e90 con 0x7f98840d0010 2023-07-18T03:41:23.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.693+0000 7f9881d9b700 1 --2- 172.21.15.49:0/3282535649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98840d0010 0x7f98840b5ef0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9878009840 tx=0x7f9878009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d31d84b52177783 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.693+0000 7f988159a700 1 -- 172.21.15.49:0/3282535649 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9878011830 con 0x7f98840d0010 2023-07-18T03:41:23.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.693+0000 7f988159a700 1 -- 172.21.15.49:0/3282535649 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9878011990 con 0x7f98840d0010 2023-07-18T03:41:23.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.693+0000 7f988159a700 1 -- 172.21.15.49:0/3282535649 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9878011d00 con 0x7f98840d0010 2023-07-18T03:41:23.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.694+0000 7f9888fff700 1 -- 172.21.15.49:0/3282535649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98840d0010 msgr2=0x7f98840b5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.694+0000 7f9888fff700 1 --2- 172.21.15.49:0/3282535649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98840d0010 0x7f98840b5ef0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9878009840 tx=0x7f9878009b50 comp rx=0 tx=0).stop 2023-07-18T03:41:23.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.694+0000 7f9888fff700 1 -- 172.21.15.49:0/3282535649 shutdown_connections 2023-07-18T03:41:23.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.694+0000 7f9888fff700 1 --2- 172.21.15.49:0/3282535649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98840d0010 0x7f98840b5ef0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.694+0000 7f9888fff700 1 -- 172.21.15.49:0/3282535649 >> 172.21.15.49:0/3282535649 conn(0x7f988409c200 msgr2=0x7f98840d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:23.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.694+0000 7f9888fff700 1 -- 172.21.15.49:0/3282535649 shutdown_connections 2023-07-18T03:41:23.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.694+0000 7f9888fff700 1 -- 172.21.15.49:0/3282535649 wait complete. 2023-07-18T03:41:23.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.696+0000 7f9888fff700 1 Processor -- start 2023-07-18T03:41:23.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.696+0000 7f9888fff700 1 -- start start 2023-07-18T03:41:23.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9888fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98840d0010 0x7f98840c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9888fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98840d0480 0x7f98840cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9888fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98840cd980 0x7f98841f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9888fff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988409d580 con 0x7f98840d0480 2023-07-18T03:41:23.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9888fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988409cc70 con 0x7f98840d0010 2023-07-18T03:41:23.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9888fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98840cde90 con 0x7f98840cd980 2023-07-18T03:41:23.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9881d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98840d0480 0x7f98840cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9881d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98840d0480 0x7f98840cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47864/0 (socket says 172.21.15.49:47864) 2023-07-18T03:41:23.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9881d9b700 1 -- 172.21.15.49:0/789563075 learned_addr learned my addr 172.21.15.49:0/789563075 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:23.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9882d9d700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98840cd980 0x7f98841f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f988259c700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98840d0010 0x7f98840c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9881d9b700 1 -- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98840cd980 msgr2=0x7f98841f13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9881d9b700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98840cd980 0x7f98841f13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9881d9b700 1 -- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98840d0010 msgr2=0x7f98840c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.697+0000 7f9881d9b700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98840d0010 0x7f98840c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.698+0000 7f9881d9b700 1 -- 172.21.15.49:0/789563075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780094a0 con 0x7f98840d0480 2023-07-18T03:41:23.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.698+0000 7f988259c700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98840d0010 0x7f98840c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:23.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.698+0000 7f9882d9d700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98840cd980 0x7f98841f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:23.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.698+0000 7f9881d9b700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98840d0480 0x7f98840cd440 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9878009b90 tx=0x7f9878004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.698+0000 7f9872ffd700 1 -- 172.21.15.49:0/789563075 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780036a0 con 0x7f98840d0480 2023-07-18T03:41:23.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.698+0000 7f9872ffd700 1 -- 172.21.15.49:0/789563075 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987801f070 con 0x7f98840d0480 2023-07-18T03:41:23.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.698+0000 7f9872ffd700 1 -- 172.21.15.49:0/789563075 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9878003a40 con 0x7f98840d0480 2023-07-18T03:41:23.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.698+0000 7f9888fff700 1 -- 172.21.15.49:0/789563075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988409cec0 con 0x7f98840d0480 2023-07-18T03:41:23.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.698+0000 7f9888fff700 1 -- 172.21.15.49:0/789563075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98840ca1a0 con 0x7f98840d0480 2023-07-18T03:41:23.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.699+0000 7f9888fff700 1 -- 172.21.15.49:0/789563075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988409d6c0 con 0x7f98840d0480 2023-07-18T03:41:23.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.700+0000 7f9872ffd700 1 -- 172.21.15.49:0/789563075 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f9878003ba0 con 0x7f98840d0480 2023-07-18T03:41:23.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.701+0000 7f9872ffd700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f986406ae00 0x7f986406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.701+0000 7f9872ffd700 1 -- 172.21.15.49:0/789563075 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9878021070 con 0x7f98840d0480 2023-07-18T03:41:23.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.701+0000 7f988259c700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f986406ae00 0x7f986406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.702+0000 7f988259c700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f986406ae00 0x7f986406d2b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9874001710 tx=0x7f9874009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.705+0000 7f9872ffd700 1 -- 172.21.15.49:0/789563075 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f988409d6c0 con 0x7f98840d0480 2023-07-18T03:41:23.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.915+0000 7f9888fff700 1 -- 172.21.15.49:0/789563075 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f98840cf620 con 0x7f986406ae00 2023-07-18T03:41:23.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.916+0000 7f9872ffd700 1 -- 172.21.15.49:0/789563075 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f98740013d0 con 0x7f986406ae00 2023-07-18T03:41:23.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.918+0000 7f9888fff700 1 -- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f986406ae00 msgr2=0x7f986406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.918+0000 7f9888fff700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f986406ae00 0x7f986406d2b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9874001710 tx=0x7f9874009040 comp rx=0 tx=0).stop 2023-07-18T03:41:23.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.918+0000 7f9888fff700 1 -- 172.21.15.49:0/789563075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98840d0480 msgr2=0x7f98840cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.918+0000 7f9888fff700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98840d0480 0x7f98840cd440 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9878009b90 tx=0x7f9878004750 comp rx=0 tx=0).stop 2023-07-18T03:41:23.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.918+0000 7f9888fff700 1 -- 172.21.15.49:0/789563075 shutdown_connections 2023-07-18T03:41:23.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.918+0000 7f9888fff700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98840cd980 0x7f98841f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.918+0000 7f9888fff700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98840d0010 0x7f98840c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.918+0000 7f9888fff700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f986406ae00 0x7f986406d2b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.918+0000 7f9888fff700 1 --2- 172.21.15.49:0/789563075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98840d0480 0x7f98840cd440 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.918+0000 7f9888fff700 1 -- 172.21.15.49:0/789563075 >> 172.21.15.49:0/789563075 conn(0x7f988409c200 msgr2=0x7f98840d2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:23.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.918+0000 7f9888fff700 1 -- 172.21.15.49:0/789563075 shutdown_connections 2023-07-18T03:41:23.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:23.918+0000 7f9888fff700 1 -- 172.21.15.49:0/789563075 wait complete. 2023-07-18T03:41:23.916 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 7 to deep-scrub 2023-07-18T03:41:23.932 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:41:24.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.142+0000 7f3c46dff700 1 Processor -- start 2023-07-18T03:41:24.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.142+0000 7f3c46dff700 1 -- start start 2023-07-18T03:41:24.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.143+0000 7f3c46dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3c400b1e00 0x7f3c400b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.143+0000 7f3c46dff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c400d2ce0 con 0x7f3c400d1220 2023-07-18T03:41:24.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.143+0000 7f3c46dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c400b3350 con 0x7f3c4015c950 2023-07-18T03:41:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.143+0000 7f3c46dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c400b3630 con 0x7f3c400b1e00 2023-07-18T03:41:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.143+0000 7f3c3ffff700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3c400d1220 0x7f3c400afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44552/0 (socket says 172.21.15.49:44552) 2023-07-18T03:41:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.143+0000 7f3c3ffff700 1 -- 172.21.15.49:0/3611702932 learned_addr learned my addr 172.21.15.49:0/3611702932 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.144+0000 7f3c4539c700 1 --2- 172.21.15.49:0/3611702932 >> v2:172.21.15.107:3300/0 conn(0x7f3c400b1e00 0x7f3c400b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.144+0000 7f3c3f7fe700 1 -- 172.21.15.49:0/3611702932 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249627350 0 0) 0x7f3c400d2ce0 con 0x7f3c400d1220 2023-07-18T03:41:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.144+0000 7f3c3f7fe700 1 -- 172.21.15.49:0/3611702932 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c300035b0 con 0x7f3c400d1220 2023-07-18T03:41:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.144+0000 7f3c4539c700 1 -- 172.21.15.49:0/3611702932 >> v1:172.21.15.107:6789/0 conn(0x7f3c4015c950 legacy=0x7f3c400d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.144+0000 7f3c4539c700 1 -- 172.21.15.49:0/3611702932 >> v1:172.21.15.49:6789/0 conn(0x7f3c400d1220 legacy=0x7f3c400afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.144+0000 7f3c3f7fe700 1 -- 172.21.15.49:0/3611702932 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1681057541 0 0) 0x7f3c300035b0 con 0x7f3c400d1220 2023-07-18T03:41:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.144+0000 7f3c4539c700 1 -- 172.21.15.49:0/3611702932 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400b3910 con 0x7f3c400b1e00 2023-07-18T03:41:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.145+0000 7f3c4539c700 1 --2- 172.21.15.49:0/3611702932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c400b1e00 0x7f3c400b43f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009d90 tx=0x7f3c3800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2bd496a874b7cd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.145+0000 7f3c3f7fe700 1 -- 172.21.15.49:0/3611702932 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800e070 con 0x7f3c400b1e00 2023-07-18T03:41:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.145+0000 7f3c3f7fe700 1 -- 172.21.15.49:0/3611702932 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c38003a20 con 0x7f3c400b1e00 2023-07-18T03:41:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.145+0000 7f3c3f7fe700 1 -- 172.21.15.49:0/3611702932 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c001670 con 0x7f3c400b1e00 2023-07-18T03:41:24.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.145+0000 7f3c46dff700 1 -- 172.21.15.49:0/3611702932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c400b1e00 msgr2=0x7f3c400b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.145+0000 7f3c46dff700 1 --2- 172.21.15.49:0/3611702932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c400b1e00 0x7f3c400b43f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009d90 tx=0x7f3c3800b330 comp rx=0 tx=0).stop 2023-07-18T03:41:24.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.145+0000 7f3c46dff700 1 -- 172.21.15.49:0/3611702932 shutdown_connections 2023-07-18T03:41:24.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.145+0000 7f3c46dff700 1 --2- 172.21.15.49:0/3611702932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c400b1e00 0x7f3c400b43f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.145+0000 7f3c46dff700 1 -- 172.21.15.49:0/3611702932 >> 172.21.15.49:0/3611702932 conn(0x7f3c4009c1d0 msgr2=0x7f3c400d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:24.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.146+0000 7f3c46dff700 1 -- 172.21.15.49:0/3611702932 shutdown_connections 2023-07-18T03:41:24.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.146+0000 7f3c46dff700 1 -- 172.21.15.49:0/3611702932 wait complete. 2023-07-18T03:41:24.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.147+0000 7f3c46dff700 1 Processor -- start 2023-07-18T03:41:24.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.147+0000 7f3c46dff700 1 -- start start 2023-07-18T03:41:24.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.148+0000 7f3c46dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c400b1e00 0x7f3c400b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.148+0000 7f3c46dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c400d1220 0x7f3c40151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.148+0000 7f3c46dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c4015c950 0x7f3c400cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.148+0000 7f3c46dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400cd550 con 0x7f3c400d1220 2023-07-18T03:41:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.148+0000 7f3c46dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400cf950 con 0x7f3c4015c950 2023-07-18T03:41:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.148+0000 7f3c46dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400cfc30 con 0x7f3c400b1e00 2023-07-18T03:41:24.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.149+0000 7f3c3ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c400d1220 0x7f3c40151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.149+0000 7f3c3ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c400d1220 0x7f3c40151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47868/0 (socket says 172.21.15.49:47868) 2023-07-18T03:41:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.149+0000 7f3c3ffff700 1 -- 172.21.15.49:0/2352344012 learned_addr learned my addr 172.21.15.49:0/2352344012 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.149+0000 7f3c44b9b700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c400b1e00 0x7f3c400b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.149+0000 7f3c4539c700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c4015c950 0x7f3c400cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.149+0000 7f3c3ffff700 1 -- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c400b1e00 msgr2=0x7f3c400b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.149+0000 7f3c3ffff700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c400b1e00 0x7f3c400b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.149+0000 7f3c3ffff700 1 -- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c4015c950 msgr2=0x7f3c400cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.149+0000 7f3c3ffff700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c4015c950 0x7f3c400cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.149+0000 7f3c3ffff700 1 -- 172.21.15.49:0/2352344012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c38009a70 con 0x7f3c400d1220 2023-07-18T03:41:24.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.149+0000 7f3c3ffff700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c400d1220 0x7f3c40151290 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00c750 tx=0x7f3c2c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.150+0000 7f3c3d7fa700 1 -- 172.21.15.49:0/2352344012 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c004c80 con 0x7f3c400d1220 2023-07-18T03:41:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.150+0000 7f3c3d7fa700 1 -- 172.21.15.49:0/2352344012 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c004de0 con 0x7f3c400d1220 2023-07-18T03:41:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.150+0000 7f3c3d7fa700 1 -- 172.21.15.49:0/2352344012 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c007310 con 0x7f3c400d1220 2023-07-18T03:41:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.150+0000 7f3c46dff700 1 -- 172.21.15.49:0/2352344012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c400cff10 con 0x7f3c400d1220 2023-07-18T03:41:24.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.150+0000 7f3c46dff700 1 -- 172.21.15.49:0/2352344012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c400cf570 con 0x7f3c400d1220 2023-07-18T03:41:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.151+0000 7f3c46dff700 1 -- 172.21.15.49:0/2352344012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c400cd9e0 con 0x7f3c400d1220 2023-07-18T03:41:24.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.152+0000 7f3c3d7fa700 1 -- 172.21.15.49:0/2352344012 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f3c2c007470 con 0x7f3c400d1220 2023-07-18T03:41:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.153+0000 7f3c3d7fa700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3c2406ae00 0x7f3c2406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.153+0000 7f3c3d7fa700 1 -- 172.21.15.49:0/2352344012 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3c2c089040 con 0x7f3c400d1220 2023-07-18T03:41:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.154+0000 7f3c44b9b700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3c2406ae00 0x7f3c2406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.154+0000 7f3c44b9b700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3c2406ae00 0x7f3c2406d2b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3c401f5600 tx=0x7f3c34009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:24.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.157+0000 7f3c3d7fa700 1 -- 172.21.15.49:0/2352344012 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3c2c0bb330 con 0x7f3c400d1220 2023-07-18T03:41:24.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.361+0000 7f3c46dff700 1 -- 172.21.15.49:0/2352344012 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c400b3ed0 con 0x7f3c2406ae00 2023-07-18T03:41:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.364+0000 7f3c3d7fa700 1 -- 172.21.15.49:0/2352344012 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73168 (secure 0 0 0) 0x7f3c3400f410 con 0x7f3c2406ae00 2023-07-18T03:41:24.360 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:24.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.367+0000 7f3c46dff700 1 -- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3c2406ae00 msgr2=0x7f3c2406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.367+0000 7f3c46dff700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3c2406ae00 0x7f3c2406d2b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3c401f5600 tx=0x7f3c34009040 comp rx=0 tx=0).stop 2023-07-18T03:41:24.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.367+0000 7f3c46dff700 1 -- 172.21.15.49:0/2352344012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c400d1220 msgr2=0x7f3c40151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.367+0000 7f3c46dff700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c400d1220 0x7f3c40151290 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00c750 tx=0x7f3c2c00ca60 comp rx=0 tx=0).stop 2023-07-18T03:41:24.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.367+0000 7f3c46dff700 1 -- 172.21.15.49:0/2352344012 shutdown_connections 2023-07-18T03:41:24.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.367+0000 7f3c46dff700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c4015c950 0x7f3c400cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.367+0000 7f3c46dff700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c400b1e00 0x7f3c400b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.367+0000 7f3c46dff700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f3c2406ae00 0x7f3c2406d2b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.367+0000 7f3c46dff700 1 --2- 172.21.15.49:0/2352344012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c400d1220 0x7f3c40151290 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.367+0000 7f3c46dff700 1 -- 172.21.15.49:0/2352344012 >> 172.21.15.49:0/2352344012 conn(0x7f3c4009c1d0 msgr2=0x7f3c400b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:24.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.367+0000 7f3c46dff700 1 -- 172.21.15.49:0/2352344012 shutdown_connections 2023-07-18T03:41:24.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:24.367+0000 7f3c46dff700 1 -- 172.21.15.49:0/2352344012 wait complete. 2023-07-18T03:41:24.365 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:41:24.380 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2023-07-18T03:41:24.138251+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238152,"kb_used_data":6152,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754736568,"statfs":{"total":773094113280,"available":772850245632,"internally_reserved":0,"allocated":6299648,"data_stored":3410576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002653"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.013880+0000","last_change":"2023-07-18T03:38:29.335852+0000","last_active":"2023-07-18T03:41:12.013880+0000","last_peered":"2023-07-18T03:41:12.013880+0000","last_clean":"2023-07-18T03:41:12.013880+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:41:12.013880+0000","last_undegraded":"2023-07-18T03:41:12.013880+0000","last_fullsized":"2023-07-18T03:41:12.013880+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:22.119395+0000","last_change":"2023-07-18T03:41:22.119438+0000","last_active":"2023-07-18T03:41:22.119395+0000","last_peered":"2023-07-18T03:41:22.119395+0000","last_clean":"2023-07-18T03:41:22.119395+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:41:22.119395+0000","last_undegraded":"2023-07-18T03:41:22.119395+0000","last_fullsized":"2023-07-18T03:41:22.119395+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:41:22.119307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:41:22.119307+0000","last_clean_scrub_stamp":"2023-07-18T03:41:22.119307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:40:06.151271+0000","scrub_duration":0.0022310770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":85,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:20.292965+0000","last_change":"2023-07-18T03:41:20.293008+0000","last_active":"2023-07-18T03:41:20.292965+0000","last_peered":"2023-07-18T03:41:20.292965+0000","last_clean":"2023-07-18T03:41:20.292965+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:41:20.292965+0000","last_undegraded":"2023-07-18T03:41:20.292965+0000","last_fullsized":"2023-07-18T03:41:20.292965+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:41:20.292890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:41:20.292890+0000","last_clean_scrub_stamp":"2023-07-18T03:41:20.292890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:27:41.420910+0000","scrub_duration":0.0020818859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"18'117","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.012586+0000","last_change":"2023-07-18T03:38:29.342002+0000","last_active":"2023-07-18T03:41:12.012586+0000","last_peered":"2023-07-18T03:41:12.012586+0000","last_clean":"2023-07-18T03:41:12.012586+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:41:12.012586+0000","last_undegraded":"2023-07-18T03:41:12.012586+0000","last_fullsized":"2023-07-18T03:41:12.012586+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:19:41.650574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.015761+0000","last_change":"2023-07-18T03:38:29.943918+0000","last_active":"2023-07-18T03:41:12.015761+0000","last_peered":"2023-07-18T03:41:12.015761+0000","last_clean":"2023-07-18T03:41:12.015761+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:41:12.015761+0000","last_undegraded":"2023-07-18T03:41:12.015761+0000","last_fullsized":"2023-07-18T03:41:12.015761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:29:32.125624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.012398+0000","last_change":"2023-07-18T03:38:29.943894+0000","last_active":"2023-07-18T03:41:12.012398+0000","last_peered":"2023-07-18T03:41:12.012398+0000","last_clean":"2023-07-18T03:41:12.012398+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:41:12.012398+0000","last_undegraded":"2023-07-18T03:41:12.012398+0000","last_fullsized":"2023-07-18T03:41:12.012398+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:14:41.054472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:13.498078+0000","last_change":"2023-07-18T03:38:29.944031+0000","last_active":"2023-07-18T03:41:13.498078+0000","last_peered":"2023-07-18T03:41:13.498078+0000","last_clean":"2023-07-18T03:41:13.498078+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:41:13.498078+0000","last_undegraded":"2023-07-18T03:41:13.498078+0000","last_fullsized":"2023-07-18T03:41:13.498078+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:24:49.449121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.438393+0000","last_change":"2023-07-18T03:38:29.944261+0000","last_active":"2023-07-18T03:41:12.438393+0000","last_peered":"2023-07-18T03:41:12.438393+0000","last_clean":"2023-07-18T03:41:12.438393+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:41:12.438393+0000","last_undegraded":"2023-07-18T03:41:12.438393+0000","last_fullsized":"2023-07-18T03:41:12.438393+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:15:38.470063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:12.438614+0000","last_change":"2023-07-18T03:38:29.344973+0000","last_active":"2023-07-18T03:41:12.438614+0000","last_peered":"2023-07-18T03:41:12.438614+0000","last_clean":"2023-07-18T03:41:12.438614+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:41:12.438614+0000","last_undegraded":"2023-07-18T03:41:12.438614+0000","last_fullsized":"2023-07-18T03:41:12.438614+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:38:28.321101+0000","last_clean_scrub_stamp":"2023-07-18T03:38:28.321101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:04:44.231921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509478,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1232896,"data_stored":869098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85599999999999998,"5min":1.167,"15min":1.167},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.59999999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71999999999999997,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.873,"15min":0.89500000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.2570000000000001,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":0.74399999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.746,"15min":0.753},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1419999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0249999999999999,"5min":1.365,"15min":1.365},"last":0.623}]},{"osd":4,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.0010000000000003,"5min":4.0010000000000003,"15min":4.0010000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.72899999999999998,"15min":0.71199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.2989999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.77700000000000002,"15min":0.76000000000000001},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.72299999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:40:31 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.81699999999999995,"15min":0.80000000000000004},"min":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0609999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.65300000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.73799999999999999,"15min":0.754},"min":{"1min":0.51400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84699999999999998,"5min":2.427,"15min":2.427},"last":0.56399999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.84099999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90100000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60299999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95999999999999996,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.77200000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91300000000000003,"5min":1.034,"15min":1.034},"last":0.626},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96399999999999997,"5min":1.113,"15min":1.113},"last":0.64600000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90400000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55900000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.74399999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81000000000000005,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0820000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.81000000000000005}]}]},{"osd":3,"up_from":14,"seq":60129542183,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.432,"15min":0.441},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.317}]},{"osd":1,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.51700000000000002,"15min":0.52600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.58899999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46500000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.69999999999999996,"15min":0.72799999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.66100000000000003,"5min":3.956,"15min":3.956},"last":0.505},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.68400000000000005,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.755,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.36699999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.94399999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.86599999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.92900000000000005,"5min":1.22,"15min":1.22},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.88200000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91300000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90400000000000003,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:40:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.94599999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.79900000000000004,"15min":0.81000000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84899999999999998,"5min":1.272,"15min":1.272},"last":0.65200000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1232896,"data_stored":869098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64500000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.436},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.46100000000000002,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.372}]},{"osd":2,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.434,"15min":0.435},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.60099999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55700000000000005}]},{"osd":3,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.65700000000000003,"5min":0.748,"15min":0.748},"last":0.49099999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90400000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.59399999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:40:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":0.73799999999999999},"min":{"1min":0.57199999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84999999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.747,"15min":0.754},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89000000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88400000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.754,"15min":0.755},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.94999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.78300000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.61499999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.01,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92000000000000004,"5min":1.042,"15min":1.042},"last":0.76800000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.47499999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.50800000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89400000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.54400000000000004}]},{"osd":3,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.81699999999999995,"5min":1.032,"15min":1.032},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.64300000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.70399999999999996,"15min":0.71599999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.751}]},{"osd":7,"last update":"Tue Jul 18 03:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92700000000000005,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92200000000000004,"5min":0.999,"15min":0.999},"last":0.77900000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542182,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.67500000000000004,"15min":0.63700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.58799999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.74099999999999999,"15min":0.68700000000000006},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":8.3780000000000001,"5min":8.3780000000000001,"15min":8.3780000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.74299999999999999,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.56699999999999995}]},{"osd":2,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.77200000000000002,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":4.8769999999999998,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.505}]},{"osd":3,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70499999999999996,"15min":0.69499999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73799999999999999,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.65500000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.755,"15min":0.73899999999999999},"min":{"1min":0.41199999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.9690000000000003,"5min":4.9690000000000003,"15min":4.9690000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.40200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.6829999999999998,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.621}]},{"osd":6,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71399999999999997,"15min":0.71899999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90800000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.83899999999999997,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.3140000000000001,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":0.72399999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:40:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.80300000000000005,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.437},"max":{"1min":3.7389999999999999,"5min":3.7389999999999999,"15min":3.7389999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94799999999999995,"5min":1.472,"15min":1.472},"last":0.70099999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.47499999999999998,"15min":0.48899999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":1.389,"15min":1.389},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.64500000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.48299999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.84099999999999997,"15min":0.90300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.74199999999999999,"5min":8.3040000000000003,"15min":8.3040000000000003},"last":0.377},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.66700000000000004,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.42199999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.77300000000000002,"15min":0.81899999999999995},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63100000000000001,"5min":8.0229999999999997,"15min":8.0229999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.88,"15min":0.93799999999999994},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.70999999999999996,"5min":8.359,"15min":8.359},"last":0.52200000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97699999999999998,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78400000000000003,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0129999999999999,"5min":1.429,"15min":1.429},"last":0.68899999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.117,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.754,"15min":0.755},"min":{"1min":0.623,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0920000000000001,"5min":1.266,"15min":1.266},"last":0.69699999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95199999999999996,"5min":1.752,"15min":1.752},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.626,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0449999999999999,"5min":1.454,"15min":1.454},"last":0.71399999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:40:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.58499999999999996,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.94299999999999995,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0249999999999999,"5min":1.222,"15min":1.222},"last":0.74399999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509477,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.503,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.498,"15min":0.497},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72499999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.61599999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.76200000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.622,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.52700000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.56499999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78700000000000003,"15min":0.78800000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0580000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.038,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.749}]},{"osd":7,"last update":"Tue Jul 18 03:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.60799999999999998,"5min":0.504,"15min":0.504},"max":{"1min":1.0209999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77400000000000002,"15min":0.77600000000000002},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:41:24.384 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-07-18T03:38:28.321101+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=38, tm_sec=28, tm_wday=1, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=41, tm_sec=17, tm_wday=1, tm_yday=199, tm_isdst=0) 2023-07-18T03:41:24.384 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-07-18T03:38:28.321101+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=38, tm_sec=28, tm_wday=1, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=41, tm_sec=17, tm_wday=1, tm_yday=199, tm_isdst=0) 2023-07-18T03:41:24.385 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-07-18T03:38:28.321101+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=38, tm_sec=28, tm_wday=1, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=41, tm_sec=17, tm_wday=1, tm_yday=199, tm_isdst=0) 2023-07-18T03:41:24.385 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-07-18T03:38:28.321101+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=38, tm_sec=28, tm_wday=1, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=41, tm_sec=17, tm_wday=1, tm_yday=199, tm_isdst=0) 2023-07-18T03:41:24.385 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-07-18T03:38:28.321101+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=38, tm_sec=28, tm_wday=1, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=41, tm_sec=17, tm_wday=1, tm_yday=199, tm_isdst=0) 2023-07-18T03:41:24.385 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-07-18T03:38:28.321101+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=38, tm_sec=28, tm_wday=1, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=41, tm_sec=17, tm_wday=1, tm_yday=199, tm_isdst=0) 2023-07-18T03:41:24.386 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-07-18T03:38:28.321101+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=38, tm_sec=28, tm_wday=1, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=18, tm_hour=3, tm_min=41, tm_sec=17, tm_wday=1, tm_yday=199, tm_isdst=0) 2023-07-18T03:41:24.386 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-07-18T03:41:30.926 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:41:30.933 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:41:44.387 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:41:44.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.609+0000 7efde49ff700 1 Processor -- start 2023-07-18T03:41:44.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.610+0000 7efde49ff700 1 -- start start 2023-07-18T03:41:44.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.611+0000 7efde49ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efde00d1220 0x7efde00b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:44.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.611+0000 7efde49ff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde00b7f50 con 0x7efde00b4510 2023-07-18T03:41:44.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.611+0000 7efde49ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde00cdca0 con 0x7efde00b5d90 2023-07-18T03:41:44.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.611+0000 7efde49ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde00b48e0 con 0x7efde00d1220 2023-07-18T03:41:44.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.611+0000 7efdde59c700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7efde00b4510 0x7efde00cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49682/0 (socket says 172.21.15.49:49682) 2023-07-18T03:41:44.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.611+0000 7efdde59c700 1 -- 172.21.15.49:0/2578072531 learned_addr learned my addr 172.21.15.49:0/2578072531 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:44.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.611+0000 7efdded9d700 1 --2- 172.21.15.49:0/2578072531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efde00d1220 0x7efde00b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:44.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.611+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23288884 0 0) 0x7efde00b7f50 con 0x7efde00b4510 2023-07-18T03:41:44.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdcc0035b0 con 0x7efde00b4510 2023-07-18T03:41:44.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599518521 0 0) 0x7efde00cdca0 con 0x7efde00b5d90 2023-07-18T03:41:44.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efde00b7f50 con 0x7efde00b5d90 2023-07-18T03:41:44.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1030516290 0 0) 0x7efdcc0035b0 con 0x7efde00b4510 2023-07-18T03:41:44.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efde00d1220 msgr2=0x7efde00b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:44.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 --2- 172.21.15.49:0/2578072531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efde00d1220 0x7efde00b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:44.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 >> v1:172.21.15.107:6789/0 conn(0x7efde00b5d90 legacy=0x7efde00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:44.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde00b7340 con 0x7efde00b4510 2023-07-18T03:41:44.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 627419376 0 0) 0x7efdd00017f0 con 0x7efde00b4510 2023-07-18T03:41:44.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 >> v1:172.21.15.49:6789/0 conn(0x7efde00b4510 legacy=0x7efde00cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:44.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 --2- 172.21.15.49:0/2578072531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdcc003bc0 0x7efdcc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 --2- 172.21.15.49:0/2578072531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efde00d1220 0x7efdcc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 --2- 172.21.15.49:0/2578072531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdcc004d90 0x7efdcc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc00bba0 con 0x7efdcc003bc0 2023-07-18T03:41:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc00bd10 con 0x7efdcc004d90 2023-07-18T03:41:44.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc00bed0 con 0x7efde00d1220 2023-07-18T03:41:44.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdde59c700 1 --2- 172.21.15.49:0/2578072531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdcc003bc0 0x7efdcc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:44.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdd7fff700 1 --2- 172.21.15.49:0/2578072531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efde00d1220 0x7efdcc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:44.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.612+0000 7efdded9d700 1 --2- 172.21.15.49:0/2578072531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdcc004d90 0x7efdcc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:44.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.613+0000 7efdde59c700 1 -- 172.21.15.49:0/2578072531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efde00d1220 msgr2=0x7efdcc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:44.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.613+0000 7efdde59c700 1 --2- 172.21.15.49:0/2578072531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efde00d1220 0x7efdcc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:44.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.613+0000 7efdde59c700 1 -- 172.21.15.49:0/2578072531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdcc004d90 msgr2=0x7efdcc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:44.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.613+0000 7efdde59c700 1 --2- 172.21.15.49:0/2578072531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdcc004d90 0x7efdcc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:44.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.613+0000 7efdde59c700 1 -- 172.21.15.49:0/2578072531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc0035b0 con 0x7efdcc003bc0 2023-07-18T03:41:44.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.613+0000 7efdde59c700 1 --2- 172.21.15.49:0/2578072531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdcc003bc0 0x7efdcc003fb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7efdd000c240 tx=0x7efdcc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14c21b215428618 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:44.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.613+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd000cd80 con 0x7efdcc003bc0 2023-07-18T03:41:44.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.613+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd0007b70 con 0x7efdcc003bc0 2023-07-18T03:41:44.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.613+0000 7efdddd9b700 1 -- 172.21.15.49:0/2578072531 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0007310 con 0x7efdcc003bc0 2023-07-18T03:41:44.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.614+0000 7efde49ff700 1 -- 172.21.15.49:0/2578072531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdcc003bc0 msgr2=0x7efdcc003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:44.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.614+0000 7efde49ff700 1 --2- 172.21.15.49:0/2578072531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdcc003bc0 0x7efdcc003fb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7efdd000c240 tx=0x7efdcc005880 comp rx=0 tx=0).stop 2023-07-18T03:41:44.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.614+0000 7efdded9d700 1 -- 172.21.15.49:0/2578072531 reap_dead start 2023-07-18T03:41:44.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.614+0000 7efde49ff700 1 -- 172.21.15.49:0/2578072531 shutdown_connections 2023-07-18T03:41:44.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.614+0000 7efde49ff700 1 -- 172.21.15.49:0/2578072531 >> 172.21.15.49:0/2578072531 conn(0x7efde009c1d0 msgr2=0x7efde00d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:44.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.614+0000 7efde49ff700 1 -- 172.21.15.49:0/2578072531 shutdown_connections 2023-07-18T03:41:44.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.614+0000 7efde49ff700 1 -- 172.21.15.49:0/2578072531 wait complete. 2023-07-18T03:41:44.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.617+0000 7efde49ff700 1 Processor -- start 2023-07-18T03:41:44.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.617+0000 7efde49ff700 1 -- start start 2023-07-18T03:41:44.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.618+0000 7efde49ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdcc003bc0 0x7efde00cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:44.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.618+0000 7efde49ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0154cd0 0x7efde00c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:44.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.618+0000 7efde49ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde00c9140 0x7efde00ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:44.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.618+0000 7efde49ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde00c95b0 con 0x7efde00c9140 2023-07-18T03:41:44.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.618+0000 7efde49ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde01e4900 con 0x7efde0154cd0 2023-07-18T03:41:44.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.618+0000 7efde49ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde01e4ba0 con 0x7efdcc003bc0 2023-07-18T03:41:44.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.618+0000 7efdded9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde00c9140 0x7efde00ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:44.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.618+0000 7efdde59c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdcc003bc0 0x7efde00cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:44.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.618+0000 7efdded9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde00c9140 0x7efde00ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39868/0 (socket says 172.21.15.49:39868) 2023-07-18T03:41:44.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.618+0000 7efdded9d700 1 -- 172.21.15.49:0/3972727245 learned_addr learned my addr 172.21.15.49:0/3972727245 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:44.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.618+0000 7efdd7fff700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0154cd0 0x7efde00c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:44.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.619+0000 7efdded9d700 1 -- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdcc003bc0 msgr2=0x7efde00cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:44.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.619+0000 7efdded9d700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdcc003bc0 0x7efde00cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:44.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.619+0000 7efdded9d700 1 -- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0154cd0 msgr2=0x7efde00c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:44.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.619+0000 7efdded9d700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0154cd0 0x7efde00c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:44.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.619+0000 7efdded9d700 1 -- 172.21.15.49:0/3972727245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde01e4e80 con 0x7efde00c9140 2023-07-18T03:41:44.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.619+0000 7efdded9d700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde00c9140 0x7efde00ca010 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efde00d0a40 tx=0x7efdd8011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:44.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.619+0000 7efdd6ffd700 1 -- 172.21.15.49:0/3972727245 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd800e040 con 0x7efde00c9140 2023-07-18T03:41:44.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.619+0000 7efdd6ffd700 1 -- 172.21.15.49:0/3972727245 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd800f040 con 0x7efde00c9140 2023-07-18T03:41:44.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.619+0000 7efdd6ffd700 1 -- 172.21.15.49:0/3972727245 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd800fc40 con 0x7efde00c9140 2023-07-18T03:41:44.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.619+0000 7efde49ff700 1 -- 172.21.15.49:0/3972727245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde01e5160 con 0x7efde00c9140 2023-07-18T03:41:44.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.620+0000 7efde49ff700 1 -- 172.21.15.49:0/3972727245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde01e5720 con 0x7efde00c9140 2023-07-18T03:41:44.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.620+0000 7efde49ff700 1 -- 172.21.15.49:0/3972727245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde004e280 con 0x7efde00c9140 2023-07-18T03:41:44.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.621+0000 7efdd6ffd700 1 -- 172.21.15.49:0/3972727245 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7efdd8015430 con 0x7efde00c9140 2023-07-18T03:41:44.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.622+0000 7efdd6ffd700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7efdc006ae00 0x7efdc006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:44.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.622+0000 7efdd6ffd700 1 -- 172.21.15.49:0/3972727245 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7efdd8089810 con 0x7efde00c9140 2023-07-18T03:41:44.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.623+0000 7efdde59c700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7efdc006ae00 0x7efdc006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:44.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.624+0000 7efdde59c700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7efdc006ae00 0x7efdc006d2b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efdd0000ef0 tx=0x7efdd000a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:44.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.627+0000 7efdd6ffd700 1 -- 172.21.15.49:0/3972727245 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7efdd8058440 con 0x7efde00c9140 2023-07-18T03:41:44.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.831+0000 7efde49ff700 1 -- 172.21.15.49:0/3972727245 --> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efde00b7900 con 0x7efdc006ae00 2023-07-18T03:41:44.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.834+0000 7efdd6ffd700 1 -- 172.21.15.49:0/3972727245 <== mgr.4100 v2:172.21.15.107:6832/2972169254 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73007 (secure 0 0 0) 0x7efdd0022070 con 0x7efdc006ae00 2023-07-18T03:41:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:44.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.836+0000 7efde49ff700 1 -- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7efdc006ae00 msgr2=0x7efdc006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:44.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.836+0000 7efde49ff700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7efdc006ae00 0x7efdc006d2b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efdd0000ef0 tx=0x7efdd000a000 comp rx=0 tx=0).stop 2023-07-18T03:41:44.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.836+0000 7efde49ff700 1 -- 172.21.15.49:0/3972727245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde00c9140 msgr2=0x7efde00ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.836+0000 7efde49ff700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde00c9140 0x7efde00ca010 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efde00d0a40 tx=0x7efdd8011ed0 comp rx=0 tx=0).stop 2023-07-18T03:41:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.836+0000 7efde49ff700 1 -- 172.21.15.49:0/3972727245 shutdown_connections 2023-07-18T03:41:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.836+0000 7efde49ff700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7efdc006ae00 0x7efdc006d2b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:44.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.836+0000 7efde49ff700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efde00c9140 0x7efde00ca010 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:44.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.836+0000 7efde49ff700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efdcc003bc0 0x7efde00cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:44.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.836+0000 7efde49ff700 1 --2- 172.21.15.49:0/3972727245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0154cd0 0x7efde00c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:44.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.836+0000 7efde49ff700 1 -- 172.21.15.49:0/3972727245 >> 172.21.15.49:0/3972727245 conn(0x7efde009c1d0 msgr2=0x7efde00cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:44.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.837+0000 7efde49ff700 1 -- 172.21.15.49:0/3972727245 shutdown_connections 2023-07-18T03:41:44.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:44.837+0000 7efde49ff700 1 -- 172.21.15.49:0/3972727245 wait complete. 2023-07-18T03:41:44.837 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-07-18T03:41:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2023-07-18T03:41:44.142709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238152,"kb_used_data":6152,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754736568,"statfs":{"total":773094113280,"available":772850245632,"internally_reserved":0,"allocated":6299648,"data_stored":3410576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002532"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:27.044429+0000","last_change":"2023-07-18T03:41:27.044479+0000","last_active":"2023-07-18T03:41:27.044429+0000","last_peered":"2023-07-18T03:41:27.044429+0000","last_clean":"2023-07-18T03:41:27.044429+0000","last_became_active":"2023-07-18T03:38:29.335619+0000","last_became_peered":"2023-07-18T03:38:29.335619+0000","last_unstale":"2023-07-18T03:41:27.044429+0000","last_undegraded":"2023-07-18T03:41:27.044429+0000","last_fullsized":"2023-07-18T03:41:27.044429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:41:27.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:41:27.044323+0000","last_clean_scrub_stamp":"2023-07-18T03:41:27.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:06:16.975955+0000","scrub_duration":0.0020823310000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:22.119395+0000","last_change":"2023-07-18T03:41:22.119438+0000","last_active":"2023-07-18T03:41:22.119395+0000","last_peered":"2023-07-18T03:41:22.119395+0000","last_clean":"2023-07-18T03:41:22.119395+0000","last_became_active":"2023-07-18T03:38:29.341202+0000","last_became_peered":"2023-07-18T03:38:29.341202+0000","last_unstale":"2023-07-18T03:41:22.119395+0000","last_undegraded":"2023-07-18T03:41:22.119395+0000","last_fullsized":"2023-07-18T03:41:22.119395+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:41:22.119307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:41:22.119307+0000","last_clean_scrub_stamp":"2023-07-18T03:41:22.119307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:40:06.151271+0000","scrub_duration":0.0022310770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":85,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:20.292965+0000","last_change":"2023-07-18T03:41:20.293008+0000","last_active":"2023-07-18T03:41:20.292965+0000","last_peered":"2023-07-18T03:41:20.292965+0000","last_clean":"2023-07-18T03:41:20.292965+0000","last_became_active":"2023-07-18T03:38:29.345132+0000","last_became_peered":"2023-07-18T03:38:29.345132+0000","last_unstale":"2023-07-18T03:41:20.292965+0000","last_undegraded":"2023-07-18T03:41:20.292965+0000","last_fullsized":"2023-07-18T03:41:20.292965+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:41:20.292890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:41:20.292890+0000","last_clean_scrub_stamp":"2023-07-18T03:41:20.292890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:27:41.420910+0000","scrub_duration":0.0020818859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'119","reported_seq":241,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:26.621898+0000","last_change":"2023-07-18T03:41:26.621961+0000","last_active":"2023-07-18T03:41:26.621898+0000","last_peered":"2023-07-18T03:41:26.621898+0000","last_clean":"2023-07-18T03:41:26.621898+0000","last_became_active":"2023-07-18T03:38:29.340774+0000","last_became_peered":"2023-07-18T03:38:29.340774+0000","last_unstale":"2023-07-18T03:41:26.621898+0000","last_undegraded":"2023-07-18T03:41:26.621898+0000","last_fullsized":"2023-07-18T03:41:26.621898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'119","last_scrub_stamp":"2023-07-18T03:41:26.621808+0000","last_deep_scrub":"54'119","last_deep_scrub_stamp":"2023-07-18T03:41:26.621808+0000","last_clean_scrub_stamp":"2023-07-18T03:41:26.621808+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:55:24.460461+0000","scrub_duration":0.0060523419999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":89,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:32.528131+0000","last_change":"2023-07-18T03:41:32.528184+0000","last_active":"2023-07-18T03:41:32.528131+0000","last_peered":"2023-07-18T03:41:32.528131+0000","last_clean":"2023-07-18T03:41:32.528131+0000","last_became_active":"2023-07-18T03:38:29.943597+0000","last_became_peered":"2023-07-18T03:38:29.943597+0000","last_unstale":"2023-07-18T03:41:32.528131+0000","last_undegraded":"2023-07-18T03:41:32.528131+0000","last_fullsized":"2023-07-18T03:41:32.528131+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:41:32.528008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:41:32.528008+0000","last_clean_scrub_stamp":"2023-07-18T03:41:32.528008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:17:19.321911+0000","scrub_duration":0.0020717909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:29.547444+0000","last_change":"2023-07-18T03:41:29.547504+0000","last_active":"2023-07-18T03:41:29.547444+0000","last_peered":"2023-07-18T03:41:29.547444+0000","last_clean":"2023-07-18T03:41:29.547444+0000","last_became_active":"2023-07-18T03:38:29.943600+0000","last_became_peered":"2023-07-18T03:38:29.943600+0000","last_unstale":"2023-07-18T03:41:29.547444+0000","last_undegraded":"2023-07-18T03:41:29.547444+0000","last_fullsized":"2023-07-18T03:41:29.547444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:41:29.547341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:41:29.547341+0000","last_clean_scrub_stamp":"2023-07-18T03:41:29.547341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:46:46.374416+0000","scrub_duration":0.001909577,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:22.681672+0000","last_change":"2023-07-18T03:41:22.681745+0000","last_active":"2023-07-18T03:41:22.681672+0000","last_peered":"2023-07-18T03:41:22.681672+0000","last_clean":"2023-07-18T03:41:22.681672+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:41:22.681672+0000","last_undegraded":"2023-07-18T03:41:22.681672+0000","last_fullsized":"2023-07-18T03:41:22.681672+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:41:22.681538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:41:22.681538+0000","last_clean_scrub_stamp":"2023-07-18T03:41:22.681538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:35:28.510976+0000","scrub_duration":0.0019983169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":85,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:27.081367+0000","last_change":"2023-07-18T03:41:27.081433+0000","last_active":"2023-07-18T03:41:27.081367+0000","last_peered":"2023-07-18T03:41:27.081367+0000","last_clean":"2023-07-18T03:41:27.081367+0000","last_became_active":"2023-07-18T03:38:29.943718+0000","last_became_peered":"2023-07-18T03:38:29.943718+0000","last_unstale":"2023-07-18T03:41:27.081367+0000","last_undegraded":"2023-07-18T03:41:27.081367+0000","last_fullsized":"2023-07-18T03:41:27.081367+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:41:27.081234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:41:27.081234+0000","last_clean_scrub_stamp":"2023-07-18T03:41:27.081234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:21:17.858523+0000","scrub_duration":0.001907251,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-07-18T03:41:24.079793+0000","last_change":"2023-07-18T03:41:24.079841+0000","last_active":"2023-07-18T03:41:24.079793+0000","last_peered":"2023-07-18T03:41:24.079793+0000","last_clean":"2023-07-18T03:41:24.079793+0000","last_became_active":"2023-07-18T03:38:29.344658+0000","last_became_peered":"2023-07-18T03:38:29.344658+0000","last_unstale":"2023-07-18T03:41:24.079793+0000","last_undegraded":"2023-07-18T03:41:24.079793+0000","last_fullsized":"2023-07-18T03:41:24.079793+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:41:24.079696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:41:24.079696+0000","last_clean_scrub_stamp":"2023-07-18T03:41:24.079696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:27:45.207715+0000","scrub_duration":0.0016291599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1232896,"data_stored":869098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80100000000000005,"5min":1.167,"15min":1.167},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.71999999999999997,"15min":0.72799999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.77900000000000003,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.82199999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86199999999999999,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.48899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88900000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.69599999999999995}]},{"osd":2,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68899999999999995,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.82699999999999996,"15min":0.88},"min":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85199999999999998,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72599999999999998,"15min":0.746},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90600000000000003,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.77600000000000002,"15min":0.79000000000000004},"min":{"1min":0.58199999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.90700000000000003,"5min":1.365,"15min":1.365},"last":0.80600000000000005}]},{"osd":4,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.68200000000000005,"15min":0.66600000000000004},"min":{"1min":0.42099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.141,"5min":4.0010000000000003,"15min":4.0010000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.028,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.56299999999999994}]},{"osd":5,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.76400000000000001,"15min":0.75600000000000001},"min":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0489999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.75800000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80300000000000005,"15min":0.79500000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.93100000000000005,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77500000000000002,"15min":0.78800000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.78200000000000003}]}]},{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88800000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.71299999999999997,"15min":0.746},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0349999999999999,"5min":2.427,"15min":2.427},"last":0.56899999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66600000000000004,"15min":0.66400000000000003},"min":{"1min":0.47699999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86399999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.64000000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.66100000000000003},"min":{"1min":0.45700000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.99099999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.751},"min":{"1min":0.54300000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.91400000000000003,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.59099999999999997}]},{"osd":3,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68600000000000005,"15min":0.69299999999999995},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.748,"15min":0.76000000000000001},"min":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.89800000000000002,"5min":1.113,"15min":1.113},"last":0.77300000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.46999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86399999999999999,"5min":1.014,"15min":1.014},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67600000000000005,"15min":0.67500000000000004},"min":{"1min":0.44900000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.52400000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72299999999999998,"15min":0.72399999999999998},"min":{"1min":0.48699999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.499,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:41:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.498,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91100000000000003,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.501},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.79500000000000004,"15min":0.80700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1240000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.74199999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542187,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42999999999999999,"15min":0.44},"min":{"1min":0.28999999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.623,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.317,"5min":0.315,"15min":0.315},"max":{"1min":0.60999999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.52300000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.436,"15min":0.432},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.441,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69299999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61299999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.66400000000000003,"15min":0.71599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.82499999999999996,"5min":3.956,"15min":3.956},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.64700000000000002,"15min":0.69699999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.99299999999999999,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.59999999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.754,"15min":0.76700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.873,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73799999999999999,"15min":0.75700000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.96999999999999997,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.86499999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76300000000000001,"15min":0.77900000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.89400000000000002,"5min":1.22,"15min":1.22},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.90900000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":1.0289999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.75700000000000001},"min":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95199999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.77700000000000002,"15min":0.78700000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.91600000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.96799999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:41:33 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80100000000000005,"15min":0.82099999999999995},"min":{"1min":0.57199999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.94299999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.78600000000000003,"15min":0.80600000000000005},"min":{"1min":0.57599999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94399999999999995,"5min":1.272,"15min":1.272},"last":0.90800000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1232896,"data_stored":869098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:41:35 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.57899999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.311,"5min":0.307,"15min":0.307},"max":{"1min":0.625,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.54700000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:41:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.438,"15min":0.436},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84199999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Tue Jul 18 03:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77800000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.5,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.57899999999999996,"5min":0.748,"15min":0.748},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72499999999999998,"15min":0.74299999999999999},"min":{"1min":0.51400000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90200000000000002,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.56499999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93300000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.85699999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73299999999999998,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92800000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.74099999999999999,"15min":0.752},"min":{"1min":0.59899999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.84599999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.875}]},{"osd":6,"last update":"Tue Jul 18 03:41:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.61199999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.89500000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.753},"min":{"1min":0.58299999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.88100000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.89700000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:41:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.60799999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.95699999999999996,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.77200000000000002,"15min":0.78300000000000003},"min":{"1min":0.57099999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97299999999999998,"5min":1.042,"15min":1.042},"last":0.70899999999999996}]}]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:41:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67700000000000005,"15min":0.66600000000000004},"min":{"1min":0.47999999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.93000000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.46400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91600000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82399999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.80700000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:41:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70499999999999996,"15min":0.71399999999999997},"min":{"1min":0.52100000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93200000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.86199999999999999,"5min":0.997,"15min":0.997},"last":0.90600000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88100000000000001,"5min":1.032,"15min":1.032},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72799999999999998,"15min":0.73299999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.95599999999999996,"5min":1.042,"15min":1.042},"last":0.876}]},{"osd":5,"last update":"Tue Jul 18 03:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.49299999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84699999999999998,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.39900000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86499999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.78700000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:41:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68300000000000005,"15min":0.68999999999999995},"min":{"1min":0.51400000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81999999999999995,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.752,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.87,"5min":1.113,"15min":1.113},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:41:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88500000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.59499999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.873,"5min":0.999,"15min":0.999},"last":0.753}]}]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.67200000000000004,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.433,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90400000000000003,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.72499999999999998,"15min":0.68200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81799999999999995,"5min":8.3780000000000001,"15min":8.3780000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73899999999999999,"15min":0.70799999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90400000000000003,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.61699999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.76000000000000001,"15min":0.73499999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92600000000000005,"5min":4.8769999999999998,"15min":4.8769999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.505,"5min":0.44,"15min":0.44},"max":{"1min":0.85899999999999999,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.71499999999999997}]},{"osd":3,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70799999999999996,"15min":0.69599999999999995},"min":{"1min":0.56399999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91100000000000003,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.52600000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88700000000000001,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.42199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95999999999999996,"5min":4.9690000000000003,"15min":4.9690000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.747,"15min":0.751},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85999999999999999,"5min":3.6829999999999998,"15min":3.6829999999999998},"last":0.75900000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69999999999999996,"15min":0.71399999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86499999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.82499999999999996,"15min":0.80900000000000005},"min":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98699999999999999,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":0.70299999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.47999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.86699999999999999,"5min":3.7389999999999999,"15min":3.7389999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.77900000000000003,"15min":0.80300000000000005},"min":{"1min":0.59899999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88200000000000001,"5min":1.472,"15min":1.472},"last":0.81799999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509484,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46600000000000003,"15min":0.48599999999999999},"min":{"1min":0.27600000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.83499999999999996,"5min":1.389,"15min":1.389},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.85699999999999998,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.754,"15min":0.874},"min":{"1min":0.34399999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89100000000000001,"5min":8.3040000000000003,"15min":8.3040000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54300000000000004,"15min":0.54800000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84899999999999998,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.55400000000000005}]},{"osd":3,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.71099999999999997,"15min":0.79900000000000004},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1000000000000001,"5min":8.0229999999999997,"15min":8.0229999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.80600000000000005,"15min":0.91400000000000003},"min":{"1min":0.46600000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0489999999999999,"5min":8.359,"15min":8.359},"last":0.68999999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97499999999999998,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77300000000000002,"15min":0.78900000000000003},"min":{"1min":0.51000000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94399999999999995,"5min":1.429,"15min":1.429},"last":1.0329999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76100000000000001,"15min":0.76500000000000001},"min":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0449999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.754,"15min":0.755},"min":{"1min":0.60099999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.022,"5min":1.266,"15min":1.266},"last":0.82299999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80800000000000005,"15min":0.83199999999999996},"min":{"1min":0.625,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.999,"5min":1.752,"15min":1.752},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.79300000000000004,"15min":0.80700000000000005},"min":{"1min":0.57199999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.038,"5min":1.454,"15min":1.454},"last":0.85699999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:41:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.80200000000000005,"15min":0.82099999999999995},"min":{"1min":0.58299999999999996,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0649999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.006},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.60099999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0589999999999999,"5min":1.222,"15min":1.222},"last":0.871}]}]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":638976,"data_stored":278730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:41:35 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47499999999999998,"15min":0.46300000000000002},"min":{"1min":0.32100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61799999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:41:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.751,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.497,"15min":0.497},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75800000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.72699999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:41:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45000000000000001,"15min":0.44400000000000001},"min":{"1min":0.38,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76200000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.66700000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:41:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.74299999999999999,"15min":0.73599999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.006,"5min":1.05,"15min":1.05},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.89900000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.79300000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:41:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71499999999999997,"15min":0.72199999999999998},"min":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.94599999999999995,"5min":1.034,"15min":1.034},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.56999999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.087,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.74199999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:41:35 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.78300000000000003,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.872},{"interface":"front","average":{"1min":0.751,"5min":0.78500000000000003,"15min":0.78800000000000003},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0780000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:41:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.77600000000000002,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.0449999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.81899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:41:44.856 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-07-18T03:41:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.060+0000 7efe7abff700 1 Processor -- start 2023-07-18T03:41:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.060+0000 7efe7abff700 1 -- start start 2023-07-18T03:41:45.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.061+0000 7efe7abff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efe740d0b10 0x7efe740d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.061+0000 7efe7abff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7efe740b0e20 0x7efe740b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.061+0000 7efe7abff700 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe740b8e30 con 0x7efe740b8a60 2023-07-18T03:41:45.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.061+0000 7efe7abff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe740d14d0 con 0x7efe740d0b10 2023-07-18T03:41:45.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.061+0000 7efe7abff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe740afaa0 con 0x7efe740b0e20 2023-07-18T03:41:45.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.061+0000 7efe73fff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efe740d0b10 0x7efe740d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.062+0000 7efe7899b700 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7efe740b8a60 0x7efe740b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49688/0 (socket says 172.21.15.49:49688) 2023-07-18T03:41:45.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.062+0000 7efe7899b700 1 -- 172.21.15.49:0/3528752682 learned_addr learned my addr 172.21.15.49:0/3528752682 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-07-18T03:41:45.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.062+0000 7efe7919c700 1 --2- 172.21.15.49:0/3528752682 >> v2:172.21.15.107:3300/0 conn(0x7efe740b0e20 0x7efe740b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.062+0000 7efe73fff700 1 -- 172.21.15.49:0/3528752682 >> v2:172.21.15.107:3300/0 conn(0x7efe740b0e20 msgr2=0x7efe740b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:45.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.062+0000 7efe73fff700 1 --2- 172.21.15.49:0/3528752682 >> v2:172.21.15.107:3300/0 conn(0x7efe740b0e20 0x7efe740b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.062+0000 7efe73fff700 1 -- 172.21.15.49:0/3528752682 >> v1:172.21.15.49:6789/0 conn(0x7efe740b8a60 legacy=0x7efe740b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:45.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.062+0000 7efe73fff700 1 -- 172.21.15.49:0/3528752682 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe740afd80 con 0x7efe740d0b10 2023-07-18T03:41:45.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.062+0000 7efe737fe700 1 -- 172.21.15.49:0/3528752682 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999056159 0 0) 0x7efe740b8e30 con 0x7efe740b8a60 2023-07-18T03:41:45.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.062+0000 7efe73fff700 1 --2- 172.21.15.49:0/3528752682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe740d0b10 0x7efe740d0f90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efe740674e0 tx=0x7efe60009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31acd85e4eaf9d12 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:45.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.062+0000 7efe737fe700 1 -- 172.21.15.49:0/3528752682 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60011830 con 0x7efe740d0b10 2023-07-18T03:41:45.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.063+0000 7efe737fe700 1 -- 172.21.15.49:0/3528752682 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe60011990 con 0x7efe740d0b10 2023-07-18T03:41:45.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.063+0000 7efe737fe700 1 -- 172.21.15.49:0/3528752682 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60011d20 con 0x7efe740d0b10 2023-07-18T03:41:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.063+0000 7efe7abff700 1 -- 172.21.15.49:0/3528752682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe740d0b10 msgr2=0x7efe740d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.063+0000 7efe7abff700 1 --2- 172.21.15.49:0/3528752682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe740d0b10 0x7efe740d0f90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efe740674e0 tx=0x7efe60009b20 comp rx=0 tx=0).stop 2023-07-18T03:41:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.063+0000 7efe7abff700 1 -- 172.21.15.49:0/3528752682 shutdown_connections 2023-07-18T03:41:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.063+0000 7efe7abff700 1 --2- 172.21.15.49:0/3528752682 >> v2:172.21.15.107:3300/0 conn(0x7efe740b0e20 0x7efe740b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.063+0000 7efe7abff700 1 --2- 172.21.15.49:0/3528752682 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe740d0b10 0x7efe740d0f90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.063+0000 7efe7abff700 1 -- 172.21.15.49:0/3528752682 >> 172.21.15.49:0/3528752682 conn(0x7efe7409c1b0 msgr2=0x7efe740d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:45.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.063+0000 7efe7abff700 1 -- 172.21.15.49:0/3528752682 shutdown_connections 2023-07-18T03:41:45.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.063+0000 7efe7abff700 1 -- 172.21.15.49:0/3528752682 wait complete. 2023-07-18T03:41:45.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.066+0000 7efe7abff700 1 Processor -- start 2023-07-18T03:41:45.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.066+0000 7efe7abff700 1 -- start start 2023-07-18T03:41:45.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.067+0000 7efe7abff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe740b0e20 0x7efe740cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.067+0000 7efe7abff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe740b8a60 0x7efe741f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.067+0000 7efe7abff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe740d0b10 0x7efe74158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.067+0000 7efe7abff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe74158a70 con 0x7efe740b0e20 2023-07-18T03:41:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.067+0000 7efe7abff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe740d0290 con 0x7efe740d0b10 2023-07-18T03:41:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.067+0000 7efe7abff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7415d2f0 con 0x7efe740b8a60 2023-07-18T03:41:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.067+0000 7efe7899b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe740b0e20 0x7efe740cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.067+0000 7efe7899b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe740b0e20 0x7efe740cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39896/0 (socket says 172.21.15.49:39896) 2023-07-18T03:41:45.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe7899b700 1 -- 172.21.15.49:0/4214182529 learned_addr learned my addr 172.21.15.49:0/4214182529 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe7919c700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe740d0b10 0x7efe74158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe73fff700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe740b8a60 0x7efe741f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe7899b700 1 -- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe740b8a60 msgr2=0x7efe741f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe7899b700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe740b8a60 0x7efe741f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe7899b700 1 -- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe740d0b10 msgr2=0x7efe74158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe7899b700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe740d0b10 0x7efe74158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe7899b700 1 -- 172.21.15.49:0/4214182529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe600094a0 con 0x7efe740b0e20 2023-07-18T03:41:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe7899b700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe740b0e20 0x7efe740cfb90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efe680011e0 tx=0x7efe6800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe717fa700 1 -- 172.21.15.49:0/4214182529 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68004c80 con 0x7efe740b0e20 2023-07-18T03:41:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe717fa700 1 -- 172.21.15.49:0/4214182529 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe68004e00 con 0x7efe740b0e20 2023-07-18T03:41:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe717fa700 1 -- 172.21.15.49:0/4214182529 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68007310 con 0x7efe740b0e20 2023-07-18T03:41:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.068+0000 7efe7abff700 1 -- 172.21.15.49:0/4214182529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7415d5d0 con 0x7efe740b0e20 2023-07-18T03:41:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.069+0000 7efe7abff700 1 -- 172.21.15.49:0/4214182529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7415db90 con 0x7efe740b0e20 2023-07-18T03:41:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.069+0000 7efe7abff700 1 -- 172.21.15.49:0/4214182529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe74074350 con 0x7efe740b0e20 2023-07-18T03:41:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.070+0000 7efe717fa700 1 -- 172.21.15.49:0/4214182529 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7efe68007470 con 0x7efe740b0e20 2023-07-18T03:41:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.072+0000 7efe717fa700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7efe5806ae00 0x7efe5806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.072+0000 7efe717fa700 1 -- 172.21.15.49:0/4214182529 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7efe68089120 con 0x7efe740b0e20 2023-07-18T03:41:45.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.072+0000 7efe73fff700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7efe5806ae00 0x7efe5806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.073+0000 7efe73fff700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7efe5806ae00 0x7efe5806d2b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efe600090b0 tx=0x7efe6001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:45.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.077+0000 7efe717fa700 1 -- 172.21.15.49:0/4214182529 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7efe68054340 con 0x7efe740b0e20 2023-07-18T03:41:45.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.285+0000 7efe7abff700 1 -- 172.21.15.49:0/4214182529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7efe740b5270 con 0x7efe740b0e20 2023-07-18T03:41:45.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.289+0000 7efe717fa700 1 -- 172.21.15.49:0/4214182529 <== mon.0 v2:172.21.15.49:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7efe68007720 con 0x7efe740b0e20 2023-07-18T03:41:45.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.290+0000 7efe717fa700 1 -- 172.21.15.49:0/4214182529 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7efe68057960 con 0x7efe740b0e20 2023-07-18T03:41:45.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.292+0000 7efe7abff700 1 -- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7efe5806ae00 msgr2=0x7efe5806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:45.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.292+0000 7efe7abff700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7efe5806ae00 0x7efe5806d2b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efe600090b0 tx=0x7efe6001c040 comp rx=0 tx=0).stop 2023-07-18T03:41:45.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.293+0000 7efe7abff700 1 -- 172.21.15.49:0/4214182529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe740b0e20 msgr2=0x7efe740cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:45.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.293+0000 7efe7abff700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe740b0e20 0x7efe740cfb90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efe680011e0 tx=0x7efe6800cae0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.293+0000 7efe7abff700 1 -- 172.21.15.49:0/4214182529 shutdown_connections 2023-07-18T03:41:45.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.293+0000 7efe7abff700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe740d0b10 0x7efe74158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.293+0000 7efe7abff700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7efe5806ae00 0x7efe5806d2b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.293+0000 7efe7abff700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe740b0e20 0x7efe740cfb90 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.293+0000 7efe7abff700 1 --2- 172.21.15.49:0/4214182529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe740b8a60 0x7efe741f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.293+0000 7efe7abff700 1 -- 172.21.15.49:0/4214182529 >> 172.21.15.49:0/4214182529 conn(0x7efe7409c1b0 msgr2=0x7efe740b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:45.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.293+0000 7efe7abff700 1 -- 172.21.15.49:0/4214182529 shutdown_connections 2023-07-18T03:41:45.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.293+0000 7efe7abff700 1 -- 172.21.15.49:0/4214182529 wait complete. 2023-07-18T03:41:45.304 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-18T03:41:45.305 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:41:45.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.516+0000 7f139ebff700 1 Processor -- start 2023-07-18T03:41:45.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.516+0000 7f139ebff700 1 -- start start 2023-07-18T03:41:45.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.517+0000 7f139ebff700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f139815e890 0x7f13980cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.517+0000 7f139ebff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f13980b3780 0x7f13980b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.517+0000 7f139ebff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f139815a8c0 0x7f139815ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.517+0000 7f139ebff700 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13980cf380 con 0x7f139815e890 2023-07-18T03:41:45.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.517+0000 7f139ebff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13980b4130 con 0x7f13980b3780 2023-07-18T03:41:45.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.517+0000 7f139ebff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13980ce110 con 0x7f139815a8c0 2023-07-18T03:41:45.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.518+0000 7f139c99b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f139815e890 0x7f13980cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.518+0000 7f139c99b700 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f139815e890 0x7f13980cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39910/0 (socket says 172.21.15.49:39910) 2023-07-18T03:41:45.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.518+0000 7f139c99b700 1 -- 172.21.15.49:0/2386611070 learned_addr learned my addr 172.21.15.49:0/2386611070 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:45.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.518+0000 7f139d19c700 1 --2- 172.21.15.49:0/2386611070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f139815a8c0 0x7f139815ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.518+0000 7f1397fff700 1 --2- 172.21.15.49:0/2386611070 >> v2:172.21.15.107:3300/0 conn(0x7f13980b3780 0x7f13980b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.518+0000 7f139c99b700 1 -- 172.21.15.49:0/2386611070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f139815a8c0 msgr2=0x7f139815ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:45.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.518+0000 7f139c99b700 1 --2- 172.21.15.49:0/2386611070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f139815a8c0 0x7f139815ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.518+0000 7f139c99b700 1 -- 172.21.15.49:0/2386611070 >> v2:172.21.15.107:3300/0 conn(0x7f13980b3780 msgr2=0x7f13980b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:45.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.518+0000 7f139c99b700 1 --2- 172.21.15.49:0/2386611070 >> v2:172.21.15.107:3300/0 conn(0x7f13980b3780 0x7f13980b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.518+0000 7f139c99b700 1 -- 172.21.15.49:0/2386611070 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139815b660 con 0x7f139815e890 2023-07-18T03:41:45.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.518+0000 7f139c99b700 1 --2- 172.21.15.49:0/2386611070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f139815e890 0x7f13980cee40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f139806b730 tx=0x7f138c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cc95cef3d00813 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:45.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.519+0000 7f13977fe700 1 -- 172.21.15.49:0/2386611070 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c00b860 con 0x7f139815e890 2023-07-18T03:41:45.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.519+0000 7f13977fe700 1 -- 172.21.15.49:0/2386611070 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f138c00b9c0 con 0x7f139815e890 2023-07-18T03:41:45.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.519+0000 7f13977fe700 1 -- 172.21.15.49:0/2386611070 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c00bd70 con 0x7f139815e890 2023-07-18T03:41:45.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.519+0000 7f139ebff700 1 -- 172.21.15.49:0/2386611070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f139815e890 msgr2=0x7f13980cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:45.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.519+0000 7f139ebff700 1 --2- 172.21.15.49:0/2386611070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f139815e890 0x7f13980cee40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f139806b730 tx=0x7f138c009c00 comp rx=0 tx=0).stop 2023-07-18T03:41:45.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.519+0000 7f139ebff700 1 -- 172.21.15.49:0/2386611070 shutdown_connections 2023-07-18T03:41:45.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.519+0000 7f139ebff700 1 --2- 172.21.15.49:0/2386611070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f139815a8c0 0x7f139815ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.519+0000 7f139ebff700 1 --2- 172.21.15.49:0/2386611070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f139815e890 0x7f13980cee40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.519+0000 7f139ebff700 1 --2- 172.21.15.49:0/2386611070 >> v2:172.21.15.107:3300/0 conn(0x7f13980b3780 0x7f13980b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.519+0000 7f139ebff700 1 -- 172.21.15.49:0/2386611070 >> 172.21.15.49:0/2386611070 conn(0x7f1398069f30 msgr2=0x7f139806a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:45.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.520+0000 7f139ebff700 1 -- 172.21.15.49:0/2386611070 shutdown_connections 2023-07-18T03:41:45.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.520+0000 7f139ebff700 1 -- 172.21.15.49:0/2386611070 wait complete. 2023-07-18T03:41:45.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.521+0000 7f139ebff700 1 Processor -- start 2023-07-18T03:41:45.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.522+0000 7f139ebff700 1 -- start start 2023-07-18T03:41:45.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.522+0000 7f139ebff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13980b3780 0x7f13981f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.522+0000 7f139ebff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f139815a8c0 0x7f1398148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f139ebff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f139815e890 0x7f1398158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f139ebff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13981496f0 con 0x7f139815a8c0 2023-07-18T03:41:45.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f139ebff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13981efdd0 con 0x7f139815e890 2023-07-18T03:41:45.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f139ebff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13981ef4a0 con 0x7f13980b3780 2023-07-18T03:41:45.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f1397fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f139815a8c0 0x7f1398148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f1397fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f139815a8c0 0x7f1398148da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39922/0 (socket says 172.21.15.49:39922) 2023-07-18T03:41:45.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f1397fff700 1 -- 172.21.15.49:0/456009026 learned_addr learned my addr 172.21.15.49:0/456009026 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-07-18T03:41:45.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f139c99b700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13980b3780 0x7f13981f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f139d19c700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f139815e890 0x7f1398158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f1397fff700 1 -- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13980b3780 msgr2=0x7f13981f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:45.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f1397fff700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13980b3780 0x7f13981f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f1397fff700 1 -- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f139815e890 msgr2=0x7f1398158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:45.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f1397fff700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f139815e890 0x7f1398158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f1397fff700 1 -- 172.21.15.49:0/456009026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c009550 con 0x7f139815a8c0 2023-07-18T03:41:45.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.523+0000 7f139d19c700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f139815e890 0x7f1398158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:45.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.524+0000 7f1397fff700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f139815a8c0 0x7f1398148da0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f138400d830 tx=0x7f138400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:45.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.524+0000 7f13957fa700 1 -- 172.21.15.49:0/456009026 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840125f0 con 0x7f139815a8c0 2023-07-18T03:41:45.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.524+0000 7f13957fa700 1 -- 172.21.15.49:0/456009026 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1384012750 con 0x7f139815a8c0 2023-07-18T03:41:45.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.524+0000 7f13957fa700 1 -- 172.21.15.49:0/456009026 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384012b20 con 0x7f139815a8c0 2023-07-18T03:41:45.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.524+0000 7f139ebff700 1 -- 172.21.15.49:0/456009026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13981ef780 con 0x7f139815a8c0 2023-07-18T03:41:45.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.524+0000 7f139ebff700 1 -- 172.21.15.49:0/456009026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13981f2250 con 0x7f139815a8c0 2023-07-18T03:41:45.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.525+0000 7f139ebff700 1 -- 172.21.15.49:0/456009026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13981eff60 con 0x7f139815a8c0 2023-07-18T03:41:45.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.526+0000 7f13957fa700 1 -- 172.21.15.49:0/456009026 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) v1 ==== 89139+0+0 (secure 0 0 0) 0x7f1384012c80 con 0x7f139815a8c0 2023-07-18T03:41:45.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.527+0000 7f13957fa700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f137c06ae00 0x7f137c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:45.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.527+0000 7f13957fa700 1 -- 172.21.15.49:0/456009026 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f13840897a0 con 0x7f139815a8c0 2023-07-18T03:41:45.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.528+0000 7f139c99b700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f137c06ae00 0x7f137c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:45.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.528+0000 7f139c99b700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f137c06ae00 0x7f137c06d2b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f138c0098f0 tx=0x7f138c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:45.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.531+0000 7f13957fa700 1 -- 172.21.15.49:0/456009026 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f13840549c0 con 0x7f139815a8c0 2023-07-18T03:41:45.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.783+0000 7f139ebff700 1 -- 172.21.15.49:0/456009026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f139804e280 con 0x7f139815a8c0 2023-07-18T03:41:45.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.783+0000 7f13957fa700 1 -- 172.21.15.49:0/456009026 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f1384057fe0 con 0x7f139815a8c0 2023-07-18T03:41:45.778 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-07-18T03:41:45.778 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":31,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/777524677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":777524677},{"type":"v1","addr":"172.21.15.107:6837","nonce":777524677}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/2382452558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2382452558},{"type":"v1","addr":"172.21.15.49:6834","nonce":2382452558}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/2843143219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2843143219},{"type":"v1","addr":"172.21.15.107:6836","nonce":2843143219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":6081,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2706178529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":2706178529},{"type":"v1","addr":"172.21.15.49:6835","nonce":2706178529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[]} 2023-07-18T03:41:45.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.785+0000 7f139ebff700 1 -- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f137c06ae00 msgr2=0x7f137c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:45.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.785+0000 7f139ebff700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f137c06ae00 0x7f137c06d2b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f138c0098f0 tx=0x7f138c008030 comp rx=0 tx=0).stop 2023-07-18T03:41:45.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.785+0000 7f139ebff700 1 -- 172.21.15.49:0/456009026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f139815a8c0 msgr2=0x7f1398148da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:45.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.785+0000 7f139ebff700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f139815a8c0 0x7f1398148da0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f138400d830 tx=0x7f138400dbf0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.786+0000 7f139ebff700 1 -- 172.21.15.49:0/456009026 shutdown_connections 2023-07-18T03:41:45.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.786+0000 7f139ebff700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f139815e890 0x7f1398158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.786+0000 7f139ebff700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13980b3780 0x7f13981f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.786+0000 7f139ebff700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.107:6832/2972169254,v1:172.21.15.107:6833/2972169254] conn(0x7f137c06ae00 0x7f137c06d2b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.786+0000 7f139ebff700 1 --2- 172.21.15.49:0/456009026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f139815a8c0 0x7f1398148da0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.786+0000 7f139ebff700 1 -- 172.21.15.49:0/456009026 >> 172.21.15.49:0/456009026 conn(0x7f1398069f30 msgr2=0x7f13980cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.786+0000 7f139ebff700 1 -- 172.21.15.49:0/456009026 shutdown_connections 2023-07-18T03:41:45.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-07-18T03:41:45.786+0000 7f139ebff700 1 -- 172.21.15.49:0/456009026 wait complete. 2023-07-18T03:41:45.783 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 31 2023-07-18T03:41:45.798 DEBUG:tasks.cephfs.filesystem:already dead... 2023-07-18T03:41:45.799 INFO:teuthology.misc:Shutting down mds daemons... 2023-07-18T03:41:45.799 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-18T03:41:45.799 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:45.812 INFO:tasks.ceph.mds.a:Stopped 2023-07-18T03:41:45.813 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-18T03:41:45.813 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:45.867 INFO:tasks.ceph.mds.c:Stopped 2023-07-18T03:41:45.868 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-18T03:41:45.869 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:45.885 INFO:tasks.ceph.mds.b:Stopped 2023-07-18T03:41:45.885 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-18T03:41:45.886 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:45.942 INFO:tasks.ceph.mds.d:Stopped 2023-07-18T03:41:45.942 INFO:teuthology.misc:Shutting down osd daemons... 2023-07-18T03:41:45.943 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-07-18T03:41:45.943 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:45.981 INFO:tasks.ceph.osd.0:Stopped 2023-07-18T03:41:45.981 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-07-18T03:41:45.982 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:46.064 INFO:tasks.ceph.osd.1:Stopped 2023-07-18T03:41:46.065 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-07-18T03:41:46.065 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:46.107 INFO:tasks.ceph.osd.2:Stopped 2023-07-18T03:41:46.108 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-07-18T03:41:46.109 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:46.163 INFO:tasks.ceph.osd.3:Stopped 2023-07-18T03:41:46.163 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-07-18T03:41:46.163 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:46.210 INFO:tasks.ceph.osd.4:Stopped 2023-07-18T03:41:46.211 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-07-18T03:41:46.214 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:46.256 INFO:tasks.ceph.osd.5:Stopped 2023-07-18T03:41:46.257 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-07-18T03:41:46.257 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:46.297 INFO:tasks.ceph.osd.6:Stopped 2023-07-18T03:41:46.298 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-07-18T03:41:46.298 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:46.348 INFO:tasks.ceph.osd.7:Stopped 2023-07-18T03:41:46.348 INFO:teuthology.misc:Shutting down mgr daemons... 2023-07-18T03:41:46.349 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-07-18T03:41:46.349 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:46.408 INFO:tasks.ceph.mgr.y:Stopped 2023-07-18T03:41:46.409 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-07-18T03:41:46.409 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:46.473 INFO:tasks.ceph.mgr.x:Stopped 2023-07-18T03:41:46.473 INFO:teuthology.misc:Shutting down mon daemons... 2023-07-18T03:41:46.474 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-07-18T03:41:46.474 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:46.494 INFO:tasks.ceph.mon.a:Stopped 2023-07-18T03:41:46.495 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-07-18T03:41:46.495 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:46.513 INFO:tasks.ceph.mon.b:Stopped 2023-07-18T03:41:46.514 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-07-18T03:41:46.514 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:41:46.571 INFO:tasks.ceph.mon.c:Stopped 2023-07-18T03:41:46.572 INFO:tasks.ceph:Checking cluster log for badness... 2023-07-18T03:41:46.572 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-07-18T03:41:46.664 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:41:46.665 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-07-18T03:41:47.008 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:41:47.009 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-07-18T03:41:47.199 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:41:47.200 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-07-18T03:41:47.424 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:41:47.425 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-07-18T03:41:47.625 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:41:47.626 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-07-18T03:41:48.017 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:41:48.019 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-07-18T03:41:48.220 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:41:48.220 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-07-18T03:41:48.434 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:41:48.434 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-07-18T03:41:48.670 INFO:tasks.ceph:Archiving mon data... 2023-07-18T03:41:48.672 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747/data/mon.a.tgz 2023-07-18T03:41:48.672 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2023-07-18T03:41:48.721 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.HVIGajieBN 2023-07-18T03:41:48.721 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.HVIGajieBN 2023-07-18T03:41:48.885 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.HVIGajieBN 2023-07-18T03:41:49.001 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.HVIGajieBN is 400KB 2023-07-18T03:41:49.099 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.HVIGajieBN 2023-07-18T03:41:49.143 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747/data/mon.b.tgz 2023-07-18T03:41:49.143 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2023-07-18T03:41:49.187 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.AtZ9Z1Rdqu 2023-07-18T03:41:49.188 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.AtZ9Z1Rdqu 2023-07-18T03:41:49.402 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.AtZ9Z1Rdqu 2023-07-18T03:41:49.515 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.AtZ9Z1Rdqu is 527KB 2023-07-18T03:41:49.612 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.AtZ9Z1Rdqu 2023-07-18T03:41:49.655 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747/data/mon.c.tgz 2023-07-18T03:41:49.656 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2023-07-18T03:41:49.738 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.O2wxtbBqyO 2023-07-18T03:41:49.738 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.O2wxtbBqyO 2023-07-18T03:41:49.943 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.O2wxtbBqyO 2023-07-18T03:41:50.056 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.O2wxtbBqyO is 527KB 2023-07-18T03:41:50.155 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.O2wxtbBqyO 2023-07-18T03:41:50.198 INFO:tasks.ceph:Cleaning ceph cluster... 2023-07-18T03:41:50.198 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-18T03:41:50.203 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-18T03:41:50.305 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-07-18T03:41:50.306 DEBUG:teuthology.orchestra.run.smithi049:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-18T03:41:50.311 DEBUG:teuthology.orchestra.run.smithi107:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-18T03:41:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-18T03:41:50.392 INFO:teuthology.orchestra.run.smithi107.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-18T03:41:50.408 INFO:tasks.ceph:Archiving crash dumps... 2023-07-18T03:41:50.410 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747/remote/smithi049/crash 2023-07-18T03:41:50.411 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-18T03:41:50.482 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747/remote/smithi107/crash 2023-07-18T03:41:50.483 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-18T03:41:50.550 INFO:tasks.ceph:Shutting down logrotate 2023-07-18T03:41:50.550 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:41:50.555 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-18T03:41:50.678 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-18T03:41:50.748 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-18T03:41:50.815 INFO:tasks.ceph:Compressing logs... 2023-07-18T03:41:50.815 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-18T03:41:50.820 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-18T03:41:50.885 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115681.log 2023-07-18T03:41:50.886 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109122.log 2023-07-18T03:41:50.886 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130425.log 2023-07-18T03:41:50.887 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107696.log 2023-07-18T03:41:50.888 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-07-18T03:41:50.888 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109122.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115681.log.gz 2023-07-18T03:41:50.889 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108724.log 2023-07-18T03:41:50.889 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109122.log.gz 2023-07-18T03:41:50.890 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107696.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107696.log.gz 2023-07-18T03:41:50.890 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121180.log 2023-07-18T03:41:50.891 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108724.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108724.log.gz 2023-07-18T03:41:50.891 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130236.log 2023-07-18T03:41:50.891 INFO:teuthology.orchestra.run.smithi107.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.130425.log.gz 2023-07-18T03:41:50.891 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-18T03:41:50.891 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130236.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130236.log.gz 2023-07-18T03:41:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115000.log 2023-07-18T03:41:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121180.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121180.log.gz 2023-07-18T03:41:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111997.log 2023-07-18T03:41:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122340.log 2023-07-18T03:41:50.894 INFO:teuthology.orchestra.run.smithi049.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115000.log.gz 2023-07-18T03:41:50.894 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130472.log 2023-07-18T03:41:50.894 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-07-18T03:41:50.894 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130472.log: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-07-18T03:41:50.895 INFO:teuthology.orchestra.run.smithi107.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130472.log.gz 2023-07-18T03:41:50.896 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130327.log 2023-07-18T03:41:50.896 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130190.log 2023-07-18T03:41:50.897 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130327.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130327.log.gz 2023-07-18T03:41:50.897 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130380.log 2023-07-18T03:41:50.897 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111997.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111997.log.gz 2023-07-18T03:41:50.898 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114678.log 2023-07-18T03:41:50.898 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122340.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122340.log.gz 2023-07-18T03:41:50.898 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113151.log 2023-07-18T03:41:50.898 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114678.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114678.log.gz 2023-07-18T03:41:50.899 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111537.log 2023-07-18T03:41:50.899 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113151.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.113151.log.gz 2023-07-18T03:41:50.899 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120316.log 2023-07-18T03:41:50.899 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111537.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.111537.log.gz-5 2023-07-18T03:41:50.899 INFO:teuthology.orchestra.run.smithi049.stderr:--verbose -- /var/log/ceph/ceph-client.admin.118569.log 2023-07-18T03:41:50.901 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120316.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120316.log.gz 2023-07-18T03:41:50.901 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111721.log 2023-07-18T03:41:50.902 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118569.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118569.log.gz 2023-07-18T03:41:50.902 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115046.log 2023-07-18T03:41:50.902 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111721.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111721.log.gz 2023-07-18T03:41:50.902 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122156.log 2023-07-18T03:41:50.902 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130190.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.130190.log.gz 2023-07-18T03:41:50.903 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-07-18T03:41:50.903 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130380.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130380.log.gz 2023-07-18T03:41:50.903 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-18T03:41:50.903 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.139047.log 2023-07-18T03:41:50.903 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-07-18T03:41:50.905 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.1.139047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.139047.log.gz 2023-07-18T03:41:50.905 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.137034.log 2023-07-18T03:41:50.905 INFO:teuthology.orchestra.run.smithi107.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-18T03:41:50.906 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.137660.log 2023-07-18T03:41:50.906 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.1.137034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.137034.log.gz 2023-07-18T03:41:50.906 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-07-18T03:41:50.907 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115046.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115046.log.gz 2023-07-18T03:41:50.907 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107992.log 2023-07-18T03:41:50.907 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122156.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122156.log.gz 2023-07-18T03:41:50.908 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116658.log 2023-07-18T03:41:50.908 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107992.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.107992.log.gz 2023-07-18T03:41:50.908 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114862.log 2023-07-18T03:41:50.908 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116658.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.116658.log.gz 2023-07-18T03:41:50.908 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119821.log 2023-07-18T03:41:50.909 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114862.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114862.log.gz 2023-07-18T03:41:50.909 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122624.log 2023-07-18T03:41:50.909 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119821.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119821.log.gz 2023-07-18T03:41:50.909 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114494.log 2023-07-18T03:41:50.910 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.3.137660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.137660.log.gz 2023-07-18T03:41:50.911 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-07-18T03:41:50.911 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130282.log 2023-07-18T03:41:50.911 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-07-18T03:41:50.912 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122624.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122624.log.gzgzip 2023-07-18T03:41:50.912 INFO:teuthology.orchestra.run.smithi049.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.109307.log 2023-07-18T03:41:50.912 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114494.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114494.log.gz 2023-07-18T03:41:50.913 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111306.log 2023-07-18T03:41:50.913 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109307.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109307.log.gz 2023-07-18T03:41:50.913 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116275.log 2023-07-18T03:41:50.913 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111306.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.111306.log.gz 2023-07-18T03:41:50.913 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120270.log 2023-07-18T03:41:50.914 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116275.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116275.log.gz 2023-07-18T03:41:50.914 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113996.log 2023-07-18T03:41:50.915 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130282.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.130282.log.gz 2023-07-18T03:41:50.915 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130144.log 2023-07-18T03:41:50.916 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120270.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120270.log.gz 2023-07-18T03:41:50.916 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-07-18T03:41:50.916 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113996.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113996.log.gz 2023-07-18T03:41:50.916 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119729.log 2023-07-18T03:41:50.917 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113515.log 2023-07-18T03:41:50.917 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119729.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.119729.log.gz 2023-07-18T03:41:50.917 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111813.log 2023-07-18T03:41:50.918 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113515.log: 89.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.113515.log.gz--verbose 2023-07-18T03:41:50.918 INFO:teuthology.orchestra.run.smithi049.stderr:-- /var/log/ceph/ceph-client.admin.106901.log 2023-07-18T03:41:50.918 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111813.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111813.log.gz 2023-07-18T03:41:50.919 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120950.log 2023-07-18T03:41:50.919 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.106901.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106901.log.gz 2023-07-18T03:41:50.920 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119181.log 2023-07-18T03:41:50.921 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120950.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120950.log.gz 2023-07-18T03:41:50.921 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107994.log 2023-07-18T03:41:50.922 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119181.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119181.log.gz 2023-07-18T03:41:50.922 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113002.log 2023-07-18T03:41:50.923 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107994.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107994.log.gz 2023-07-18T03:41:50.923 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.admin.130144.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130144.log.gz 2023-07-18T03:41:50.924 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-07-18T03:41:50.924 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120408.log 2023-07-18T03:41:50.925 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113002.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113002.log.gz 2023-07-18T03:41:50.925 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113197.log 2023-07-18T03:41:50.925 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120408.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120408.log.gz 2023-07-18T03:41:50.926 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113812.log 2023-07-18T03:41:50.927 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113197.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113197.log.gz 2023-07-18T03:41:50.927 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-18T03:41:50.928 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122202.log 2023-07-18T03:41:50.929 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113812.log.gz 2023-07-18T03:41:50.929 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116413.log 2023-07-18T03:41:50.930 INFO:teuthology.orchestra.run.smithi049.stderr: 85.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-07-18T03:41:50.931 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122202.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122202.log.gz 2023-07-18T03:41:50.931 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109675.log 2023-07-18T03:41:50.932 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116413.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116413.log.gz 2023-07-18T03:41:50.932 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120747.log 2023-07-18T03:41:50.933 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109675.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109675.log.gz 2023-07-18T03:41:50.933 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108362.log 2023-07-18T03:41:50.934 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120747.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120747.log.gz 2023-07-18T03:41:50.935 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-07-18T03:41:50.935 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108363.log 2023-07-18T03:41:50.935 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108362.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108362.log.gz 2023-07-18T03:41:50.936 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116842.log 2023-07-18T03:41:50.937 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108363.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108363.log.gz 2023-07-18T03:41:50.937 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108353.log 2023-07-18T03:41:50.938 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116842.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116842.log.gz 2023-07-18T03:41:50.938 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116505.log 2023-07-18T03:41:50.939 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108353.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108353.log.gz 2023-07-18T03:41:50.939 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107985.log 2023-07-18T03:41:50.940 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116505.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.116505.log.gz 2023-07-18T03:41:50.940 INFO:teuthology.orchestra.run.smithi107.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-07-18T03:41:50.941 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107650.log 2023-07-18T03:41:50.941 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107985.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107985.log.gz 2023-07-18T03:41:50.941 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121972.log 2023-07-18T03:41:50.942 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107650.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107650.log.gz 2023-07-18T03:41:50.942 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120163.log 2023-07-18T03:41:50.943 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121972.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.121972.log.gz 2023-07-18T03:41:50.943 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108732.log 2023-07-18T03:41:50.944 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120163.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.120163.log.gz 2023-07-18T03:41:50.945 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mds.b.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-07-18T03:41:50.945 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.138448.log 2023-07-18T03:41:50.946 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112497.log 2023-07-18T03:41:50.946 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108732.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108732.log.gz 2023-07-18T03:41:50.946 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106579.log 2023-07-18T03:41:50.947 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112497.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112497.log.gz 2023-07-18T03:41:50.947 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116550.log 2023-07-18T03:41:50.947 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.106579.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106579.log.gz 2023-07-18T03:41:50.948 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108938.log 2023-07-18T03:41:50.948 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.1.138448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.138448.log.gz 2023-07-18T03:41:50.949 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116550.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116550.log.gz 2023-07-18T03:41:50.949 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120996.log 2023-07-18T03:41:50.950 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108938.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108938.log.gz 2023-07-18T03:41:50.950 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107282.log 2023-07-18T03:41:50.951 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120996.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120996.log.gz 2023-07-18T03:41:50.951 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121088.log 2023-07-18T03:41:50.952 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107282.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107282.log.gz 2023-07-18T03:41:50.952 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117896.log 2023-07-18T03:41:50.953 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121088.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.121088.log.gz 2023-07-18T03:41:50.953 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112089.log 2023-07-18T03:41:50.954 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117896.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117896.log.gz 2023-07-18T03:41:50.954 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116321.log 2023-07-18T03:41:50.955 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112089.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112089.log.gz 2023-07-18T03:41:50.955 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-07-18T03:41:50.956 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116321.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116321.log.gz 2023-07-18T03:41:50.956 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107989.log 2023-07-18T03:41:50.957 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107988.log 2023-07-18T03:41:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107989.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107989.log.gz 2023-07-18T03:41:50.959 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115635.log 2023-07-18T03:41:50.960 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107988.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107988.log.gz 2023-07-18T03:41:50.960 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111859.log 2023-07-18T03:41:50.961 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115635.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115635.log.gz 2023-07-18T03:41:50.962 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115773.log 2023-07-18T03:41:50.962 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111859.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111859.log.gz 2023-07-18T03:41:50.962 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111951.log 2023-07-18T03:41:50.963 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115773.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115773.log.gz 2023-07-18T03:41:50.964 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106516.log 2023-07-18T03:41:50.964 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111951.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111951.log.gz 2023-07-18T03:41:50.965 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109629.log 2023-07-18T03:41:50.966 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.106516.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106516.log.gz 2023-07-18T03:41:50.966 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119227.log 2023-07-18T03:41:50.967 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109629.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.109629.log.gz 2023-07-18T03:41:50.967 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117022.log 2023-07-18T03:41:50.968 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119227.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119227.log.gz 2023-07-18T03:41:50.968 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112864.log 2023-07-18T03:41:50.969 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117022.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117022.log.gz 2023-07-18T03:41:50.970 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118345.log 2023-07-18T03:41:50.970 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112864.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.112864.log.gz 2023-07-18T03:41:50.971 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-07-18T03:41:50.972 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118345.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118345.log.gz 2023-07-18T03:41:50.972 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120689.log 2023-07-18T03:41:50.974 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117279.log 2023-07-18T03:41:50.974 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120689.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-07-18T03:41:50.975 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120689.log.gz 2023-07-18T03:41:50.975 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122808.log 2023-07-18T03:41:50.976 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117279.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117279.log.gz 2023-07-18T03:41:50.977 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116750.log 2023-07-18T03:41:50.977 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122808.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122808.log.gz 2023-07-18T03:41:50.978 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114318.log 2023-07-18T03:41:50.979 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116750.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.116750.log.gz 2023-07-18T03:41:50.979 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108984.log 2023-07-18T03:41:50.980 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114318.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114318.log.gz 2023-07-18T03:41:50.981 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113423.log 2023-07-18T03:41:50.981 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108984.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108984.log.gz 2023-07-18T03:41:50.982 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122920.log 2023-07-18T03:41:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113423.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113423.log.gz 2023-07-18T03:41:50.983 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112638.log 2023-07-18T03:41:50.984 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122920.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122920.log.gz 2023-07-18T03:41:50.984 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116459.log 2023-07-18T03:41:50.985 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112638.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112638.log.gz 2023-07-18T03:41:50.986 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106948.log 2023-07-18T03:41:50.986 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116459.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116459.log.gz 2023-07-18T03:41:50.987 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107995.log 2023-07-18T03:41:50.988 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.106948.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106948.log.gz 2023-07-18T03:41:50.988 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107140.log 2023-07-18T03:41:50.989 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107995.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107995.log.gz 2023-07-18T03:41:50.989 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118255.log 2023-07-18T03:41:50.990 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107140.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107140.log.gz 2023-07-18T03:41:50.990 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121567.log 2023-07-18T03:41:50.991 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118255.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118255.log.gz 2023-07-18T03:41:50.992 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111353.log 2023-07-18T03:41:50.992 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121567.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.121567.log.gz 2023-07-18T03:41:50.993 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113105.log 2023-07-18T03:41:50.994 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111353.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111353.log.gz 2023-07-18T03:41:50.994 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122294.log 2023-07-18T03:41:50.995 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113105.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113105.log.gz 2023-07-18T03:41:50.996 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115242.log 2023-07-18T03:41:50.996 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122294.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.122294.log.gz 2023-07-18T03:41:50.997 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117601.log 2023-07-18T03:41:50.998 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115242.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115242.log.gz 2023-07-18T03:41:50.998 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114816.log 2023-07-18T03:41:50.999 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117601.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117601.log.gz 2023-07-18T03:41:51.000 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-07-18T03:41:51.000 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114816.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114816.log.gz 2023-07-18T03:41:51.001 INFO:teuthology.orchestra.run.smithi107.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-07-18T03:41:51.002 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115196.log 2023-07-18T03:41:51.002 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107987.log 2023-07-18T03:41:51.003 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115196.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115196.log.gz 2023-07-18T03:41:51.004 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117988.log 2023-07-18T03:41:51.005 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107987.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107987.log.gz 2023-07-18T03:41:51.005 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114088.log 2023-07-18T03:41:51.006 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117988.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117988.log.gz 2023-07-18T03:41:51.007 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107512.log 2023-07-18T03:41:51.008 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114088.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114088.log.gz 2023-07-18T03:41:51.008 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102605.log 2023-07-18T03:41:51.010 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.116168.log 2023-07-18T03:41:51.010 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107512.log.gz 2023-07-18T03:41:51.011 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.102605.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102605.log.gz 2023-07-18T03:41:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121134.log 2023-07-18T03:41:51.012 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.0.116168.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.116168.log.gz 2023-07-18T03:41:51.013 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108706.log 2023-07-18T03:41:51.014 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121134.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121134.log.gz 2023-07-18T03:41:51.014 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110585.log 2023-07-18T03:41:51.015 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108706.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108706.log.gz 2023-07-18T03:41:51.016 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108360.log 2023-07-18T03:41:51.017 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.110585.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.110585.log.gz 2023-07-18T03:41:51.018 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117509.log 2023-07-18T03:41:51.018 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108360.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108360.log.gz 2023-07-18T03:41:51.019 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122394.log 2023-07-18T03:41:51.020 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117509.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117509.log.gz 2023-07-18T03:41:51.021 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114954.log 2023-07-18T03:41:51.022 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122394.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.122394.log.gz 2023-07-18T03:41:51.022 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113950.log 2023-07-18T03:41:51.023 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114954.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114954.log.gz 2023-07-18T03:41:51.024 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121880.log 2023-07-18T03:41:51.025 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113950.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.113950.log.gz 2023-07-18T03:41:51.025 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117114.log 2023-07-18T03:41:51.027 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121880.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121880.log.gz 2023-07-18T03:41:51.027 INFO:teuthology.orchestra.run.smithi107.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-07-18T03:41:51.028 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117168.log 2023-07-18T03:41:51.028 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117114.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117114.log.gz 2023-07-18T03:41:51.029 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122578.log 2023-07-18T03:41:51.030 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117168.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117168.log.gz 2023-07-18T03:41:51.030 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117555.log 2023-07-18T03:41:51.031 INFO:teuthology.orchestra.run.smithi107.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-07-18T03:41:51.031 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122578.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.122578.log.gz 2023-07-18T03:41:51.032 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118661.log 2023-07-18T03:41:51.033 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121659.log 2023-07-18T03:41:51.033 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117555.log.gz 2023-07-18T03:41:51.034 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118661.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.118661.log.gz 2023-07-18T03:41:51.035 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109030.log 2023-07-18T03:41:51.036 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113766.log 2023-07-18T03:41:51.036 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121659.log.gz 2023-07-18T03:41:51.037 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109030.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109030.log.gz 2023-07-18T03:41:51.038 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112818.log 2023-07-18T03:41:51.038 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113766.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113766.log.gz 2023-07-18T03:41:51.039 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.96014.log 2023-07-18T03:41:51.040 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112818.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112818.log.gz 2023-07-18T03:41:51.041 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107558.log 2023-07-18T03:41:51.042 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph.tmp-client.admin.96014.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.96014.log.gz 2023-07-18T03:41:51.042 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117942.log 2023-07-18T03:41:51.043 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107558.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107558.log.gz 2023-07-18T03:41:51.044 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107742.log 2023-07-18T03:41:51.045 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117942.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117942.log.gz 2023-07-18T03:41:51.045 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117068.log 2023-07-18T03:41:51.046 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107742.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107742.log.gz 2023-07-18T03:41:51.047 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109491.log 2023-07-18T03:41:51.048 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109169.log 2023-07-18T03:41:51.049 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117068.log.gz 2023-07-18T03:41:51.050 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109491.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109491.log.gz 2023-07-18T03:41:51.050 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112592.log 2023-07-18T03:41:51.051 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109169.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109169.log.gz 2023-07-18T03:41:51.052 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112910.log 2023-07-18T03:41:51.052 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112592.log: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-07-18T03:41:51.053 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112592.log.gz 2023-07-18T03:41:51.053 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119775.log 2023-07-18T03:41:51.054 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112910.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112910.log.gz 2023-07-18T03:41:51.055 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112446.log 2023-07-18T03:41:51.055 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119775.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119775.log.gz 2023-07-18T03:41:51.056 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-07-18T03:41:51.056 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112446.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112446.log.gz 2023-07-18T03:41:51.057 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107328.log 2023-07-18T03:41:51.058 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107604.log 2023-07-18T03:41:51.059 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107328.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107328.log.gz 2023-07-18T03:41:51.059 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108718.log 2023-07-18T03:41:51.060 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107604.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107604.log.gz 2023-07-18T03:41:51.061 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106627.log 2023-07-18T03:41:51.062 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108718.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108718.log.gz 2023-07-18T03:41:51.063 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116367.log 2023-07-18T03:41:51.063 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.106627.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106627.log.gz 2023-07-18T03:41:51.064 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122532.log 2023-07-18T03:41:51.065 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111675.log 2023-07-18T03:41:51.066 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116367.log.gz 2023-07-18T03:41:51.067 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122532.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122532.log.gz 2023-07-18T03:41:51.067 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122762.log 2023-07-18T03:41:51.068 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111675.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111675.log.gz 2023-07-18T03:41:51.069 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121272.log 2023-07-18T03:41:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122762.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.122762.log.gz 2023-07-18T03:41:51.070 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109875.log 2023-07-18T03:41:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121272.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121272.log.gz 2023-07-18T03:41:51.071 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120362.log 2023-07-18T03:41:51.073 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109875.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109875.log.gz 2023-07-18T03:41:51.073 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115142.log 2023-07-18T03:41:51.074 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120362.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120362.log.gz 2023-07-18T03:41:51.075 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112684.log 2023-07-18T03:41:51.076 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115142.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115142.log.gz 2023-07-18T03:41:51.076 INFO:teuthology.orchestra.run.smithi107.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-07-18T03:41:51.077 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119959.log 2023-07-18T03:41:51.077 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112684.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112684.log.gz 2023-07-18T03:41:51.078 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122064.log 2023-07-18T03:41:51.079 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119959.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119959.log.gz 2023-07-18T03:41:51.080 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120454.log 2023-07-18T03:41:51.080 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122064.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122064.log.gz 2023-07-18T03:41:51.081 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109399.log 2023-07-18T03:41:51.082 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118475.log 2023-07-18T03:41:51.083 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120454.log.gz 2023-07-18T03:41:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109399.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109399.log.gz 2023-07-18T03:41:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122440.log 2023-07-18T03:41:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118475.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118475.log.gz 2023-07-18T03:41:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114632.log 2023-07-18T03:41:51.087 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122440.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.122440.log.gz 2023-07-18T03:41:51.087 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122110.log 2023-07-18T03:41:51.088 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114632.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114632.log.gz 2023-07-18T03:41:51.089 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115935.log 2023-07-18T03:41:51.090 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115451.log 2023-07-18T03:41:51.090 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122110.log.gz 2023-07-18T03:41:51.091 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115935.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115935.log.gz 2023-07-18T03:41:51.091 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119683.log 2023-07-18T03:41:51.093 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115451.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115451.log.gz 2023-07-18T03:41:51.093 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-18T03:41:51.094 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119683.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119683.log.gz 2023-07-18T03:41:51.094 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122966.log 2023-07-18T03:41:51.096 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121613.log 2023-07-18T03:41:51.097 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122966.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-18T03:41:51.098 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.122966.log.gz 2023-07-18T03:41:51.098 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121474.log 2023-07-18T03:41:51.099 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108715.log 2023-07-18T03:41:51.099 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121613.log.gz 2023-07-18T03:41:51.100 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121474.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121474.log.gz 2023-07-18T03:41:51.101 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118757.log 2023-07-18T03:41:51.102 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108715.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108715.log.gz 2023-07-18T03:41:51.102 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117371.log 2023-07-18T03:41:51.103 INFO:teuthology.orchestra.run.smithi107.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-07-18T03:41:51.104 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120588.log 2023-07-18T03:41:51.104 INFO:teuthology.orchestra.run.smithi049.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118757.log.gz 2023-07-18T03:41:51.105 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117371.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117371.log.gz 2023-07-18T03:41:51.105 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120839.log 2023-07-18T03:41:51.106 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120588.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120588.log.gz 2023-07-18T03:41:51.107 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102296.log 2023-07-18T03:41:51.108 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120839.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120839.log.gz 2023-07-18T03:41:51.108 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112273.log 2023-07-18T03:41:51.110 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.102296.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.102296.log.gz 2023-07-18T03:41:51.110 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117214.log 2023-07-18T03:41:51.111 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112273.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112273.log.gz 2023-07-18T03:41:51.111 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115336.log 2023-07-18T03:41:51.113 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108356.log 2023-07-18T03:41:51.113 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117214.log.gz 2023-07-18T03:41:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115336.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115336.log.gz 2023-07-18T03:41:51.114 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122716.log 2023-07-18T03:41:51.116 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108356.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108356.log.gz 2023-07-18T03:41:51.117 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114272.log 2023-07-18T03:41:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122716.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122716.log.gz 2023-07-18T03:41:51.118 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108358.log 2023-07-18T03:41:51.120 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117739.log 2023-07-18T03:41:51.120 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114272.log.gz 2023-07-18T03:41:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108358.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108358.log.gz 2023-07-18T03:41:51.121 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107186.log 2023-07-18T03:41:51.122 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117739.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117739.log.gz 2023-07-18T03:41:51.123 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115819.log 2023-07-18T03:41:51.124 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107186.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107186.log.gz 2023-07-18T03:41:51.124 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107420.log 2023-07-18T03:41:51.125 INFO:teuthology.orchestra.run.smithi107.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-07-18T03:41:51.125 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115819.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115819.log.gz 2023-07-18T03:41:51.126 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116796.log 2023-07-18T03:41:51.127 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111445.log 2023-07-18T03:41:51.128 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107420.log.gz 2023-07-18T03:41:51.128 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116796.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.116796.log.gz 2023-07-18T03:41:51.129 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121521.log 2023-07-18T03:41:51.129 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114586.log 2023-07-18T03:41:51.130 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111445.log.gz 2023-07-18T03:41:51.131 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121521.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121521.log.gz 2023-07-18T03:41:51.131 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111905.log 2023-07-18T03:41:51.132 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112181.log 2023-07-18T03:41:51.133 INFO:teuthology.orchestra.run.smithi049.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114586.log.gz 2023-07-18T03:41:51.133 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111905.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111905.log.gz 2023-07-18T03:41:51.134 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109261.log 2023-07-18T03:41:51.135 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122018.log 2023-07-18T03:41:51.135 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112181.log.gz 2023-07-18T03:41:51.136 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109261.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109261.log.gz 2023-07-18T03:41:51.136 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122486.log 2023-07-18T03:41:51.137 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115543.log 2023-07-18T03:41:51.138 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122018.log.gz 2023-07-18T03:41:51.139 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122486.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.122486.log.gz 2023-07-18T03:41:51.139 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113720.log 2023-07-18T03:41:51.140 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113335.log 2023-07-18T03:41:51.140 INFO:teuthology.orchestra.run.smithi049.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115543.log.gz 2023-07-18T03:41:51.141 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113720.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113720.log.gz 2023-07-18T03:41:51.142 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114134.log 2023-07-18T03:41:51.143 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111767.log 2023-07-18T03:41:51.143 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113335.log.gz 2023-07-18T03:41:51.144 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114134.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114134.log.gz 2023-07-18T03:41:51.144 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106717.log 2023-07-18T03:41:51.145 INFO:teuthology.orchestra.run.smithi107.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-07-18T03:41:51.146 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121042.log 2023-07-18T03:41:51.146 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111767.log.gz 2023-07-18T03:41:51.146 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.106717.log: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-07-18T03:41:51.147 INFO:teuthology.orchestra.run.smithi049.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106717.log.gz 2023-07-18T03:41:51.147 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108628.log 2023-07-18T03:41:51.148 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118709.log 2023-07-18T03:41:51.149 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121042.log.gz 2023-07-18T03:41:51.149 INFO:teuthology.orchestra.run.smithi107.stderr: 2023-07-18T03:41:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m0.288s 2023-07-18T03:41:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m1.135s 2023-07-18T03:41:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m0.195s 2023-07-18T03:41:51.151 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108628.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108628.log.gz 2023-07-18T03:41:51.151 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119043.log 2023-07-18T03:41:51.151 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118147.log 2023-07-18T03:41:51.152 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118709.log.gz 2023-07-18T03:41:51.152 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119043.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119043.log.gz 2023-07-18T03:41:51.153 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116888.log 2023-07-18T03:41:51.153 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118147.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118147.log.gz 2023-07-18T03:41:51.154 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109583.log 2023-07-18T03:41:51.154 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116888.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116888.log.gz 2023-07-18T03:41:51.155 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122670.log 2023-07-18T03:41:51.156 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109583.log: gzip -5 88.1%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.109583.log.gz/var/log/ceph/ceph-osd.1.log 2023-07-18T03:41:51.156 INFO:teuthology.orchestra.run.smithi049.stderr: 2023-07-18T03:41:51.157 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122670.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122670.log.gz 2023-07-18T03:41:51.157 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118997.log 2023-07-18T03:41:51.158 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111399.log 2023-07-18T03:41:51.159 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118997.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118997.log.gz 2023-07-18T03:41:51.160 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.111037.log 2023-07-18T03:41:51.161 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.110352.log 2023-07-18T03:41:51.161 INFO:teuthology.orchestra.run.smithi049.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111399.log.gz 2023-07-18T03:41:51.162 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.2.111037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.111037.log.gz 2023-07-18T03:41:51.163 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109353.log 2023-07-18T03:41:51.164 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.0.110352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.110352.log.gz 2023-07-18T03:41:51.164 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120073.log 2023-07-18T03:41:51.165 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109353.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109353.log.gz 2023-07-18T03:41:51.165 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113674.log 2023-07-18T03:41:51.166 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107094.log 2023-07-18T03:41:51.167 INFO:teuthology.orchestra.run.smithi049.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120073.log.gz 2023-07-18T03:41:51.167 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113674.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113674.log.gz 2023-07-18T03:41:51.168 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118857.log 2023-07-18T03:41:51.169 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121410.log 2023-07-18T03:41:51.169 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107094.log.gz 2023-07-18T03:41:51.170 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118857.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118857.log.gz 2023-07-18T03:41:51.171 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114042.log 2023-07-18T03:41:51.172 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121926.log 2023-07-18T03:41:51.172 INFO:teuthology.orchestra.run.smithi049.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.121410.log.gz 2023-07-18T03:41:51.173 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114042.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114042.log.gz 2023-07-18T03:41:51.173 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109215.log 2023-07-18T03:41:51.174 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115094.log 2023-07-18T03:41:51.175 INFO:teuthology.orchestra.run.smithi049.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.121926.log.gz 2023-07-18T03:41:51.175 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109215.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109215.log.gz 2023-07-18T03:41:51.176 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120904.log 2023-07-18T03:41:51.177 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111491.log 2023-07-18T03:41:51.177 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115094.log.gz 2023-07-18T03:41:51.178 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120904.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120904.log.gz 2023-07-18T03:41:51.179 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109721.log 2023-07-18T03:41:51.180 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113904.log 2023-07-18T03:41:51.180 INFO:teuthology.orchestra.run.smithi049.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111491.log.gz 2023-07-18T03:41:51.181 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109721.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.109721.log.gz 2023-07-18T03:41:51.182 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114724.log 2023-07-18T03:41:51.182 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118615.log 2023-07-18T03:41:51.183 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113904.log.gz 2023-07-18T03:41:51.183 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114724.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114724.log.gz 2023-07-18T03:41:51.184 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117785.log 2023-07-18T03:41:51.185 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112227.log 2023-07-18T03:41:51.186 INFO:teuthology.orchestra.run.smithi049.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118615.log.gz 2023-07-18T03:41:51.186 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117785.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117785.log.gz 2023-07-18T03:41:51.187 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119273.log 2023-07-18T03:41:51.188 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111629.log 2023-07-18T03:41:51.189 INFO:teuthology.orchestra.run.smithi049.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-07-18T03:41:51.189 INFO:teuthology.orchestra.run.smithi049.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112227.log.gz 2023-07-18T03:41:51.189 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116976.log 2023-07-18T03:41:51.190 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.119273.log.gz 2023-07-18T03:41:51.190 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111629.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111629.log.gz 2023-07-18T03:41:51.192 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113858.log 2023-07-18T03:41:51.193 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116976.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116976.log.gz 2023-07-18T03:41:51.193 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117463.log 2023-07-18T03:41:51.194 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113858.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113858.log.gz 2023-07-18T03:41:51.194 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121364.log 2023-07-18T03:41:51.195 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117463.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117463.log.gz 2023-07-18T03:41:51.195 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118905.log 2023-07-18T03:41:51.196 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121364.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.121364.log.gz 2023-07-18T03:41:51.197 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115727.log 2023-07-18T03:41:51.197 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118905.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118905.log.gz 2023-07-18T03:41:51.198 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107466.log 2023-07-18T03:41:51.199 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115727.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.115727.log.gz-5 2023-07-18T03:41:51.199 INFO:teuthology.orchestra.run.smithi049.stderr:--verbose -- /var/log/ceph/ceph-client.admin.109537.log 2023-07-18T03:41:51.200 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107466.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107466.log.gz 2023-07-18T03:41:51.200 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113628.log 2023-07-18T03:41:51.201 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109537.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109537.log.gz 2023-07-18T03:41:51.201 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111260.log 2023-07-18T03:41:51.202 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113628.log: 89.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.113628.log.gz--verbose 2023-07-18T03:41:51.202 INFO:teuthology.orchestra.run.smithi049.stderr:-- /var/log/ceph/ceph-client.admin.109076.log 2023-07-18T03:41:51.203 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111260.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.111260.log.gz 2023-07-18T03:41:51.204 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107048.log 2023-07-18T03:41:51.205 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109076.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109076.log.gz 2023-07-18T03:41:51.205 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119135.log 2023-07-18T03:41:51.206 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107048.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107048.log.gz 2023-07-18T03:41:51.206 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113582.log 2023-07-18T03:41:51.207 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119135.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119135.log.gz 2023-07-18T03:41:51.207 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112135.log 2023-07-18T03:41:51.208 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113582.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113582.log.gz 2023-07-18T03:41:51.208 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119089.log 2023-07-18T03:41:51.209 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112135.log: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-07-18T03:41:51.210 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112135.log.gzgzip 2023-07-18T03:41:51.210 INFO:teuthology.orchestra.run.smithi049.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.108711.log 2023-07-18T03:41:51.210 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119089.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119089.log.gz 2023-07-18T03:41:51.211 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115497.log 2023-07-18T03:41:51.212 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108711.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108711.log.gz 2023-07-18T03:41:51.212 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118951.log 2023-07-18T03:41:51.213 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115497.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115497.log.gz 2023-07-18T03:41:51.213 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114226.log 2023-07-18T03:41:51.214 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118951.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.118951.log.gz 2023-07-18T03:41:51.214 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118429.log 2023-07-18T03:41:51.215 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114226.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114226.log.gz 2023-07-18T03:41:51.216 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121226.log 2023-07-18T03:41:51.217 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118429.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118429.log.gz 2023-07-18T03:41:51.217 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112772.log 2023-07-18T03:41:51.217 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121226.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121226.log.gz 2023-07-18T03:41:51.218 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120500.log 2023-07-18T03:41:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112772.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112772.log.gz 2023-07-18T03:41:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113243.log 2023-07-18T03:41:51.220 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120500.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120500.log.gz 2023-07-18T03:41:51.220 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112319.log 2023-07-18T03:41:51.221 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113243.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113243.log.gz 2023-07-18T03:41:51.222 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115865.log 2023-07-18T03:41:51.222 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112319.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112319.log.gz 2023-07-18T03:41:51.223 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120793.log 2023-07-18T03:41:51.224 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.115865.log.gz 2023-07-18T03:41:51.225 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112043.log 2023-07-18T03:41:51.226 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120793.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120793.log.gz 2023-07-18T03:41:51.226 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118811.log 2023-07-18T03:41:51.227 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112043.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112043.log.gz 2023-07-18T03:41:51.227 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119343.log 2023-07-18T03:41:51.228 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118811.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118811.log.gz 2023-07-18T03:41:51.228 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107973.log 2023-07-18T03:41:51.229 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119343.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119343.log.gz 2023-07-18T03:41:51.229 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115384.log 2023-07-18T03:41:51.230 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107973.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107973.log.gz 2023-07-18T03:41:51.231 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114770.log 2023-07-18T03:41:51.231 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115384.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115384.log.gz 2023-07-18T03:41:51.232 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115290.log 2023-07-18T03:41:51.233 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114770.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114770.log.gz 2023-07-18T03:41:51.233 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107788.log 2023-07-18T03:41:51.234 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115290.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115290.log.gz 2023-07-18T03:41:51.234 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102731.log 2023-07-18T03:41:51.235 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107788.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107788.log.gz 2023-07-18T03:41:51.235 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112956.log 2023-07-18T03:41:51.236 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.102731.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102731.log.gz 2023-07-18T03:41:51.237 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122248.log 2023-07-18T03:41:51.238 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112956.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112956.log.gz 2023-07-18T03:41:51.238 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120634.log 2023-07-18T03:41:51.239 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122248.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122248.log.gz 2023-07-18T03:41:51.239 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117325.log 2023-07-18T03:41:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120634.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120634.log.gz 2023-07-18T03:41:51.240 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118101.log 2023-07-18T03:41:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117325.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117325.log.gz 2023-07-18T03:41:51.241 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117417.log 2023-07-18T03:41:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118101.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118101.log.gz 2023-07-18T03:41:51.242 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107002.log 2023-07-18T03:41:51.243 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117417.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117417.log.gz 2023-07-18T03:41:51.244 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114180.log 2023-07-18T03:41:51.244 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107002.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107002.log.gz 2023-07-18T03:41:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119867.log 2023-07-18T03:41:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114180.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114180.log.gz 2023-07-18T03:41:51.246 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108355.log 2023-07-18T03:41:51.246 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119867.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.119867.log.gz 2023-07-18T03:41:51.247 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120005.log 2023-07-18T03:41:51.247 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108355.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108355.log.gz 2023-07-18T03:41:51.248 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.119576.log 2023-07-18T03:41:51.249 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.120005.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120005.log.gz 2023-07-18T03:41:51.249 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116704.log 2023-07-18T03:41:51.249 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.0.119576.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.119576.log.gz 2023-07-18T03:41:51.250 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112546.log 2023-07-18T03:41:51.251 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.116704.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116704.log.gz 2023-07-18T03:41:51.251 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117849.log 2023-07-18T03:41:51.252 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.112546.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112546.log.gz 2023-07-18T03:41:51.252 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114364.log 2023-07-18T03:41:51.253 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117849.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117849.log.gz 2023-07-18T03:41:51.253 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107234.log 2023-07-18T03:41:51.254 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114364.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114364.log.gz 2023-07-18T03:41:51.254 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118523.log 2023-07-18T03:41:51.255 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107234.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.107234.log.gz 2023-07-18T03:41:51.255 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113289.log 2023-07-18T03:41:51.256 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118523.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118523.log.gz 2023-07-18T03:41:51.256 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-07-18T03:41:51.257 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113289.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113289.log.gz 2023-07-18T03:41:51.257 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108730.log 2023-07-18T03:41:51.258 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114540.log 2023-07-18T03:41:51.259 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108730.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108730.log.gz 2023-07-18T03:41:51.260 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113469.log 2023-07-18T03:41:51.261 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114540.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114540.log.gz 2023-07-18T03:41:51.261 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117693.log 2023-07-18T03:41:51.262 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.113469.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113469.log.gz 2023-07-18T03:41:51.262 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107374.log 2023-07-18T03:41:51.263 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117693.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117693.log.gz 2023-07-18T03:41:51.264 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118055.log 2023-07-18T03:41:51.264 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.107374.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107374.log.gz 2023-07-18T03:41:51.265 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109445.log 2023-07-18T03:41:51.266 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.118055.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118055.log.gz 2023-07-18T03:41:51.266 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106819.log 2023-07-18T03:41:51.267 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.109445.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109445.log.gz 2023-07-18T03:41:51.267 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111583.log 2023-07-18T03:41:51.268 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.106819.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106819.log.gz 2023-07-18T03:41:51.269 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117647.log 2023-07-18T03:41:51.269 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.111583.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111583.log.gz 2023-07-18T03:41:51.270 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-07-18T03:41:51.271 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.117647.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117647.log.gz 2023-07-18T03:41:51.271 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122876.log 2023-07-18T03:41:51.272 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115589.log 2023-07-18T03:41:51.273 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.122876.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122876.log.gz 2023-07-18T03:41:51.274 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119913.log 2023-07-18T03:41:51.275 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.115589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114908.log 2023-07-18T03:41:51.275 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115589.log.gz 2023-07-18T03:41:51.276 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.119913.log: 93.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-07-18T03:41:51.276 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119913.log.gz 2023-07-18T03:41:51.276 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108345.log 2023-07-18T03:41:51.277 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.114908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121318.log 2023-07-18T03:41:51.278 INFO:teuthology.orchestra.run.smithi049.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.114908.log.gz 2023-07-18T03:41:51.279 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.108345.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108345.log.gz 2023-07-18T03:41:51.280 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.121318.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121318.log.gz 2023-07-18T03:41:51.325 INFO:teuthology.orchestra.run.smithi049.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-07-18T03:41:51.510 INFO:teuthology.orchestra.run.smithi049.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-07-18T03:41:51.513 INFO:teuthology.orchestra.run.smithi049.stderr: 2023-07-18T03:41:51.513 INFO:teuthology.orchestra.run.smithi049.stderr:real 0m0.656s 2023-07-18T03:41:51.513 INFO:teuthology.orchestra.run.smithi049.stderr:user 0m1.113s 2023-07-18T03:41:51.513 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m0.850s 2023-07-18T03:41:51.514 INFO:tasks.ceph:Archiving logs... 2023-07-18T03:41:51.515 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747/remote/smithi049/log 2023-07-18T03:41:51.516 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-18T03:41:51.942 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747/remote/smithi107/log 2023-07-18T03:41:51.943 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-18T03:41:52.138 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-18T03:41:52.156 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-18T03:41:52.157 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-18T03:41:52.162 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-18T03:41:52.291 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-18T03:41:52.301 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-18T03:41:52.301 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:41:52.306 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:41:52.344 INFO:teuthology.orchestra.run.smithi049.stderr:bash: ntpq: command not found 2023-07-18T03:41:52.347 INFO:teuthology.orchestra.run.smithi107.stderr:bash: ntpq: command not found 2023-07-18T03:41:52.352 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:41:52.352 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2023-07-18T03:41:52.353 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 +2798us[+2796us] +/- 85ms 2023-07-18T03:41:52.353 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -4369us[-4369us] +/- 78ms 2023-07-18T03:41:52.353 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv03.front.sepia.ceph.com 3 6 377 27 +1784us[+1782us] +/- 70ms 2023-07-18T03:41:52.353 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:41:52.354 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:41:52.355 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2023-07-18T03:41:52.355 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 +2670us[+2670us] +/- 85ms 2023-07-18T03:41:52.355 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -4457us[-4457us] +/- 78ms 2023-07-18T03:41:52.355 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv03.front.sepia.ceph.com 3 6 377 27 +1680us[+1667us] +/- 70ms 2023-07-18T03:41:52.356 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:41:52.357 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-18T03:41:52.367 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-18T03:41:52.368 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-18T03:41:52.379 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-18T03:41:52.574 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-18T03:41:52.738 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:41:52.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:41:52.817 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:41:52.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:41:52.893 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-18T03:41:52.905 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-18T03:41:52.916 INFO:teuthology.task.internal:Duration was 1400.611680 seconds 2023-07-18T03:41:52.916 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-18T03:41:52.926 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-18T03:41:52.927 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-18T03:41:52.931 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-18T03:41:53.048 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:41:53.049 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:41:53.257 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-18T03:41:53.257 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2023-07-18T03:41:53.258 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-18T03:41:53.319 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2023-07-18T03:41:53.320 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-18T03:41:53.378 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-18T03:41:53.379 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-18T03:41:53.384 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-18T03:41:53.469 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-18T03:41:53.469 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-18T03:41:53.474 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-18T03:41:53.603 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-18T03:41:53.614 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-18T03:41:53.615 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-18T03:41:53.685 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-18T03:41:53.754 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-18T03:41:53.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:41:53.770 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:41:53.835 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2023-07-18T03:41:53.838 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2023-07-18T03:41:53.874 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:41:53.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:41:53.916 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:41:53.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:41:53.957 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-18T03:41:53.969 INFO:teuthology.task.internal:Transferring archived files... 2023-07-18T03:41:53.970 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747/remote/smithi049 2023-07-18T03:41:53.970 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-18T03:41:54.050 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747/remote/smithi107 2023-07-18T03:41:54.050 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-18T03:41:54.131 INFO:teuthology.task.internal:Removing archive directory... 2023-07-18T03:41:54.131 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-18T03:41:54.136 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-18T03:41:54.179 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-18T03:41:54.192 INFO:teuthology.task.internal:Not uploading archives. 2023-07-18T03:41:54.193 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-18T03:41:54.207 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-18T03:41:54.208 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-18T03:41:54.220 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-18T03:41:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: 524879 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 18 03:41 /home/ubuntu/cephtest 2023-07-18T03:41:54.263 INFO:teuthology.orchestra.run.smithi107.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 18 03:41 /home/ubuntu/cephtest 2023-07-18T03:41:54.267 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-18T03:41:54.279 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-18T03:41:54.300 INFO:teuthology.nuke:Checking targets against current locks 2023-07-18T03:41:54.331 DEBUG:teuthology.nuke:shortname: smithi049 2023-07-18T03:41:54.332 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T03:41:54.357 DEBUG:teuthology.nuke:shortname: smithi107 2023-07-18T03:41:54.357 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T03:41:54.360 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 03:03:08.163847', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-18T03:41:54.380 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342747', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 03:03:08.162157', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-18T03:41:54.413 INFO:teuthology.orchestra.console:Power off smithi049 2023-07-18T03:41:54.414 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-18T03:41:54.441 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:41:54.461 INFO:teuthology.orchestra.console:Power off smithi107 2023-07-18T03:41:54.462 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-18T03:41:54.488 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:41:58.483 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:41:58.511 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:42:02.600 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:42:02.629 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:42:02.816 INFO:teuthology.orchestra.console:Power off for smithi049 completed 2023-07-18T03:42:06.744 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:42:06.958 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2023-07-18T03:42:07.064 INFO:teuthology.run:Summary data: client.0-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 client.1-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} duration: 1400.6116797924042 failure_reason: 'Test failure: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=9fd979daa84f4190bd86dceeab7afd6e status: fail success: false 2023-07-18T03:42:07.065 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T03:42:07.144 INFO:teuthology.run:FAIL